Convenient default/upstream test failure comparison for Git based plans

XMLWordPrintable

    • 0

      Since quarantined tests don't delegate down branch plan configs, I often find myself doing the following click orgy:

      1. check the execution on my branch, and determine if I merged stuff in a bad state from upstream in or I caused the failure, e.g. https://confluence-bamboo.atlassian.com/browse/CONFFUNC-MAIN183-104/test
      2. in order to determine if the test was quarantined, I try to find the plan testing the commit of the merge-base on default, e.g.
        git merge-base canonical/master canonical/issue/pdl_development
        999e9474eadd2465e9502c03a83b6190fffc92c2
        

        https://confluence-bamboo.atlassian.com/browse/CONFFUNC-MAIN-BATCH7-13961/commit

      3. now I could assume that I produced the failure on the branch, or better check https://confluence-bamboo.atlassian.com/browse/CONFFUNC-MAIN-13961 to see that there are quarantined tests for that execution, so I go to https://confluence-bamboo.atlassian.com/browse/CONFFUNC-MAIN-13961/test
      4. now I could check the latest build on upstream to see if they are fixed now, so I go to https://confluence-bamboo.atlassian.com/browse/CONFFUNC-MAIN/latest
      5. now I want to know who quarantined legitimate test failures, but this information is not presented on https://confluence-bamboo.atlassian.com/browse/CONFFUNC-MAIN-13965/test/#quarantined-tests
      6. so I go back to the plan view, https://confluence-bamboo.atlassian.com/browse/CONFFUNC-MAIN and click on "Quarantined Tests" https://confluence-bamboo.atlassian.com/browse/CONFFUNC-MAIN/quarantine

      I think we can make this much more convenient. The optimal implementation would be sth. like:

      #When a branch config is executed, determine the merge-base with upstream/default and save it for this execution
      #When the execution is over and the test results are parsed, determine the corresponding execution on upstream/default and see if the test failed there too
      #Show the following information on the test result view for the branch execution

      • test failed for merge-base run on upstream/default
      • test is still failing on upstream/default
      • test is quarantined on upstream/default, who quarantined it and ask if you would like to quarantine

            Assignee:
            Unassigned
            Reporter:
            fabs (Inactive)
            Votes:
            1 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: