Uploaded image for project: 'Bamboo Data Center'
  1. Bamboo Data Center
  2. BAM-12493

Convenient default/upstream test failure comparison for Git based plans

XMLWordPrintable

    • 0
    • Our product teams collect and evaluate feedback from a number of different sources. To learn more about how we use customer feedback in the planning process, check out our new feature policy.

      Since quarantined tests don't delegate down branch plan configs, I often find myself doing the following click orgy:

      1. check the execution on my branch, and determine if I merged stuff in a bad state from upstream in or I caused the failure, e.g. https://confluence-bamboo.atlassian.com/browse/CONFFUNC-MAIN183-104/test
      2. in order to determine if the test was quarantined, I try to find the plan testing the commit of the merge-base on default, e.g.
        git merge-base canonical/master canonical/issue/pdl_development
        999e9474eadd2465e9502c03a83b6190fffc92c2
        

        https://confluence-bamboo.atlassian.com/browse/CONFFUNC-MAIN-BATCH7-13961/commit

      3. now I could assume that I produced the failure on the branch, or better check https://confluence-bamboo.atlassian.com/browse/CONFFUNC-MAIN-13961 to see that there are quarantined tests for that execution, so I go to https://confluence-bamboo.atlassian.com/browse/CONFFUNC-MAIN-13961/test
      4. now I could check the latest build on upstream to see if they are fixed now, so I go to https://confluence-bamboo.atlassian.com/browse/CONFFUNC-MAIN/latest
      5. now I want to know who quarantined legitimate test failures, but this information is not presented on https://confluence-bamboo.atlassian.com/browse/CONFFUNC-MAIN-13965/test/#quarantined-tests
      6. so I go back to the plan view, https://confluence-bamboo.atlassian.com/browse/CONFFUNC-MAIN and click on "Quarantined Tests" https://confluence-bamboo.atlassian.com/browse/CONFFUNC-MAIN/quarantine

      I think we can make this much more convenient. The optimal implementation would be sth. like:

      #When a branch config is executed, determine the merge-base with upstream/default and save it for this execution
      #When the execution is over and the test results are parsed, determine the corresponding execution on upstream/default and see if the test failed there too
      #Show the following information on the test result view for the branch execution

      • test failed for merge-base run on upstream/default
      • test is still failing on upstream/default
      • test is quarantined on upstream/default, who quarantined it and ask if you would like to quarantine

              Unassigned Unassigned
              fakraemer fabs
              Votes:
              1 Vote for this issue
              Watchers:
              3 Start watching this issue

                Created:
                Updated:
                Resolved: