• 1
    • 1
    • Our product teams collect and evaluate feedback from a number of different sources. To learn more about how we use customer feedback in the planning process, check out our new feature policy.

      Would be cool if quarantined tests in 'master' plan would also result in being quarantined in child plan-branches. For example we have those two SAL tests failing on tardigrade now, and if I work on feature-branches and use "gatekeeper" in Bamboo for them - I have to manually quarantine those two SAL tests in my BambooBranch too. Would be cool if I didn't have and the Bamboo would guess that those two SAL tests can fail and Bamboo can push to master (gatekeeper) if no other tests fail.

            [BAM-11196] Quarantine tests across branches

            PiotrA added a comment -

            Hi Nicholas,

            I think you have already figured this out (as you commented on BAM-12968), but just we're standing on the same ground: the expected behavior of Bamboo is to "(B) - start branches with the set of quarantines from the "default" branch". Unfortunately, it seems that there is some bug in Bamboo (BAM-12968) that prevents a few quarantined tests to be properly cloned to a newly created plan-branch. That's why you experience the list that doesn't match any other branch of the plan, nor "default".

            PiotrA added a comment - Hi Nicholas, I think you have already figured this out (as you commented on BAM-12968 ), but just we're standing on the same ground: the expected behavior of Bamboo is to "(B) - start branches with the set of quarantines from the "default" branch" . Unfortunately, it seems that there is some bug in Bamboo ( BAM-12968 ) that prevents a few quarantined tests to be properly cloned to a newly created plan-branch. That's why you experience the list that doesn't match any other branch of the plan, nor "default".

            I am confused by the current implementation (4.4.3). After creating a branch, I have some set of quarantined tests that do not match any other branch of the plan. Where did this set come from?

            It would be nice to be able to "copy" the list of quarantined tests from one branch to another. Then, maybe, branches should start with either: (A) an empty set of quarantines or (B) the set from the "default" branch.

            Nicholas Parrish added a comment - I am confused by the current implementation (4.4.3). After creating a branch, I have some set of quarantined tests that do not match any other branch of the plan. Where did this set come from? It would be nice to be able to "copy" the list of quarantined tests from one branch to another. Then, maybe, branches should start with either: (A) an empty set of quarantines or (B) the set from the "default" branch.

            Daz added a comment -

            I just got burned by not realising that there were 13 quarantined tests in a build I cloned for my new branch. I cloned against a SHA I saw was green in our selenium tests, made some changes, pushed my branch. I saw failures, assumed they were mine. A few minutes and brief sanity check later, I realised the only reason the main build was passing was because all the failures were quarantined. Heh.

            Daz added a comment - I just got burned by not realising that there were 13 quarantined tests in a build I cloned for my new branch. I cloned against a SHA I saw was green in our selenium tests, made some changes, pushed my branch. I saw failures, assumed they were mine. A few minutes and brief sanity check later, I realised the only reason the main build was passing was because all the failures were quarantined. Heh.

            I hope that when this is implemented, it won't simply be cloning quarantines at the time of creation of the branch build, but a "live" inheritance of quarantines from default to all branch builds. Since our branches frequently merge from master back to the topic branches, we need the branches to track the quarantined tests from master.

            Currently, it results in a lot of confusion when an intermittent test failure is quarantined in the master build but failing in topic branches. Branch authors waste time investigating failures that have nothing to do with their changes.

            Tim Moore [Atlassian] added a comment - I hope that when this is implemented, it won't simply be cloning quarantines at the time of creation of the branch build, but a "live" inheritance of quarantines from default to all branch builds. Since our branches frequently merge from master back to the topic branches, we need the branches to track the quarantined tests from master. Currently, it results in a lot of confusion when an intermittent test failure is quarantined in the master build but failing in topic branches. Branch authors waste time investigating failures that have nothing to do with their changes.

              Unassigned Unassigned
              pstefaniak PiotrA
              Votes:
              20 Vote for this issue
              Watchers:
              15 Start watching this issue

                Created:
                Updated: