Details
-
Bug
-
Resolution: Fixed
-
Medium
-
2.3 M4
-
None
Description
Case A
Consider a series of builds:
BUILD-1 Test X passed
BUILD-2 Test X failed
BUILD-3 Build failure, no test results
BUILD-4 Test X failed
BUILD-5 Test X failed
In the results of BUILD-4 and BUILD-5, Bamboo reports that test X has been failing since BUILD-4. From this, developers incorrectly infer that the test passed in BUILD-3 and the change that caused this failure was introduced in BUILD-4. In this case, it would be better to report that the test has been failing since BUILD-2.
Case B
There's another interesting case that needs to be considered:
BUILD-6 Test X passed
BUILD-7 Build failure, no test results
BUILD-8 Build failure, no test results
BUILD-9 Test X failed
Here, the failure of test X could have been introduced in BUILD-7, BUILD-8 or BUILD-9. In this case, perhaps it's best to report that Test X has been failing since BUILD-7?