Uploaded image for project: 'Bamboo Data Center'
  1. Bamboo Data Center
  2. BAM-3905

"Test failing since" doesn't properly consider builds without test results

    XMLWordPrintable

Details

    Description

      Case A

      Consider a series of builds:

      BUILD-1 Test X passed
      BUILD-2 Test X failed
      BUILD-3 Build failure, no test results
      BUILD-4 Test X failed
      BUILD-5 Test X failed

      In the results of BUILD-4 and BUILD-5, Bamboo reports that test X has been failing since BUILD-4. From this, developers incorrectly infer that the test passed in BUILD-3 and the change that caused this failure was introduced in BUILD-4. In this case, it would be better to report that the test has been failing since BUILD-2.

      Case B

      There's another interesting case that needs to be considered:

      BUILD-6 Test X passed
      BUILD-7 Build failure, no test results
      BUILD-8 Build failure, no test results
      BUILD-9 Test X failed

      Here, the failure of test X could have been introduced in BUILD-7, BUILD-8 or BUILD-9. In this case, perhaps it's best to report that Test X has been failing since BUILD-7?

      Attachments

        Activity

          People

            jdumay James Dumay
            ahempel Adrian Hempel [Atlassian]
            Votes:
            2 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: