Pipelines show 2 times the number of tests reported in junit.xml

XMLWordPrintable

      If you have a test suite that contains multiple (embedded) tests suites, Bitbucket Pipelines will count the tests executed and failed twice. See for example https://bitbucket.org/graphicresources/tutpad_web/addon/pipelines/home#!/results/1706

      The xml is something like this:

      #!xml
      
      <?xml version="1.0" encoding="UTF-8"?>
      <testsuites>
          <testsuite name="CodeIgniter Application Test Suite" tests="260" assertions="1110" failures="0" errors="9" time="51.481371">
              <testsuite name="Admin_course_test" file="/code/application/tests/unit/controllers/Admin_course_test.php" tests="7" assertions="45" failures="0" errors="3" time="3.380968">
                  <testcase name="test_admin_create_course" class="Admin_course_test" file="/code/application/tests/unit/controllers/Admin_course_test.php" line="52" assertions="6" time="1.127683"/>
                  [...]
            </testsuite>
          </testsuite>
      </testsuites>
      
      

      PHPUnit creates a global testsuite (named "CodeIgniter Application Test Suite" in our example) and puts all the tests suites inside this tag.

      Also, for some reason, if I remove this external testsuite, leaving a flat list of testsuites, Bitbucket Pipelines does not parse my junit.xml, see for example this run:

      https://bitbucket.org/graphicresources/tutpad_web/addon/pipelines/home#!/results/1707

      The junit.xml is the same as before, except in this run I deleted the third line and the penultimate line with sed -i '3d;$!x' junit.xml. Pipelines did not found any tests in this xml, but it displayed no errors. I verified this file with verify-junit-xml (npm module) and it seems it does not have any errors.

            Assignee:
            Unassigned
            Reporter:
            e98cuenc
            Votes:
            5 Vote for this issue
            Watchers:
            9 Start watching this issue

              Created:
              Updated:
              Resolved: