-
Type:
Bug
-
Resolution: Fixed
-
Priority:
Medium
-
Component/s: Pipelines - Run Failures
If you have a test suite that contains multiple (embedded) tests suites, Bitbucket Pipelines will count the tests executed and failed twice. See for example https://bitbucket.org/graphicresources/tutpad_web/addon/pipelines/home#!/results/1706
The xml is something like this:
#!xml <?xml version="1.0" encoding="UTF-8"?> <testsuites> <testsuite name="CodeIgniter Application Test Suite" tests="260" assertions="1110" failures="0" errors="9" time="51.481371"> <testsuite name="Admin_course_test" file="/code/application/tests/unit/controllers/Admin_course_test.php" tests="7" assertions="45" failures="0" errors="3" time="3.380968"> <testcase name="test_admin_create_course" class="Admin_course_test" file="/code/application/tests/unit/controllers/Admin_course_test.php" line="52" assertions="6" time="1.127683"/> [...] </testsuite> </testsuite> </testsuites>
PHPUnit creates a global testsuite (named "CodeIgniter Application Test Suite" in our example) and puts all the tests suites inside this tag.
Also, for some reason, if I remove this external testsuite, leaving a flat list of testsuites, Bitbucket Pipelines does not parse my junit.xml, see for example this run:
https://bitbucket.org/graphicresources/tutpad_web/addon/pipelines/home#!/results/1707
The junit.xml is the same as before, except in this run I deleted the third line and the penultimate line with sed -i '3d;$!x' junit.xml. Pipelines did not found any tests in this xml, but it displayed no errors. I verified this file with verify-junit-xml (npm module) and it seems it does not have any errors.