Bring it!

Voted! 

Some automated UI end-to-end tests are just too unpredictable for red/green results.

Selenium's IE webdriver failed to create the "@" symbol and instead wrote 2... failing the test early despite it being a driver issue. 

@Joseph You can vote for this feature if you haven't done that already.

@Kumar, navigated to this issue for the same reason. Noticing browserstack failures (with screenshots showing the webdriver failed for unpredictable reasons).. would like to have the option to mark passed. 

My Usecase: 

We are running UI tests in release pipeline. Now, UI tests can be fragile and can have failures sometimes because of several reasons. We are working on making them more and more stable however we also want to have option to make failed build pass (red -> green) so that we can test the failed cases manually and make the failed build in bamboo pass if we verify that the failed cases are false positives.

I think this feature will be very useful for UI tests. Is there any plan for implementation?

From zigarn on BAM-14725:

With plan branches, we have this problem :

• trunk has no tests
• branch has tests

If we configure Maven with 'The build will produce test results.' or JUnit Parser task, the trunk will fail, but if we disable tests parsing, branch will not display tests results.

It would be better to have 2 levels : 'The build MUST produce test results' (current behavior) and 'The build MAY produce test results' (will not fail if there is no tests but will parse them if they exist)

Quarantine only covers some of the use cases specified here. Let's keep this open, or spin off a splinter ticket for the remaining functionality.

It seems this issue is still open so, I'll comment here first. It would be nice if the threshold percentage were capable of taking fractions of a percent. 99.8 for example. Currently it only accepts integers like 99.

For test plans with thousands of cases, this doesn't offer enough granularity if you only want to accept as passing a build with 2 or 3 failed test cases.

Thanks
David

Here's another scenario, where failing test shouldn't necessarily break a build.

We're using test driven development (TDD), and therefore our task workflow is:

• Write a test case, validating the implementation the new feature or bugfix. This test should fail, until the functionality specified in the issue is implemented. The test will be part of a test first/in progress suite.
• Implement the functionality according to the test. When the feature has been implemented correctly and the associated tests will pass, the test will be upgraded to a regression test suite.

In Bamboo I then have 2 builds,

• A regression test build, which should fail if one or more of the tests in the regression test categorys.
• A complete build which is should never fail because of test faulures. This build is used as a indicator om the total test and functionality status, including tasks in progress.

At the moment I'm unable to follow the progress of incomplete functionality/bug fixes in Bamboo, because this would cause breaking builds .

Best Regards
Mikis