I'm not completely sure how to reproduce this yet, but I noticed that we were losing our quarantine history. I initially thought it was an OPS issue (within our company), but looking at our failures I notice that they all seem to follow the same pattern.

Our build process is as follows:
1. Compile
2. Run unit tests
3. Deploy to a remote server (using ansible)
4. Run integration tests
5. Save the build artifact.

We have a handful of quarantined tests (which we'll get round to tidying up one day ), they are all integration tests. As long as we achieve a successful deploy, then even if some integration tests fail, the quarantine history works as expected. However if the deployment step fails, then none of the integration tests are run (rightly so), our issue is that in this case the quarantine history disappears for our integration tests, and only starts to build up again once the deployment is successful (and integration tests start running again). The behaviour I would expect is for the history to continue, but show a 'did not run' icon, or some equivalent. This still isn't ideal because several successive deployment failures would pollute the history slightly with irrelevant information (at the end of the day I want to know what the success-failure ratio is), but then I'm not sure how much history is preserved... If the full history was preserved, then being shown 'did not run' wouldn't affect this ratio.

If I had to make a black-box guess I would say that there's an assumption that because the test didn't run it has been removed from the build, and its quarantine history can be removed, but of course that's just conjecture.