Could just be random fluctuations, but it appears that pipelines take longer when watching the progress on the site, to when run without. Marking as major as this is conceivably costing us money.

Example of randomly selected step, with no code changes to the code being tested...

Not watching progress on the website:

Test Users JS
45 tests passed
51s

Watching on the website:

Test Users JS
45 tests passed
1m 50s