Details
-
Suggestion
-
Resolution: Fixed
-
None
Description
We've been doing a number of scalability tests and from these tests it's clear that response times for clone operations degrade exponentially as the number of concurrent clone requests exceed the number of available processers by a larger fraction.
We'll get optimal throughput if instead of allowing more processes to execute in parallel than we have CPUs to service, we would queue up the processes that we cannot service straightaway.
This will introduce an new issue, which we should address while building the queueing solution. The clone process uses a lot of CPU and disk IO while it's preparing the pack files. After the pack file is complete the process falls back to very low resource usage while the pack file is being streamed to the client. Depending on the connection speed of the client, this can take a long time. We should release the 'ticket' when the server starts streaming data back to the client, because we know that from that point on there is very little resource usage on the server.