Uploaded image for project: 'Bitbucket Data Center'
  1. Bitbucket Data Center
  2. BSERV-2500

Improve scalability by queueing of scm hosting operations under high load

    XMLWordPrintable

Details

    • Suggestion
    • Resolution: Fixed
    • 1.0.1
    • Git Hosting
    • None
    • We collect Bitbucket feedback from various sources, and we evaluate what we've collected when planning our product roadmap. To understand how this piece of feedback will be reviewed, see our Implementation of New Features Policy.

    Description

      We've been doing a number of scalability tests and from these tests it's clear that response times for clone operations degrade exponentially as the number of concurrent clone requests exceed the number of available processers by a larger fraction.

      We'll get optimal throughput if instead of allowing more processes to execute in parallel than we have CPUs to service, we would queue up the processes that we cannot service straightaway.

      This will introduce an new issue, which we should address while building the queueing solution. The clone process uses a lot of CPU and disk IO while it's preparing the pack files. After the pack file is complete the process falls back to very low resource usage while the pack file is being streamed to the client. Depending on the connection speed of the client, this can take a long time. We should release the 'ticket' when the server starts streaming data back to the client, because we know that from that point on there is very little resource usage on the server.

      Attachments

        Activity

          People

            mheemskerk Michael Heemskerk (Inactive)
            mheemskerk Michael Heemskerk (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: