Bitbucket should allow multi-threaded code search indexing

Currently, when (re)indexing the content for search, Bitbucket Server uses only one thread, to prevent the indexing task from overloading the server's capacity to respond to the usual load.

However, if the content being indexed is large, it may take days to complete the indexing task, and during this time, the search functionality will be impacted (the search may only return the contents that have been already indexed).

• In order to speed up this potentially lengthy process, the System Admin should be given some way to throttle it. For example, if the load is low on weekends, and the server has many CPU cores available, it should be possible to dedicate more than just one CPU to this task (at least temporarily).

• Also, while indexing of all the content is underway, there is no reliable way to monitor its progress. The REST endpoint: /rest/indexing/latest/status most of the time returns:

  {"queues":{"delay":0,"event":0},"status":"IDLE"}
  

  and so it cannot be used for the monitoring.

The System Admins are confronted with this issue each time the content has to be reindexed, ie.

• When Bitbucket is first installed (or migrated from another server),
• After each major Bitbucket upgrade,
• Whenever a search index becomes corrupt, or search doesn't return the expected results, as advised in:
  Bitbucket Server Code Search is unable to find results from some or all repositories