Uploaded image for project: 'Bitbucket Data Center'
  1. Bitbucket Data Center
  2. BSERV-14388

Slow indexed commit properties lookups in Bitbucket 8.13

XMLWordPrintable

      Issue Summary

      The introduction of the "Show Signed Commits" feature in Bitbucket 8.13.0 has caused a performance regression when the system is under load.

      Indicators of this problem include:
      1) Profiling logs showing calls to IndexedCommitDao.getProperties() are slow:

      2023-08-18 05:16:16,744 | http-nio-7990-exec-64 | *1WXC635x315x11475234x32 | user | fz5q12
[33808.1ms] - "GET /projects/myproject/reposmyrepo/commits HTTP/1.1"
 [5.6ms] - ApplicationUser com.atlassian.bitbucket.user.UserService.getUserById(int)
 [7.0ms] - NormalUserPermissionVoterCalculator.calculate()
 [6.3ms] - ApplicationUser com.atlassian.bitbucket.user.UserService.getUserById(int)
 [2.5ms] - boolean com.atlassian.bitbucket.repository.RepositoryService.isEmpty(Repository)
  [2.4ms] - boolean com.atlassian.bitbucket.scm.ScmService.isEmpty(Repository)
   [2.4ms] - boolean com.atlassian.stash.internal.scm.git.mesh.RpcRepositoryClient.isEmpty(Repository)
 [2.2ms] - Branch com.atlassian.bitbucket.repository.RefService.getDefaultBranch(Repository)
  [2.2ms] - Branch com.atlassian.stash.internal.scm.git.mesh.RpcRefClient.resolveDefaultBranch(Repository,Builder)
 [27385.4ms] - Page com.atlassian.bitbucket.commit.CommitService.getCommits(CommitsRequest,PageRequest)
  [9.4ms] - void com.atlassian.stash.internal.scm.git.mesh.RpcCommitClient.getCommits(Repository,Builder,CommitCallback)
  [27375.2ms] - Page com.atlassian.bitbucket.commit.CommitEnricher.enrichPage(Repository,Page,Collection)
   [60.5ms] [count: 16, avg: 3.8ms] - String com.atlassian.stash.internal.plugin.PluginSettingDao.get(String,String)
   [27139.9ms] [count: 2, avg: 13570.0ms] - Map com.atlassian.stash.internal.content.IndexedCommitDao.getProperties(Iterable,Iterable)

      2) Thread dump (i.e. jstack) shows threads in GpgSignaturePropertyProvider.getProperties() waiting on a database lookup

      3) Database query logging/profiling shows an increase in average execution time for the following query:

      select internalin0_.id as col_0_0_, attributes1_.att_name as col_1_0_, attributes1_.att_value as col_2_0_ 
  from changeset internalin0_ inner join cs_attribute attributes1_ on internalin0_.id=attributes1_.cs_id and (attributes1_.att_name in (?)) 
 where internalin0_.id in (?)

      Anything longer than about 20ms on average could be a problem.

      4) Index scans per second on the changeset_pk index are substantially higher (more than 5x higher) than prior to Bitbucket 8.13.0.

      Workaround

      Disable the "Show Signed Commits feature by adding the following to $BITBUCKET_HOME/shared/bitbucket.properties and restarting all nodes:

      feature.commit.show.signatures=false

              Unassigned Unassigned
              behumphreys Ben Humphreys
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated:
                Resolved: