Details
-
Bug
-
Resolution: Fixed
-
Highest
-
None
-
None
-
Severity 1 - Critical
-
3
-
Description
When the system performs a merge (pull request merges, ref sync merges to synchronize branches, etc.), extremely large merges can trigger automatic GC. When that happens, the ref being updated by the merge is packed, which triggers Bitbucket Server's overall merge processing to fail because it can't resolve the ref after the merge completes.
An example error looks like this:
2017-10-03 13:32:23,732 INFO [http-nio-8443-exec-3] jdoe *OUL6JRx811x88948x0 t4udt2 1.1.1.1,10.10.10.10 "GET /rest/api/latest/projects/KEY/repos/slug/pull-requests/1/merge HTTP/1.1" c.a.s.i.s.g.p.DefaultPullRequestAutoMergeStrategy KEY/slug[1]: Pull request 1@0 failed automatic merging com.atlassian.bitbucket.scm.MergeException: The merge could not be completed automatically. Please clone 'KEY/slug', checkout 'master' from 'KEY/slug' and merge 'master' from '~USER/slug' (or commit 'd9e1df79a80a50d7ce9e12d8e88fe8ef0710edd5') manually, resolving any conflicts, and push the results. at com.atlassian.stash.internal.scm.git.merge.MergeCommand.newMergeException(MergeCommand.java:170) ~[bitbucket-git-4.13.1.jar:na] at com.atlassian.stash.internal.scm.git.merge.MergeCommand.doWithWorkTree(MergeCommand.java:165) ~[bitbucket-git-4.13.1.jar:na] at com.atlassian.stash.internal.scm.git.merge.MergeCommand.call(MergeCommand.java:67) ~[bitbucket-git-4.13.1.jar:na] at com.atlassian.stash.internal.scm.git.pull.DefaultPullRequestAutoMergeStrategy.doMerge(DefaultPullRequestAutoMergeStrategy.java:107) ~[bitbucket-git-4.13.1.jar:na] at com.atlassian.stash.internal.scm.git.pull.DefaultPullRequestAutoMergeStrategy.merge(DefaultPullRequestAutoMergeStrategy.java:75) ~[bitbucket-git-4.13.1.jar:na] at com.atlassian.stash.internal.scm.git.pull.CompositePullRequestAutoMergeStrategy.merge(CompositePullRequestAutoMergeStrategy.java:22) [bitbucket-git-4.13.1.jar:na] at com.atlassian.stash.internal.scm.git.pull.CachingPullRequestAutoMergeStrategy.merge(CachingPullRequestAutoMergeStrategy.java:49) [bitbucket-git-4.13.1.jar:na] at com.atlassian.stash.internal.scm.git.pull.CompositePullRequestAutoMergeStrategy.merge(CompositePullRequestAutoMergeStrategy.java:22) [bitbucket-git-4.13.1.jar:na] at com.atlassian.stash.internal.scm.git.pull.DefaultPullRequestRefHelper.resolveNewEffectiveDiff(DefaultPullRequestRefHelper.java:425) [bitbucket-git-4.13.1.jar:na] at com.atlassian.stash.internal.scm.git.pull.DefaultPullRequestRefHelper.access$500(DefaultPullRequestRefHelper.java:53) [bitbucket-git-4.13.1.jar:na] at com.atlassian.stash.internal.scm.git.pull.DefaultPullRequestRefHelper$EffectiveDiffOperation.perform(DefaultPullRequestRefHelper.java:545) [bitbucket-git-4.13.1.jar:na] at com.atlassian.stash.internal.scm.git.pull.DefaultPullRequestRefHelper$EffectiveDiffOperation.perform(DefaultPullRequestRefHelper.java:510) [bitbucket-git-4.13.1.jar:na] at com.atlassian.stash.internal.scm.git.pull.DefaultPullRequestRefHelper$AbstractOperation.perform(DefaultPullRequestRefHelper.java:476) [bitbucket-git-4.13.1.jar:na] at com.atlassian.stash.internal.concurrent.ClusterMappedLock.withLock(ClusterMappedLock.java:49) [bitbucket-service-impl-4.13.1.jar:na] at com.atlassian.stash.internal.concurrent.ClusterPullRequestLock.withLock(ClusterPullRequestLock.java:25) [bitbucket-service-impl-4.13.1.jar:na] at com.atlassian.stash.internal.scm.git.pull.DefaultPullRequestRefHelper.withLock(DefaultPullRequestRefHelper.java:458) [bitbucket-git-4.13.1.jar:na] at com.atlassian.stash.internal.scm.git.pull.DefaultPullRequestRefHelper.access$300(DefaultPullRequestRefHelper.java:53) [bitbucket-git-4.13.1.jar:na] at com.atlassian.stash.internal.scm.git.pull.DefaultPullRequestRefHelper$EffectiveDiffOperation.call(DefaultPullRequestRefHelper.java:522) [bitbucket-git-4.13.1.jar:na] at com.atlassian.stash.internal.scm.git.pull.DefaultPullRequestRefHelper$EffectiveDiffOperation.call(DefaultPullRequestRefHelper.java:510) [bitbucket-git-4.13.1.jar:na] at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_131] at com.atlassian.util.concurrent.ConcurrentOperationMapImpl$CallerRunsFuture.get(ConcurrentOperationMapImpl.java:63) [atlassian-util-concurrent-3.0.0.jar:na] at com.atlassian.util.concurrent.ConcurrentOperationMapImpl.runOperation(ConcurrentOperationMapImpl.java:51) [atlassian-util-concurrent-3.0.0.jar:na] at com.atlassian.stash.internal.scm.git.pull.DefaultPullRequestRefHelper.resolveEffectiveDiff(DefaultPullRequestRefHelper.java:132) [bitbucket-git-4.13.1.jar:na] at com.atlassian.stash.internal.scm.git.DefaultGitPullRequestCommandFactory$3.call(DefaultGitPullRequestCommandFactory.java:211) [bitbucket-git-4.13.1.jar:na] at com.atlassian.stash.internal.scm.git.DefaultGitPullRequestCommandFactory$3.call(DefaultGitPullRequestCommandFactory.java:207) [bitbucket-git-4.13.1.jar:na] at com.atlassian.stash.internal.pull.DefaultMergeRequestCheckService.tryMerge(DefaultMergeRequestCheckService.java:95) [bitbucket-service-impl-4.13.1.jar:na] at com.atlassian.stash.internal.pull.DefaultMergeRequestCheckService.checkMergeability(DefaultMergeRequestCheckService.java:57) [bitbucket-service-impl-4.13.1.jar:na] at com.atlassian.stash.internal.pull.DefaultPullRequestService.canMerge(DefaultPullRequestService.java:348) [bitbucket-service-impl-4.13.1.jar:na] at com.atlassian.plugin.util.ContextClassLoaderSettingInvocationHandler.invoke(ContextClassLoaderSettingInvocationHandler.java:26) [atlassian-plugins-core-4.2.4.jar:na] at org.eclipse.gemini.blueprint.service.importer.support.internal.aop.ServiceInvoker.doInvoke(ServiceInvoker.java:56) [gemini-blueprint-core-2.0.0.BUILD-atlassian-m002.jar:2.0.0.BUILD-atlassian-m002] at org.eclipse.gemini.blueprint.service.importer.support.internal.aop.ServiceInvoker.invoke(ServiceInvoker.java:60) [gemini-blueprint-core-2.0.0.BUILD-atlassian-m002.jar:2.0.0.BUILD-atlassian-m002] at org.eclipse.gemini.blueprint.service.util.internal.aop.ServiceTCCLInterceptor.invokeUnprivileged(ServiceTCCLInterceptor.java:70) [gemini-blueprint-core-2.0.0.BUILD-atlassian-m002.jar:2.0.0.BUILD-atlassian-m002] at org.eclipse.gemini.blueprint.service.util.internal.aop.ServiceTCCLInterceptor.invoke(ServiceTCCLInterceptor.java:53) [gemini-blueprint-core-2.0.0.BUILD-atlassian-m002.jar:2.0.0.BUILD-atlassian-m002] at org.eclipse.gemini.blueprint.service.importer.support.LocalBundleContextAdvice.invoke(LocalBundleContextAdvice.java:57) [gemini-blueprint-core-2.0.0.BUILD-atlassian-m002.jar:2.0.0.BUILD-atlassian-m002] at com.atlassian.stash.internal.rest.pull.PullRequestResource.canMerge(PullRequestResource.java:427) [bitbucket-rest-4.13.1.jar:na] at com.atlassian.applinks.core.rest.context.ContextFilter.doFilter(ContextFilter.java:24) [applinks-plugin-5.2.4_1488188354000.jar:na] at com.atlassian.applinks.core.rest.context.ContextFilter.doFilter(ContextFilter.java:24) [applinks-plugin-5.2.4_1488188354000.jar:na] at com.atlassian.applinks.core.rest.context.ContextFilter.doFilter(ContextFilter.java:24) [applinks-plugin-5.2.4_1488188354000.jar:na] at com.atlassian.applinks.core.rest.context.ContextFilter.doFilter(ContextFilter.java:24) [applinks-plugin-5.2.4_1488188354000.jar:na] at com.atlassian.applinks.core.rest.context.ContextFilter.doFilter(ContextFilter.java:24) [applinks-plugin-5.2.4_1488188354000.jar:na] at com.atlassian.analytics.client.filter.UniversalAnalyticsFilter.doFilter(UniversalAnalyticsFilter.java:92) [analytics-client-5.2.20_1488188354000.jar:na] at com.atlassian.analytics.client.filter.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:39) [analytics-client-5.2.20_1488188354000.jar:na] at com.atlassian.stash.internal.spring.lifecycle.LifecycleJohnsonServletFilterModuleContainerFilter.doFilter(LifecycleJohnsonServletFilterModuleContainerFilter.java:42) [classes/:na] at com.atlassian.plugin.connect.plugin.auth.scope.ApiScopingFilter.doFilter(ApiScopingFilter.java:81) [atlassian-connect-plugin-1.1.86-bitbucket-04.jar:na] at com.atlassian.stash.internal.spring.lifecycle.LifecycleJohnsonServletFilterModuleContainerFilter.doFilter(LifecycleJohnsonServletFilterModuleContainerFilter.java:42) [classes/:na] at com.atlassian.stash.internal.spring.security.StashAuthenticationFilter.doFilter(StashAuthenticationFilter.java:88) [classes/:na] at com.atlassian.stash.internal.web.auth.BeforeLoginPluginAuthenticationFilter.doInsideSpringSecurityChain(BeforeLoginPluginAuthenticationFilter.java:112) [classes/:na] at com.atlassian.stash.internal.web.auth.BeforeLoginPluginAuthenticationFilter.doFilter(BeforeLoginPluginAuthenticationFilter.java:75) [classes/:na] at com.atlassian.security.auth.trustedapps.filter.TrustedApplicationsFilter.doFilter(TrustedApplicationsFilter.java:94) [atlassian-trusted-apps-core-4.2.0.jar:na] at com.atlassian.oauth.serviceprovider.internal.servlet.OAuthFilter.doFilter(OAuthFilter.java:67) [atlassian-oauth-service-provider-plugin-2.0.3_1488188354000.jar:na] at com.atlassian.core.filters.ServletContextThreadLocalFilter.doFilter(ServletContextThreadLocalFilter.java:21) [atlassian-core-4.6.19.jar:na] at com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:31) [atlassian-core-4.6.19.jar:na] at com.atlassian.stash.internal.spring.lifecycle.LifecycleJohnsonServletFilterModuleContainerFilter.doFilter(LifecycleJohnsonServletFilterModuleContainerFilter.java:42) [classes/:na] at com.atlassian.plugin.connect.plugin.auth.user.ThreeLeggedAuthFilter.doFilter(ThreeLeggedAuthFilter.java:109) [atlassian-connect-plugin-1.1.86-bitbucket-04.jar:na] at com.atlassian.jwt.internal.servlet.JwtAuthFilter.doFilter(JwtAuthFilter.java:32) [jwt-plugin-1.5.11-0002_1488188354000.jar:na] at com.atlassian.analytics.client.filter.DefaultAnalyticsFilter.doFilter(DefaultAnalyticsFilter.java:38) [analytics-client-5.2.20_1488188354000.jar:na] at com.atlassian.analytics.client.filter.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:39) [analytics-client-5.2.20_1488188354000.jar:na] at com.atlassian.stash.internal.spring.lifecycle.LifecycleJohnsonServletFilterModuleContainerFilter.doFilter(LifecycleJohnsonServletFilterModuleContainerFilter.java:42) [classes/:na] at com.atlassian.stash.internal.web.auth.BeforeLoginPluginAuthenticationFilter.doBeforeBeforeLoginFilters(BeforeLoginPluginAuthenticationFilter.java:90) [classes/:na] at com.atlassian.stash.internal.web.auth.BeforeLoginPluginAuthenticationFilter.doFilter(BeforeLoginPluginAuthenticationFilter.java:73) [classes/:na] at com.atlassian.stash.internal.request.DefaultRequestManager.doAsRequest(DefaultRequestManager.java:83) ~[bitbucket-service-impl-4.13.1.jar:na] at com.atlassian.stash.internal.hazelcast.ConfigurableWebFilter.doFilter(ConfigurableWebFilter.java:38) ~[classes/:na] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[na:1.8.0_131] at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_131] ... 304 frames trimmed Caused by: com.atlassian.bitbucket.scm.git.command.merge.GitMergeException: After the pull request was merged, the merge commit could not be resolved. at com.atlassian.stash.internal.scm.git.merge.ObjectFetchStrategy.resolveMerge(ObjectFetchStrategy.java:332) ~[bitbucket-git-4.13.1.jar:na] at com.atlassian.stash.internal.scm.git.merge.ObjectFetchStrategy.fetch(ObjectFetchStrategy.java:106) ~[bitbucket-git-4.13.1.jar:na] at com.atlassian.stash.internal.scm.git.merge.TimedFetchStrategy.fetch(TimedFetchStrategy.java:30) ~[bitbucket-git-4.13.1.jar:na] at com.atlassian.stash.internal.scm.git.merge.MergeCommand.doFetch(MergeCommand.java:102) ~[bitbucket-git-4.13.1.jar:na] at com.atlassian.stash.internal.scm.git.merge.MergeCommand.doWithWorkTree(MergeCommand.java:127) ~[bitbucket-git-4.13.1.jar:na] ... 63 common frames omitted Caused by: java.io.FileNotFoundException: /var/atlassian/application-data/bitbucket/tmp/git/slug-work7381871372300250730.git/.git/refs/heads/master (No such file or directory) at java.io.FileInputStream.open0(Native Method) ~[na:1.8.0_131] at java.io.FileInputStream.open(FileInputStream.java:195) ~[na:1.8.0_131] at java.io.FileInputStream.<init>(FileInputStream.java:138) ~[na:1.8.0_131] at com.google.common.io.Files$FileByteSource.openStream(Files.java:125) ~[guava-18.0.jar:na] at com.google.common.io.Files$FileByteSource.openStream(Files.java:115) ~[guava-18.0.jar:na] at com.google.common.io.ByteSource$AsCharSource.openStream(ByteSource.java:420) ~[guava-18.0.jar:na] at com.google.common.io.CharSource.openBufferedStream(CharSource.java:91) ~[guava-18.0.jar:na] at com.google.common.io.CharSource.readFirstLine(CharSource.java:168) ~[guava-18.0.jar:na] at com.google.common.io.Files.readFirstLine(Files.java:513) ~[guava-18.0.jar:na] at com.atlassian.stash.internal.scm.git.merge.ObjectFetchStrategy.resolveMerge(ObjectFetchStrategy.java:328) ~[bitbucket-git-4.13.1.jar:na] ... 67 common frames omitted
Enabling debug logging includes output similar to this:
2017-10-03 13:31:42,430 DEBUG [http-nio-8443-exec-3] jdoe *OUL6JRx811x88948x0 t4udt2 1.1.1.1,10.10.10.10 "GET /rest/api/latest/projects/KEY/repos/slug/pull-requests/1/merge HTTP/1.1" c.a.bitbucket.scm.BaseCommand Executed /usr/bin/git clone --no-checkout --shared -b master /var/atlassian/application-data/bitbucket/shared/data/repositories/1 /var/atlassian/application-data/bitbucket/tmp/git/slug-work7381871372300250730.git 2017-10-03 13:31:42,431 DEBUG [http-nio-8443-exec-3] jdoe *OUL6JRx811x88948x0 t4udt2 1.1.1.1,10.10.10.10 "GET /rest/api/latest/projects/KEY/repos/slug/pull-requests/1/merge HTTP/1.1" c.a.b.s.g.c.GitCommandExitHandler /usr/bin/git clone --no-checkout --shared -b master /var/atlassian/application-data/bitbucket/shared/data/repositories/1 /var/atlassian/application-data/bitbucket/tmp/git/slug-work7381871372300250730.git did not fail, but the following was written to stderr: Cloning into '/var/atlassian/application-data/bitbucket/tmp/git/slug-work7381871372300250730.git'... done. 2017-10-03 13:31:42,860 DEBUG [http-nio-8443-exec-3] jdoe *OUL6JRx811x88948x0 t4udt2 1.1.1.1,10.10.10.10 "GET /rest/api/latest/projects/KEY/repos/slug/pull-requests/1/merge HTTP/1.1" c.a.bitbucket.scm.BaseCommand Executed /usr/bin/git reset --quiet 9cda5ad612503ddd7cbfb71ed9638a06e2e67613 -- 2017-10-03 13:32:23,185 DEBUG [http-nio-8443-exec-3] jdoe *OUL6JRx811x88948x0 t4udt2 1.1.1.1,10.10.10.10 "GET /rest/api/latest/projects/KEY/repos/slug/pull-requests/1/merge HTTP/1.1" c.a.bitbucket.scm.BaseCommand Executed /usr/bin/git merge -m Automatic merge --no-ff --log --allow-unrelated-histories d9e1df79a80a50d7ce9e12d8e88fe8ef0710edd5 2017-10-03 13:32:23,185 DEBUG [http-nio-8443-exec-3] jdoe *OUL6JRx811x88948x0 t4udt2 1.1.1.1,10.10.10.10 "GET /rest/api/latest/projects/KEY/repos/slug/pull-requests/1/merge HTTP/1.1" c.a.b.s.g.c.GitCommandExitHandler /usr/bin/git merge -m Automatic merge --no-ff --log --allow-unrelated-histories d9e1df79a80a50d7ce9e12d8e88fe8ef0710edd5 did not fail, but the following was written to stderr: Auto packing the repository in background for optimum performance. See "git help gc" for manual housekeeping.