Uploaded image for project: 'Bitbucket Data Center'
  1. Bitbucket Data Center
  2. BSERV-12444

Mirror node can time out while building local diffs

XMLWordPrintable

      Issue Summary

      Mirror nodes can time out while building the local diffs, as the idle timeout is set to the default value of 60 seconds and the process may take longer than that to produce the response.

      Steps to Reproduce

      1. Mirror a large repository

      Expected Results

      The repository completes synchronization.

      Actual Results

      The repository fails to sync with the following:

      2020-06-25 04:34:19,098 ERROR [farm-operation-4]  c.a.b.i.m.m.f.t.ReplyingTopicListener Error while processing request TopicRequest{message=RefChangeChunk{farmSynchronizationRequest=FarmSynchronizationRequest{externalRepositoryId=345, type=snapshot}, last=true, index=0, digest=..., payload.length=...}, replyTopic=refChangeChunk_..._reply, anticipatedResponders=1}
      java.lang.RuntimeException: Unexpected error generating a list of changes
              at com.atlassian.bitbucket.internal.mirroring.mirror.farm.refchange.RefChangeCalculator.localDiff(RefChangeCalculator.java:83)
              at com.atlassian.bitbucket.internal.mirroring.mirror.farm.synchronization.RepositorySynchronizationOperation.perform(RepositorySynchronizationOperation.java:111)
              at com.atlassian.bitbucket.internal.mirroring.mirror.farm.synchronization.RepositorySynchronizationOperation.perform(RepositorySynchronizationOperation.java:39)
              at com.atlassian.bitbucket.internal.mirroring.mirror.farm.topic.operation.MetricCollectingMirrorOperation.perform(MetricCollectingMirrorOperation.java:66)
              at com.atlassian.bitbucket.internal.mirroring.mirror.farm.refchange.RefChangeChunkOperation.perform(RefChangeChunkOperation.java:100)
              at com.atlassian.bitbucket.internal.mirroring.mirror.farm.refchange.RefChangeChunkOperation.perform(RefChangeChunkOperation.java:62)
              at com.atlassian.bitbucket.internal.mirroring.mirror.farm.topic.operation.MetricCollectingMirrorOperation.perform(MetricCollectingMirrorOperation.java:66)
              at com.atlassian.bitbucket.internal.mirroring.mirror.farm.topic.operation.AuthenticatedMirrorOperation.lambda$perform$0(AuthenticatedMirrorOperation.java:37)
              at com.atlassian.stash.internal.user.DefaultEscalatedSecurityContext.call(DefaultEscalatedSecurityContext.java:58)
              at com.atlassian.bitbucket.internal.mirroring.mirror.DefaultUpstreamUserHelper.performAsUpstreamUser(DefaultUpstreamUserHelper.java:93)
              at com.atlassian.bitbucket.internal.mirroring.mirror.farm.topic.operation.AuthenticatedMirrorOperation.perform(AuthenticatedMirrorOperation.java:37)
              at com.atlassian.bitbucket.internal.mirroring.mirror.farm.topic.operation.MultiThreadedMirrorOperation.lambda$perform$0(MultiThreadedMirrorOperation.java:45)
              at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)
              at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57)
              at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
              at com.atlassian.sal.core.executor.ThreadLocalDelegateRunnable.run(ThreadLocalDelegateRunnable.java:34)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
              at java.lang.Thread.run(Thread.java:748)
              ... 1 frame trimmed
      Caused by: com.atlassian.bitbucket.ServerException: An error occurred while executing an external process: process timed out
              at com.atlassian.bitbucket.scm.git.command.GitCommandExitHandler.evaluateThrowable(GitCommandExitHandler.java:120)
              at com.atlassian.bitbucket.scm.git.command.GitCommandExitHandler.onError(GitCommandExitHandler.java:208)
              at com.atlassian.bitbucket.scm.DefaultCommandExitHandler.onExit(DefaultCommandExitHandler.java:32)
              at com.atlassian.bitbucket.scm.BaseCommand.callExitHandler(BaseCommand.java:214)
              at com.atlassian.bitbucket.scm.BaseCommand$CommandFuture.internalGet(BaseCommand.java:348)
              at com.atlassian.bitbucket.scm.BaseCommand$CommandFuture.get(BaseCommand.java:312)
              at com.atlassian.bitbucket.scm.BaseCommand.call(BaseCommand.java:144)
              at com.atlassian.bitbucket.internal.mirroring.mirror.farm.refchange.RefChangeCalculator.localDiff(RefChangeCalculator.java:81)
              ... 18 common frames omitted
      Caused by: com.atlassian.utils.process.ProcessTimeoutException: process timed out
              at com.atlassian.utils.process.ExternalProcessImpl.wrapUpProcess(ExternalProcessImpl.java:664)
              at com.atlassian.utils.process.ExternalProcessImpl.finish(ExternalProcessImpl.java:163)
              at com.atlassian.bitbucket.scm.BaseCommand$CommandFuture.get(BaseCommand.java:310)
              ... 20 common frames omitted

      Workaround

      No known workaround.

            fdoherty@atlassian.com Frank Doherty
            fdoherty@atlassian.com Frank Doherty
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: