Uploaded image for project: 'Bitbucket Data Center'
  1. Bitbucket Data Center
  2. BSERV-12444

Mirror node can time out while building local diffs

    XMLWordPrintable

Details

    Description

      Issue Summary

      Mirror nodes can time out while building the local diffs, as the idle timeout is set to the default value of 60 seconds and the process may take longer than that to produce the response.

      Steps to Reproduce

      1. Mirror a large repository

      Expected Results

      The repository completes synchronization.

      Actual Results

      The repository fails to sync with the following:

      2020-06-25 04:34:19,098 ERROR [farm-operation-4]  c.a.b.i.m.m.f.t.ReplyingTopicListener Error while processing request TopicRequest{message=RefChangeChunk{farmSynchronizationRequest=FarmSynchronizationRequest{externalRepositoryId=345, type=snapshot}, last=true, index=0, digest=..., payload.length=...}, replyTopic=refChangeChunk_..._reply, anticipatedResponders=1}
      java.lang.RuntimeException: Unexpected error generating a list of changes
              at com.atlassian.bitbucket.internal.mirroring.mirror.farm.refchange.RefChangeCalculator.localDiff(RefChangeCalculator.java:83)
              at com.atlassian.bitbucket.internal.mirroring.mirror.farm.synchronization.RepositorySynchronizationOperation.perform(RepositorySynchronizationOperation.java:111)
              at com.atlassian.bitbucket.internal.mirroring.mirror.farm.synchronization.RepositorySynchronizationOperation.perform(RepositorySynchronizationOperation.java:39)
              at com.atlassian.bitbucket.internal.mirroring.mirror.farm.topic.operation.MetricCollectingMirrorOperation.perform(MetricCollectingMirrorOperation.java:66)
              at com.atlassian.bitbucket.internal.mirroring.mirror.farm.refchange.RefChangeChunkOperation.perform(RefChangeChunkOperation.java:100)
              at com.atlassian.bitbucket.internal.mirroring.mirror.farm.refchange.RefChangeChunkOperation.perform(RefChangeChunkOperation.java:62)
              at com.atlassian.bitbucket.internal.mirroring.mirror.farm.topic.operation.MetricCollectingMirrorOperation.perform(MetricCollectingMirrorOperation.java:66)
              at com.atlassian.bitbucket.internal.mirroring.mirror.farm.topic.operation.AuthenticatedMirrorOperation.lambda$perform$0(AuthenticatedMirrorOperation.java:37)
              at com.atlassian.stash.internal.user.DefaultEscalatedSecurityContext.call(DefaultEscalatedSecurityContext.java:58)
              at com.atlassian.bitbucket.internal.mirroring.mirror.DefaultUpstreamUserHelper.performAsUpstreamUser(DefaultUpstreamUserHelper.java:93)
              at com.atlassian.bitbucket.internal.mirroring.mirror.farm.topic.operation.AuthenticatedMirrorOperation.perform(AuthenticatedMirrorOperation.java:37)
              at com.atlassian.bitbucket.internal.mirroring.mirror.farm.topic.operation.MultiThreadedMirrorOperation.lambda$perform$0(MultiThreadedMirrorOperation.java:45)
              at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)
              at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57)
              at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
              at com.atlassian.sal.core.executor.ThreadLocalDelegateRunnable.run(ThreadLocalDelegateRunnable.java:34)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
              at java.lang.Thread.run(Thread.java:748)
              ... 1 frame trimmed
      Caused by: com.atlassian.bitbucket.ServerException: An error occurred while executing an external process: process timed out
              at com.atlassian.bitbucket.scm.git.command.GitCommandExitHandler.evaluateThrowable(GitCommandExitHandler.java:120)
              at com.atlassian.bitbucket.scm.git.command.GitCommandExitHandler.onError(GitCommandExitHandler.java:208)
              at com.atlassian.bitbucket.scm.DefaultCommandExitHandler.onExit(DefaultCommandExitHandler.java:32)
              at com.atlassian.bitbucket.scm.BaseCommand.callExitHandler(BaseCommand.java:214)
              at com.atlassian.bitbucket.scm.BaseCommand$CommandFuture.internalGet(BaseCommand.java:348)
              at com.atlassian.bitbucket.scm.BaseCommand$CommandFuture.get(BaseCommand.java:312)
              at com.atlassian.bitbucket.scm.BaseCommand.call(BaseCommand.java:144)
              at com.atlassian.bitbucket.internal.mirroring.mirror.farm.refchange.RefChangeCalculator.localDiff(RefChangeCalculator.java:81)
              ... 18 common frames omitted
      Caused by: com.atlassian.utils.process.ProcessTimeoutException: process timed out
              at com.atlassian.utils.process.ExternalProcessImpl.wrapUpProcess(ExternalProcessImpl.java:664)
              at com.atlassian.utils.process.ExternalProcessImpl.finish(ExternalProcessImpl.java:163)
              at com.atlassian.bitbucket.scm.BaseCommand$CommandFuture.get(BaseCommand.java:310)
              ... 20 common frames omitted

      Workaround

      No known workaround.

      Attachments

        Issue Links

          Activity

            People

              fdoherty@atlassian.com Frank Doherty
              fdoherty@atlassian.com Frank Doherty
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: