Details
-
Bug
-
Resolution: Fixed
-
Highest
-
7.3.1, 7.21.4
-
22
-
Severity 2 - Major
-
79
-
Description
Issue Summary
When a branch is renamed upstream, mirror synchronization breaks.
Steps to Reproduce
Note: The key to the steps below is that there was a branch called feature/story (where story is stored as a file in the git ref data) which was deleted. Then, a branch called feature/story/test123 is created (where story is now a folder in the git ref data).
- Create New Project: TEST_PROJECT
- Create New Repository: TEST_REPOSITORY
- Push a simple 'hello world' text file: 'test_file.txt' to the repository with branch: master
- Add project: TEST_PROJECT to a smart mirror sync list
- LOCALLY: Create branch feature/story git checkout -b feature/story
- PUSH Branch to upstream git push --set-upstream origin feature/story
- Ensure changes have synchronised to the smart mirror
- STOP Bitbucket Smart Mirror (This is just an easy way to reproduce. Alternatively - introduce short delay between syncs)
- LOCALLY (while mirror is down or during the short delay): Delete feature/story, create a new branch: feature/story/test123 and push to upstream
- git checkout master
- git branch -d feature/story
- git checkout -b feature/story/test123
- git push origin --delete feature/story
- git push --set-upstream origin feature/story/test123
- START Bitbucket Smart Mirror
Expected Results
The Smart Mirror should synchronize the branch changes.
Actual Results
Smart Mirror is unable synchronize any changes to the repository. It fails to create the new branch feature/story/test123 because it is in conflict with a deleted on upstream branch: feature/story, and has not been deleted from the mirror yet.
The below exception is thrown in the atlassian-bitbucket.log file:
2020-09-25 05:24:42,425 WARN [farm-operation-1] c.a.b.i.m.m.f.t.o.RetryingMirrorOperation MirrorOperation: updateRef failed attempt 1/5 for request: RepositorySynchronizationRequest{changeId=ad812faadc66914859ea8a9b584da3a27e92c775, externalRepositoryId=12, orchestratingNodeVmId=aff79ec8-ff6a-4be6-b4ec-092e748e64bc, type=snapshot} waiting 1530 ms before retrying
2020-09-25 05:24:43,983 WARN [farm-operation-2] c.a.b.i.m.m.f.t.o.RetryingMirrorOperation MirrorOperation: updateRef failed attempt 2/5 for request: RepositorySynchronizationRequest{changeId=ad812faadc66914859ea8a9b584da3a27e92c775, externalRepositoryId=12, orchestratingNodeVmId=aff79ec8-ff6a-4be6-b4ec-092e748e64bc, type=snapshot} waiting 6604 ms before retrying
2020-09-25 05:24:50,611 WARN [farm-operation-4] c.a.b.i.m.m.f.t.o.RetryingMirrorOperation MirrorOperation: updateRef failed attempt 3/5 for request: RepositorySynchronizationRequest{changeId=ad812faadc66914859ea8a9b584da3a27e92c775, externalRepositoryId=12, orchestratingNodeVmId=aff79ec8-ff6a-4be6-b4ec-092e748e64bc, type=snapshot} waiting 7219 ms before retrying
2020-09-25 05:24:51,621 INFO [Caesium-1-4] @OVFWQIx324x1366568052x1 c.a.b.i.m.m.UpstreamRegistrationJob Upgraded mirror add-on in https://upstream
2020-09-25 05:24:57,855 WARN [farm-operation-1] c.a.b.i.m.m.f.t.o.RetryingMirrorOperation MirrorOperation: updateRef failed attempt 4/5 for request: RepositorySynchronizationRequest{changeId=ad812faadc66914859ea8a9b584da3a27e92c775, externalRepositoryId=12, orchestratingNodeVmId=aff79ec8-ff6a-4be6-b4ec-092e748e64bc, type=snapshot} waiting 28671 ms before retrying
2020-09-25 05:25:26,545 ERROR [farm-operation-2] c.a.b.i.m.m.f.t.o.RetryingMirrorOperation MirrorOperation: updateRef failed attempt 5/5 for request: RepositorySynchronizationRequest{changeId=ad812faadc66914859ea8a9b584da3a27e92c775, externalRepositoryId=12, orchestratingNodeVmId=aff79ec8-ff6a-4be6-b4ec-092e748e64bc, type=snapshot}; giving up
com.atlassian.bitbucket.scm.CommandFailedException: '/usr/bin/git update-ref --stdin -z' exited with code 128 saying: fatal: cannot lock ref 'refs/heads/feature/story/test123': 'refs/heads/feature/story' exists; cannot create 'refs/heads/feature/story/test123'
at com.atlassian.bitbucket.scm.DefaultCommandExitHandler.onError(DefaultCommandExitHandler.java:47)
at com.atlassian.bitbucket.scm.git.command.GitCommandExitHandler.evaluateThrowable(GitCommandExitHandler.java:111)
at com.atlassian.bitbucket.scm.git.command.GitCommandExitHandler.onError(GitCommandExitHandler.java:208)
at com.atlassian.bitbucket.scm.DefaultCommandExitHandler.onExit(DefaultCommandExitHandler.java:32)
at com.atlassian.bitbucket.internal.process.nu.NioNuProcessHandler.callExitHandler(NioNuProcessHandler.java:285)
at com.atlassian.bitbucket.internal.process.nu.NioNuProcessHandler.finish(NioNuProcessHandler.java:326)
at com.atlassian.bitbucket.internal.process.nu.NioNuProcessHandler.onExit(NioNuProcessHandler.java:123)
at com.zaxxer.nuprocess.internal.BasePosixProcess.onExit(BasePosixProcess.java:319)
at com.zaxxer.nuprocess.linux.ProcessEpoll.handleExit(ProcessEpoll.java:371)
at com.zaxxer.nuprocess.linux.ProcessEpoll.cleanupProcess(ProcessEpoll.java:334)
at com.zaxxer.nuprocess.linux.ProcessEpoll.process(ProcessEpoll.java:272)
at com.zaxxer.nuprocess.internal.BaseEventProcessor.run(BaseEventProcessor.java:81)
at com.zaxxer.nuprocess.linux.ProcessEpoll.run(ProcessEpoll.java:188)
at com.zaxxer.nuprocess.linux.LinuxProcess.run(LinuxProcess.java:114)
at com.zaxxer.nuprocess.linux.LinProcessFactory.runProcess(LinProcessFactory.java:50)
at com.zaxxer.nuprocess.NuProcessBuilder.run(NuProcessBuilder.java:273)
at com.atlassian.bitbucket.internal.process.nu.NuNioProcessHelper.run(NuNioProcessHelper.java:75)
at com.atlassian.bitbucket.internal.process.NioCommand.call(NioCommand.java:52)
at com.atlassian.bitbucket.internal.mirroring.mirror.farm.synchronization.UpdateRefHelper.perform(UpdateRefHelper.java:51)
at com.atlassian.bitbucket.internal.mirroring.mirror.farm.synchronization.RepositorySynchronizationOperation.perform(RepositorySynchronizationOperation.java:125)
at com.atlassian.bitbucket.internal.mirroring.mirror.farm.synchronization.RepositorySynchronizationOperation.perform(RepositorySynchronizationOperation.java:41)
at com.atlassian.bitbucket.internal.mirroring.mirror.farm.topic.operation.AuthenticatedMirrorOperation.lambda$perform$0(AuthenticatedMirrorOperation.java:37)
at com.atlassian.stash.internal.user.DefaultEscalatedSecurityContext.call(DefaultEscalatedSecurityContext.java:59)
at com.atlassian.bitbucket.internal.mirroring.mirror.DefaultUpstreamUserHelper.performAsUpstreamUser(DefaultUpstreamUserHelper.java:93)
at com.atlassian.bitbucket.internal.mirroring.mirror.farm.topic.operation.AuthenticatedMirrorOperation.perform(AuthenticatedMirrorOperation.java:37)
at com.atlassian.bitbucket.internal.mirroring.mirror.farm.topic.operation.MultiThreadedMirrorOperation.lambda$perform$0(MultiThreadedMirrorOperation.java:45)
at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)
at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57)
at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
at com.atlassian.sal.core.executor.ThreadLocalDelegateRunnable.run(ThreadLocalDelegateRunnable.java:34)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.lang.Thread.run(Thread.java:748)
... 1 frame trimmed
Caused by: com.atlassian.utils.process.ProcessException: Non-zero exit code: 128
at com.atlassian.bitbucket.internal.process.nu.NioNuProcessHandler.callExitHandler(NioNuProcessHandler.java:277)
... 28 common frames omitted
Workaround
Remove the project from syncing to the Mirror/farm. Restart the Smart Mirror, wait during initial sync whilst its deleted. Add the project back to be synced to the Mirror/Farm.
Alternate Workaround
- Remove the Problematic branch on the Bitbucket that caused the replication issue on the Mirror and try to resync the Mirror