Uploaded image for project: 'Bitbucket Data Center'
  1. Bitbucket Data Center
  2. BSERV-12489

Deadlock during initial sync on mirror

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: High High
    • 7.5.1
    • 6.7.5, 7.0.5, 7.4.0, 7.5.0
    • Smart Mirroring

      Issue Summary

      When farm vet detects lot of content inconsistencies during mirror startup and tries to sync them, suddenly syncing process stops and mirror never moves to SYNCHRONIZED status.

      Steps to Reproduce

      1. Add a project with at least 500-600 repositories on a single mirror (stand-alone or farm)
      2. Wait for sync to finish.
      3. Shut down the mirror and keep it down while users are pushing new commits/making ref changes on these repositories. Wait till there is at least 1 ref change in at least 100 (ideally 500-600) repositories.
      4. Start mirror with -Dexecutor.max.threads=4

      Expected Results

      Mirror moves to SYNCHRONIZED status

      Actual Results

      Mirror is stuck in METADATA_SYNCHRONIZED status. Attached thread-dump shows all 4 threads in "threadpool" are in WAITING status forever.

      Workaround

      Avoid keeping mirror down when there is lot of push activity happening on primary server. Mirror and primary should be down at same time OR mirror should be down when there is none or very minimal user activity on primary.

        1. thread-dumps.zip
          458 kB
          Manish

            mgoyal2@atlassian.com Manish
            mgoyal2@atlassian.com Manish
            Votes:
            2 Vote for this issue
            Watchers:
            8 Start watching this issue

              Created:
              Updated:
              Resolved: