Add structured throughput logging to CCMA Dashboard and migration logs

XMLWordPrintable

    • 31

      Issue Summary

      When running large or complex Confluence migrations using CCMA, one of the most common questions during the migration or when reviewing a slow or stalled run is: "What is the throughput of this migration?"

      Currently, the CCMA dashboard shows a limited attachment upload rate (bytes/s) while the migration is running, but this data is:

      • Only available for the attachment upload phase — not for content export or import
      • Only visible live in the browser — if you close the tab or the session times out, the rate is gone
      • Never written to the migration logs — the atlassian-confluence.log and support.zip contain no summarized throughput data

      This means there is no way to understand how fast a Confluence migration is progressing, how long it is likely to take, or whether performance has degraded — from the logs alone. This is a recurring gap for any migrator running a large migration where time, data volume, and cutover windows are critical.

       

      Steps to Reproduce

      1. Start a CCMA migration (preferably with multiple spaces or large content volumes)
      2. Once migration completes (or during the run), generate a support.zip
      3. Check atlassian-confluence.log for any throughput or rate information
      4. No migration throughput details are recorded

      Expected Results

      Structured, summarized throughput entries written to the CCMA migration logs so that migrators can understand migration speed and progress directly from the logs — without needing to monitor the browser dashboard in real time or contact Atlassian support.

      What Should Be Logged:

      1. Per-Phase Throughput Summary — written when each phase completes:
       

      [THROUGHPUT] Phase=EXPORT Duration=38m SpacesProcessed=12 PagesExported=64,200 Rate=1,689 pages/min Status=COMPLETED
      [THROUGHPUT] Phase=UPLOAD Duration=55m EntitiesUploaded=63,800 Rate=1,160 entities/min Status=COMPLETED
      [THROUGHPUT] Phase=ATTACHMENTS Duration=140m DataTransferred=32.6GB FilesUploaded=41,000 Rate=3.88 MB/s Status=COMPLETED 

      2. Periodic Progress Snapshots — written every ~15 minutes during active migration:
       

      [THROUGHPUT] Snapshot Phase=ATTACHMENTS Elapsed=45m Completed=15,200/41,000 files DataSent=11.8GB/32.6GB CurrentRate=3.92 MB/s ETA=~95min 

      3. Migration-Level Summary — written on migration completion:
       

      [THROUGHPUT] MIGRATION COMPLETE TotalDuration=233m Spaces=12 Pages=64,200 Attachments=41,000 (32.6GB) ExportRate=1,689 pages/min UploadRate=1,160 ent/min AttachmentRate=3.88 MB/s 

       


      Actual Results

      • No throughput summary is written to any log file
      • The only rate data (attachment bytes/s) is computed in memory, shown transiently in the browser UI, and discarded on completion
      • The periodic migration logs uploaded to MCS are raw debug/info entries, not structured throughput summaries

              Assignee:
              Unassigned
              Reporter:
              Ismail Shaikh
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Created:
                Updated: