Migrating attachments takes a lot of time with errors in the log

XMLWordPrintable

    • 2
    • Minor
    • 1

      Issue Summary

      Media responded with a 400 error for the API call. Some fileIds were successfully uploaded, while others failed.
      JCMA, upon receiving the error, called the API again with the same set of fileIds.
      Since some of these fileIds were already uploaded, Media returned a 409 (conflict error) for those attachments, while the previously failed ones migrated successfully this time.
      The repeated retries of already successful attachments caused the migration to take longer than expected.

      This is reproducible on Data Center: / (no)

      Steps to Reproduce

      1. None

      Expected Results

      1. Media upload API call successful for all fileIds
      2. On retry, JCMA calls upload only for failed fileIds

      Actual Results

      1. Media returning 400 frequently
      2. JCMA calling upload again for all successful and failed fileIds causing 409 exceptions

      The below exception is thrown in the migration error log file:

      2024-06-13 11:30:12,588+0000 AttUpload-Thread-SXRAY-106077 WARN      [c.a.j.m.workers.attachments.AttachmentMigrator] creating file in Media failed for attachment = 427908
2024-06-13 11:30:12,591+0000 AttUpload-Thread-SXRAY-106077 ERROR      [c.a.j.migration.log.ExportLog] 2024-06-13 11:30:12.591381 ERROR SXRAY attachment-export We couldn't export Attachment 'S2942 - Restituer partenaire-2024-02-12-10-13-45-838-0.html' (on Issue SXRAY-38466). Reason: com.atlassian.jira.migration.mediaclient.MediaClientErrorException: request to Media failed. [JCMA 401]
2024-06-13 11:30:12,679+0000 AttUpload-Thread-SXRAY-106073 WARN      [c.a.j.m.workers.attachments.AttachmentMigrator] creating file in Media failed for attachment = 427903
2024-06-13 11:30:12,682+0000 AttUpload-Thread-SXRAY-106073 ERROR      [c.a.j.migration.log.ExportLog] 2024-06-13 11:30:12.682361 ERROR SXRAY attachment-export We couldn't export Attachment 'S2942 - Restituer partenaire-2024-02-12-10-12-30-655-0.html' (on Issue SXRAY-38465). Reason: com.atlassian.jira.migration.mediaclient.MediaClientErrorException: request to Media failed. [JCMA 401]
2024-06-13 11:30:12,820+0000 AttUpload-Thread-SXRAY-106078 WARN      [c.a.j.m.workers.attachments.AttachmentMigrator] creating file in Media failed for attachment = 427939
2024-06-13 11:30:12,822+0000 AttUpload-Thread-SXRAY-106078 ERROR      [c.a.j.migration.log.ExportLog] 2024-06-13 11:30:12.822794 ERROR SXRAY attachment-export We couldn't export Attachment 'newman_htmlextra-2024-02-12-10-32-38-736-0.html' (on Issue SXRAY-38468). Reason: com.atlassian.jira.migration.mediaclient.MediaClientErrorException: request to Media failed. [JCMA 401]
2024-06-13 11:30:13,170+0000 AttUpload-Thread-SXRAY-106082 WARN      [c.a.j.m.workers.attachments.AttachmentMigrator] creating file in Media failed for attachment = 428054
2024-06-13 11:30:13,172+0000 AttUpload-Thread-SXRAY-106082 ERROR      [c.a.j.migration.log.ExportLog] 2024-06-13 11:30:13.172722 ERROR SXRAY attachment-export We couldn't export Attachment 'S2942 - Restituer partenaire-2024-02-12-13-53-43-803-0.html' (on Issue SXRAY-38472). Reason: com.atlassian.jira.migration.mediaclient.MediaClientErrorException: request to Media failed. [JCMA 401]
2024-06-13 11:30:13,351+0000 AttUpload-Thread-SXRAY-106050 INFO      [c.a.j.m.workers.attachments.AttachmentMigrator] no read permissions for attachment 428055, ignoring
2024-06-13 11:30:13,693+0000 AttUpload-Thread-SXRAY-106018 WARN      [c.a.j.m.workers.attachments.MediaUploader] Conflicted file upload. FileMediaId = c8da7cb3-840f-4dca-9444-f8a5c6184c01, MigrationId = 599d60bc-79a6-4104-a406-17bcd40844be, ProjectKey = SXRAY
com.atlassian.jira.migration.workers.attachments.MediaUploadConflictException: Conflicted file upload. FileMediaId = c8da7cb3-840f-4dca-9444-f8a5c6184c01, MigrationId = 599d60bc-79a6-4104-a406-17bcd40844be, ProjectKey = SXRAY
	at com.atlassian.jira.migration.workers.attachments.MediaUploader.createUploadIdForFile(MediaUploader.kt:156)
	at com.atlassian.jira.migration.workers.attachments.MediaUploader.uploadMediaItem(MediaUploader.kt:110)
	at com.atlassian.jira.migration.workers.attachments.MediaUploader.access$uploadMediaItem(MediaUploader.kt:29)
	at com.atlassian.jira.migration.workers.attachments.MediaUploader$uploadMediaInParallel$count$1.invoke$lambda$0(MediaUploader.kt:76)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at java.base/java.lang.Thread.run(Unknown Source)
Caused by: com.atlassian.jira.migration.mediaclient.MediaClientErrorException: request to Media failed
	at com.atlassian.jira.migration.mediaclient.MediaClient.execute(MediaClient.kt:194)
	at com.atlassian.jira.migration.mediaclient.MediaClient.createUploadWithFiles(MediaClient.kt:106)
	at com.atlassian.jira.migration.workers.attachments.MediaUploader.createUploadIdForFile(MediaUploader.kt:144)
	... 6 more

      Workaround

      Currently there is no known workaround for this behavior. A workaround will be added here when available

        1. Screenshot 2024-06-20 at 1.00.07 PM.png
          Screenshot 2024-06-20 at 1.00.07 PM.png
          411 kB

            Assignee:
            Pobitra Pradhan (Inactive)
            Reporter:
            Kirti Gupta
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated:
              Resolved: