Uploaded image for project: 'Migration Platform'
  1. Migration Platform
  2. MIG-877

External system import service might create duplicated issues/comments

    • 25
    • Severity 3 - Minor
    • 300

      Issue Summary

      External system import service might create duplicated issues/comments because the thread drops the task in the middle and start to import issues again

      Steps to Reproduce

      1. create a CSV/JSON file
      2. import the file into Jira

      Expected Results

      the issues should be created without any errors

      Actual Results

      the thread drops the task in the middle and start importing again, leading to duplicated tickets

      Workaround

      It's better to reduce the size of the file used for importing. This reduces the import time. Hence the import getting stopped due to node shutting down will be reduced. Also, perform the import during off-business hours to increase the chances of success. Note that off-business hours in a shared infrastructure would mean off-business for the entire region and not just the instance.

            [MIG-877] External system import service might create duplicated issues/comments

            This is an absolute absurd! Force customers to play around their data chunks, instead of just making sure that the node which is running the import must be left alone until the import process is over 🤦🏼‍♂️

            Yevgen Lasman added a comment - This is an absolute absurd! Force customers to play around their data chunks, instead of just making sure that the node which is running the import must be left alone until the import process is over 🤦🏼‍♂️

             pm2 I cannot use JIM because it does not support attachments and comments, or at least it is not [documented here|https://support.atlassian.com/jira-work-management/docs/import-data-from-other-tools-into-jira-work-management/] The tool in ints current state is a joke and cannot be used for serious businesses who were once convinced by an Atlassian sales rep and want to move off another platform to Atlassian Cloud.

            Yevgen Lasman added a comment -  pm2 I cannot use JIM because it does not support attachments and comments, or at least it is not [documented here| https://support.atlassian.com/jira-work-management/docs/import-data-from-other-tools-into-jira-work-management/ ] The tool in ints current state is a joke and cannot be used for serious businesses who were once convinced by an Atlassian sales rep and want to move off another platform to Atlassian Cloud.

            Hello!

            We’re looking to improve the import experience in Jira and are keen to understand how our community is using the Jira Import Module (JIM). If you’ve used JIM to migrate/move data into Jira recently, we’d love to hear about your experience. Please take a few minutes to fill out this survey. Your feedback will help us learn how we can improve your experience in importing data.

            Survey link - https://forms.gle/NYNkmS92r96z42QV9

            Thanks!

            Prashanth M
            Product Manager, Jira Platform

            Prashanth M added a comment - Hello! We’re looking to improve the import experience in Jira and are keen to understand how our community is using the Jira Import Module (JIM). If you’ve used JIM to migrate/move data into Jira recently, we’d love to hear about your experience. Please take a few minutes to  fill out this survey . Your feedback will help us learn how we can improve your experience in importing data. Survey link -  https://forms.gle/NYNkmS92r96z42QV9 Thanks! Prashanth M Product Manager, Jira Platform

            Hi all,

            Although we appreciate that this bug is not great in terms of User Experience, unfortunately it is unlikely that we will address it in the near future.

            We are working on a new Importer for Jira (JWM, JSW, JSM, JPD) to solve many of the existing JIM bugs and provide a better experience while importing data. 

            We have already shipped it in JWM with more features to come soon: [_Import data from other tools into Jira Work Management | Jira Work Management Cloud | Atlassian Support_|https://support.atlassian.com/jira-work-management/docs/import-data-from-other-tools-into-jira-work-management/] . 

            As a result, we decided to close this bug as Won't fix.

            Follow the updates on our public roadmap [_Cloud Roadmap | Atlassian_|https://www.atlassian.com/wac/roadmap/cloud?selectedProduct=jiraWork;jsw;jiraService;jpd&]

            Thank you again for providing valuable feedback to our team!
            Jira Cloud team

            Kamila Czubaj added a comment - Hi all, Although we appreciate that this bug is not great in terms of User Experience, unfortunately it is unlikely that we will address it in the near future. We are working on a new Importer for Jira (JWM, JSW, JSM, JPD) to solve many of the existing JIM bugs and provide a better experience while importing data.   We have already shipped it in JWM with more features to come soon: [_Import data from other tools into Jira Work Management | Jira Work Management Cloud | Atlassian Support_|https://support.atlassian.com/jira-work-management/docs/import-data-from-other-tools-into-jira-work-management/] .   As a result, we decided to close this bug as Won't fix. Follow the updates on our public roadmap [_Cloud Roadmap | Atlassian_|https://www.atlassian.com/wac/roadmap/cloud?selectedProduct=jiraWork;jsw;jiraService;jpd&] Thank you again for providing valuable feedback to our team! Jira Cloud team

            This is impacting our client migration from their external system. No amount of load, import size, or timing changes seem to affect the outcome. It's completely random. 

            Jesse Wisener added a comment - This is impacting our client migration from their external system. No amount of load, import size, or timing changes seem to affect the outcome. It's completely random. 

            Mark Caruso added a comment - - edited

            I think a good (and timely) workaround is for Atlassian to change the behavior such that no attempt is made to restart after the initial failure.  That would sidestep a lot of the harmful and time-consuming after-effects from the creation of duplicate records.  After all, you have to realize this is neither limited to peak-demand times nor large-batch imports.  It can impact anyone and at any time or level of demand.  See comments by Stephen Cox 20/Mar/2023

            I think the "Workaround" currently posted is inaccurate, misleading, and ineffective (i.e., "It's better to reduce the size...".  There is evidence discrediting this advice.

            Mark Caruso added a comment - - edited I think a good (and timely) workaround is for Atlassian to change the behavior such that no attempt is made to restart after the initial failure.  That would sidestep a lot of the harmful and time-consuming after-effects from the creation of duplicate records.  After all, you have to realize this is neither limited to peak-demand times nor large-batch imports.  It can impact anyone and at any time or level of demand.  See comments by Stephen Cox 20/Mar/2023 I think the "Workaround" currently posted is inaccurate, misleading, and ineffective (i.e., "It's better to reduce the size...".  There is evidence discrediting this advice.

            As a customer this has significantly soured the migration process. We developed an export tool for our existing system which generated JSON import format. We tested with batches of issues up to 2000 in size with no problems. We saw the 1GB limit for import files, and knew that our full import was only 160MB, and expected no problems with full size imports. But this problem has reliably manifested when attempting to do full imports of our ~25,000 issues.

            If the migration didn't automatically restart without prompting, we would be able to manually trim our import file and resume from the failure point, but due to this "helpful" automatic restart, instead our imported issues are rendered unusable with duplicated comments. The "workaround" to run the migration out of hours, combined with the requirement to split a single file unattended upgrade into a multiple-part import which requires intervention (or indeed automate this process), imposes additional labour and cost requirements on customers who are trying to migrate to your services.

            Stephen Cox added a comment - As a customer this has significantly soured the migration process. We developed an export tool for our existing system which generated JSON import format. We tested with batches of issues up to 2000 in size with no problems. We saw the 1GB limit for import files, and knew that our full import was only 160MB, and expected no problems with full size imports. But this problem has reliably manifested when attempting to do full imports of our ~25,000 issues. If the migration didn't automatically restart without prompting, we would be able to manually trim our import file and resume from the failure point, but due to this "helpful" automatic restart, instead our imported issues are rendered unusable with duplicated comments. The "workaround" to run the migration out of hours, combined with the requirement to split a single file unattended upgrade into a multiple-part import which requires intervention (or indeed automate this process), imposes additional labour and cost requirements on customers who are trying to migrate to your services.

            Sean Harding added a comment - - edited

            This issue annoys me as a customer. I spent hours, days and weeks migrating data from our legacy system. When talking to support they kept telling me the same solutions that didn't work.

            In the end I had to cut up my data into 500 record samples, and create an automation with clicks to do the import process and report back when it failed. I did find it can be time dependant too, I'm in the UK but seemed I had to do the import in the morning. Almost like my instance is not hosted on European servers?

            We got there in the end but safe to say, this customer/admin was not happy.

            The problem is the import process drops out and restarts the import so if you have 1000 records, if it stops at 300, the 301st record is the 1st record imported again.

            Sean Harding added a comment - - edited This issue annoys me as a customer. I spent hours, days and weeks migrating data from our legacy system. When talking to support they kept telling me the same solutions that didn't work. In the end I had to cut up my data into 500 record samples, and create an automation with clicks to do the import process and report back when it failed. I did find it can be time dependant too, I'm in the UK but seemed I had to do the import in the morning. Almost like my instance is not hosted on European servers? We got there in the end but safe to say, this customer/admin was not happy. The problem is the import process drops out and restarts the import so if you have 1000 records, if it stops at 300, the 301st record is the 1st record imported again.

            Sara Linger added a comment - - edited

            This issue is now happening with imports less than 1000. Upon attempting an import of 750 into One True Tenant, i received "229 issues imported" but over 1300 imported total (with duplicates) .

             

            Sara Linger added a comment - - edited This issue is now happening with imports less than 1000. Upon attempting an import of 750 into One True Tenant, i received "229 issues imported" but over 1300 imported total (with duplicates) .  

            This bug is greatly affecting my team, see https://support.atlassian.com/requests/#!/PCS-146597?src=email. Can this bug be prioritized?

            Sara Linger added a comment - This bug is greatly affecting my team, see https://support.atlassian.com/requests/#!/PCS-146597?src=email. Can this bug be prioritized?

              Unassigned Unassigned
              bad759000199 Carlos Bodini (Inactive)
              Affected customers:
              66 This affects my team
              Watchers:
              34 Start watching this issue

                Created:
                Updated:
                Resolved: