Uploaded image for project: 'Jira Service Management Data Center'
  1. Jira Service Management Data Center
  2. JSDSERVER-8451

Longer time needed for imports in Insight version 8.6.12 and above

XMLWordPrintable

      Atlassian Update – 17 March 2022

      Hi everyone,

      We are pleased to let you know that a bug fix for the import performance regression is now available in Jira Service Management Data Center versions 4.20.7, 4.22.1 and the Insight Marketplace app version 8.9.10.

      This performance fix increases the import speed when there are object type hierarchies in the schemas. We’ve seen some significant improvements and look forward to having customers put it to use!

      We're initially releasing the fix behind a feature flag for early adopters to experiment with. We’re ready to provide a rollback for you if anything unexpected occurs. Follow the instructions below to enable the flag.

      Kind regards,

      Craig Shannon

      Jira Service Management, Data Center & Server

      Instructions to enable the feature flags in 4.20.7 / 4.22.1

      There are two feature flags which will turn on this new behavior:

      insight.index.object.type.children

      insight.index.object.type.reindex.on.failure

      To activate them, see Enable Dark Feature in Jira.

      Once you enable those feature flags, you need to run a reindex as the object type structure in the index changes to improve the lookup performance of the object type hierarchy. You can do it from <JIRA URL>/secure/admin/InsightPluginShowIndex.jspa. Make sure to do a clean reindex to ensure that the old object type index file on disk is removed.

      If you encounter any issues, disabling the feature flags will revert back the behavior and there is no need to perform a reindex again.

      Issue Summary

      Much longer time needed for imports in Insight version 8.6.12 and above. This also causes CPU spike up to 99% and the import process may seem halted on the UI.

      Steps to Reproduce

      1. With Insight 8.6.11, set up and import from the CVE-Database as per this documentation. For testing purpose, we choose CVE 2020.
      2. Note the time taken to complete the import.
      3. Upgrade to Insight 8.6.12 or any newer version, create another schema and perform the exact same import.
      4. Note the time taken to complete the import and the CPU usage.

      Expected Results

      The time taken to complete the import in both versions is almost the same

      Actual Results

      • It takes much longer to complete the import in Insight version 8.6.12 and above. Based on the comparison of the logs, we can see that the step to decide actions for all objects is taking more time in the newer Insight versions (103ms vs 401608ms).
      • CPU spike can also be observed along the import process especially on step 3 (decide actions for all objects) and as soon as the import is done, CPU usage is back to normal. Due to the much longer process, the impact of CPU spike is amplified.
        In some cases, the import seems halted for hours on step 3 and CPU usage is constantly at its peak. The import has to be terminated manually by canceling the import, disabling and re-enabling Insight or restart Jira.

        Insight 8.6.11

        2021-07-08 13:47:39,598 [insight-InsightThreadGroup-worker-thread-0] | NVD>>> Mapped all data to internal objects in 62805 ms
2021-07-08 13:47:39,702 [insight-InsightThreadGroup-worker-thread-0] | NVD>>> Decided actions for all objects in 103 ms
2021-07-08 13:47:39,702 [insight-InsightThreadGroup-worker-thread-0] | NVD>>> Objects to import 38801
...
2021-07-08 13:55:50,762 [insight-InsightThreadGroup-worker-thread-0] | Handled 32200 objects with outbound references
2021-07-08 13:55:50,763 [insight-InsightThreadGroup-worker-thread-0] | Finished updating object references in 250704 ms
2021-07-08 13:55:50,763 [insight-InsightThreadGroup-worker-thread-0] | NVD>>> Finished import with id 18 of type insight-cve-import at time 2021-07-08T13:55:50.763233. Import took 660570 ms

      Insight 8.6.12 and above

      2021-07-08 11:26:05,055 [insight-InsightThreadGroup-worker-thread-4] | NVD>>> Mapped all data to internal objects in 59958 ms
2021-07-08 11:32:46,664 [insight-InsightThreadGroup-worker-thread-4] | NVD>>> Decided actions for all objects in 401608 ms
2021-07-08 11:32:46,664 [insight-InsightThreadGroup-worker-thread-4] | NVD>>> Objects to import 38254
...
2021-07-08 11:40:27,822 [insight-InsightThreadGroup-worker-thread-4] | Handled 31326 objects with outbound references
2021-07-08 11:40:27,823 [insight-InsightThreadGroup-worker-thread-4] | Finished updating object references in 242887 ms
2021-07-08 11:40:27,823 [insight-InsightThreadGroup-worker-thread-4] | NVD>>> Finished import with id 17 of type insight-cve-import at time 2021-07-08T11:40:27.823297. Import took 1040906 ms

      No blocked threads but attached threads below are often accompanied with high CPU usage.

      Workaround

      Currently there is no known workaround for this behavior. A workaround will be added here when available

        1. AttributeValuePredicateBase.txt
          4 kB
        2. InsightIndexBase.txt
          3 kB
        3. Screen Shot 2022-03-23 at 7.53.03 pm.png
          Screen Shot 2022-03-23 at 7.53.03 pm.png
          33 kB
        4. Screen Shot 2022-03-23 at 8.29.33 pm.png
          Screen Shot 2022-03-23 at 8.29.33 pm.png
          504 kB

          Form Name

            [JSDSERVER-8451] Longer time needed for imports in Insight version 8.6.12 and above

            Dhanapal Mohanasamy made changes -
            Remote Link New: This issue links to "Page (Confluence)" [ 805105 ]
            Edson Araujo made changes -
            Remote Link Original: This issue links to "JSMDC-11496 (Bulldog)" [ 621047 ] New: This issue links to "JSMDC-11496 (JIRA Server (Bulldog))" [ 621047 ]
            Craig Shannon made changes -
            Remote Link New: This issue links to "Page (Confluence)" [ 726234 ]
            g8sy (Inactive) made changes -
            Remote Link New: This issue links to "Page (Confluence)" [ 725921 ]
            Garima Katyal made changes -
            Remote Link Original: This issue links to "Page (Confluence)" [ 706953 ]
            Craig Shannon made changes -
            Remote Link New: This issue links to "Page (Confluence)" [ 706953 ]
            Alex Cooksey made changes -
            Remote Link New: This issue links to "Page (Confluence)" [ 659099 ]
            Mike Robertson made changes -
            Comment [
            Hello folks! Would the "object type hierarchies" mentioned in the description of this Jira refer to a structure like this?
            {code}
            CustomServers # Abstract object type with two child object types. Attributes contain outbound references to other object types
            ├── Physicals # Child object type of CustomServers
            └── Virtuals # Child object type of CustomServers
            {code}

            {quote}
            This performance fix increases the import speed when there are object type hierarchies in the schemas.
            {quote}

            ]
            Marc Dacanay made changes -
            Remote Link New: This issue links to "Page (Confluence)" [ 634024 ]
            Josh Steckler (Inactive) made changes -
            Remote Link New: This issue links to "Page (Confluence)" [ 633845 ]

              Unassigned Unassigned
              michin Michelle Chin
              Affected customers:
              35 This affects my team
              Watchers:
              45 Start watching this issue

                Created:
                Updated:
                Resolved: