Uploaded image for project: 'Jira Service Management Server and Data Center'
  1. Jira Service Management Server and Data Center
  2. JSDSERVER-11169

Exporting a large Object Type does not generated a CSV file

    XMLWordPrintable

Details

    Description

      Issue Summary

      When exporting objects to CSV (Data Consistent or User Friendly) the CSV will not be generated and nothing happens on the browser due to either number of references objects have or number of data on Objects
      Issue is reproducible with Super Admin User - so no permission issue here

      Steps to Reproduce

      In order to reproduce this, we need a big number of references. An easy way to reproduce this is by using the NVD data.

      1. Set up NVD integration and sync from the CVE-2020, CVE-2019 and CVE-2018 databases
        • It will create over 34k Objects within the Vulnerability Object Type. Enough to reproduce the issue.
      2. Attempt to export ALL Vulnerability objects.
        • Select "User Friendly"

      Expected Results

      A CSV file should be generated with all Objects attributes

      Actual Results

      • Nothing will happen. No CSV will be generated
      • After 60 seconds, the POST request returns HTTP 200 but no body is returned.
      • Via Postman, results are returned after 70 seconds but browser timed out before that
      • Thread Dumps showed that threads are busy for roughly a minute. The thread eventually finished <= 1 minute but the request seems to be executing.

      17:28:20

      "http-nio-8080-exec-17 url: /rest/insight/1.0/objecttypeexport/35 url:/rest/insight/1...bjecttypeexport/35 username:admin" #261 daemon prio=5 os_prio=0 cpu=102058.73ms elapsed=16407.91s tid=0x00007f18c0146000 nid=0x2b3 runnable  [0x00007f189b0f5000]
         java.lang.Thread.State: RUNNABLE
      	at com.google.common.cache.LocalCache$Segment.getEntry(LocalCache.java:2673)
      	at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2027)
      	at com.google.common.cache.LocalCache.get(LocalCache.java:3952)
      	at com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4871)
      	at com.atlassian.cache.memory.DelegatingCache.get(DelegatingCache.java:178)
      ...
      	at com.atlassian.jira.datetime.DateTimeFormatterImpl.forUser(DateTimeFormatterImpl.java:61)
      	at com.riadalabs.jira.plugins.insight.common.tools.InsightDateTimeFormatterInJira.getDateTimeFormatter(InsightDateTimeFormatterInJira.java:111)
      	at com.riadalabs.jira.plugins.insight.common.tools.InsightDateTimeFormatterInJira.formatDateTimeToString(InsightDateTimeFormatterInJira.java:92)
      	at com.riadalabs.jira.plugins.insight.services.imports.ObjectExportServiceImpl.convertDefault(ObjectExportServiceImpl.java:389)
      	at com.riadalabs.jira.plugins.insight.services.imports.ObjectExportServiceImpl.convert(ObjectExportServiceImpl.java:244)
      	at com.riadalabs.jira.plugins.insight.services.imports.ObjectExportServiceImpl.lambda$writeObjects$2(ObjectExportServiceImpl.java:176)
      	at com.riadalabs.jira.plugins.insight.services.imports.ObjectExportServiceImpl$$Lambda$2743/0x0000000102aad840.apply(Unknown Source)
      	at java.util.stream.ReferencePipeline$3$1.accept(java.base@11.0.13/ReferencePipeline.java:195)
      	at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(java.base@11.0.13/ArrayList.java:1655)
      	at java.util.stream.AbstractPipeline.copyInto(java.base@11.0.13/AbstractPipeline.java:484)
      	at java.util.stream.AbstractPipeline.wrapAndCopyInto(java.base@11.0.13/AbstractPipeline.java:474)
      	at java.util.stream.AbstractPipeline.evaluate(java.base@11.0.13/AbstractPipeline.java:550)
      	at java.util.stream.AbstractPipeline.evaluateToArrayNode(java.base@11.0.13/AbstractPipeline.java:260)
      	at java.util.stream.ReferencePipeline.toArray(java.base@11.0.13/ReferencePipeline.java:517)
      	at com.riadalabs.jira.plugins.insight.services.imports.ObjectExportServiceImpl.writeObjects(ObjectExportServiceImpl.java:176)
      	at com.riadalabs.jira.plugins.insight.services.imports.ObjectExportServiceImpl.exportObjects(ObjectExportServiceImpl.java:131)
      	at com.riadalabs.jira.plugins.insight.channel.web.api.rest.services.objecttype.ObjectTypeExportResource.lambda$exportObjectTypeFilteredObjects$0(ObjectTypeExportResource.java:90)
      	at com.riadalabs.jira.plugins.insight.channel.web.api.rest.services.objecttype.ObjectTypeExportResource$$Lambda$2738/0x0000000102aac440.write(Unknown Source)
      

      17:29:13

      "http-nio-8080-exec-17 url: /rest/insight/1.0/objecttypeexport/35 url:/rest/insight/1...bjecttypeexport/35 username:admin" #261 daemon prio=5 os_prio=0 cpu=142492.61ms elapsed=16461.65s tid=0x00007f18c0146000 nid=0x2b3 runnable  [0x00007f189b0f5000]
         java.lang.Thread.State: RUNNABLE
      	at org.apache.commons.dbcp2.AbandonedTrace.init(AbandonedTrace.java:69)
      	at org.apache.commons.dbcp2.AbandonedTrace.<init>(AbandonedTrace.java:59)
      	at org.apache.commons.dbcp2.DelegatingResultSet.<init>(DelegatingResultSet.java:83)
      	at org.apache.commons.dbcp2.DelegatingResultSet.wrapResultSet(DelegatingResultSet.java:109)
      	at org.apache.commons.dbcp2.DelegatingPreparedStatement.executeQuery(DelegatingPreparedStatement.java:83)
      ..
      	at com.riadalabs.jira.plugins.insight.services.core.dal.dao.impl.InsightActiveObject.stream(InsightActiveObject.java:112)
      	at com.riadalabs.jira.plugins.insight.services.core.dal.dao.impl.ObjectAttributeDaoImpl.findObjectAttributeAOs(ObjectAttributeDaoImpl.java:178)
      	at com.riadalabs.jira.plugins.insight.services.core.dal.impl.ObjectAttributeDalImpl.findObjectAttributeBeans(ObjectAttributeDalImpl.java:384)
      	at com.riadalabs.jira.plugins.insight.services.core.ObjectManagerInJira.loadObject(ObjectManagerInJira.java:91)
      	at com.riadalabs.jira.plugins.insight.services.core.IQLServiceImpl.lambda$findObjects$2(IQLServiceImpl.java:100)
      	at com.riadalabs.jira.plugins.insight.services.core.IQLServiceImpl$$Lambda$1744/0x0000000102469c40.apply(Unknown Source
      

      17:29:24

      "http-nio-8080-exec-17" #261 daemon prio=5 os_prio=0 cpu=142730.97ms elapsed=16472.40s tid=0x00007f18c0146000 nid=0x2b3 waiting on condition  [0x00007f189b100000]
         java.lang.Thread.State: WAITING (parking)
      	at jdk.internal.misc.Unsafe.park(java.base@11.0.13/Native Method)
      	- parking to wait for  <0x0000000080fe1860> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
      	at java.util.concurrent.locks.LockSupport.park(java.base@11.0.13/LockSupport.java:194)
      	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(java.base@11.0.13/AbstractQueuedSynchronizer.java:2081)
      	at java.util.concurrent.LinkedBlockingQueue.take(java.base@11.0.13/LinkedBlockingQueue.java:433)
      	at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:108)
      	at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:33)
      	at java.util.concurrent.ThreadPoolExecutor.getTask(java.base@11.0.13/ThreadPoolExecutor.java:1054)
      

      Workaround

      Use Insight searching for objects

      Select Insight > Object search > Run the IQL and Export.

      Attachments

        1. Export.png
          Export.png
          383 kB
        2. Har.png
          Har.png
          169 kB
        3. Postman_result.png
          Postman_result.png
          538 kB

        Issue Links

          Activity

            People

              c8bcca445054 Benjamin Suess
              9f03f4114502 Pablo Bartolome
              Votes:
              9 Vote for this issue
              Watchers:
              16 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Backbone Issue Sync