Uploaded image for project: 'Jira Service Management Data Center'
  1. Jira Service Management Data Center
  2. JSDSERVER-10850

Graph view fails to load when object has very large number of reference on given depth and spikes the CPU

      Issue Summary

      Graph view timeout after 60s when an object has a very large number of references on depth 2 or greater.

      Steps to Reproduce

      Not all circumstances of successful reproduction are known, but the general steps to reproduce this are:

      1. Create an Insight Schema
      2. Create over 20 Objects Types
      3. In some Objects Types, create over 20 attributes that reference other objects and for second level depth, those objects have further references
      4. Go to the Object and open the Object Graph
        1. Click on Configure and change the "Reference depth" to 2
        2. View Object Graph will not load any objects nor their relations.

      Expected Results

      Object graph view should display specific objects and their relations.

      Actual Results

      1. After 60 seconds, the rest api gets cancelled and page remain blank
        1. Screenshot is from Chrome but on Safari also occurs.
      2. Thread Dumps showed that threads are busy iterating through the references. The thread eventually finished but the UI time-out way sooner than that.
        Thread dump 1 captured at 22:17:33
        "http-nio-8080-exec-4" #513 daemon prio=5 os_prio=0 tid=0x00007f60f04bc000 nid=0xe9c runnable [0x00007f51f9ab7000]
           java.lang.Thread.State: RUNNABLE
        	at io.riada.insight.index.model.ObjectTypeIndexImpl.findAllObjectTypes(ObjectTypeIndexImpl.java:49)
        	at io.riada.insight.index.filter.ObjectTypeFilterImpl.findObjectTypeChildrens(ObjectTypeFilterImpl.java:68)
        	at io.riada.insight.index.filter.ObjectTypeFilterImpl.findObjectTypeDescendants(ObjectTypeFilterImpl.java:80)
        	at io.riada.insight.index.filter.ObjectTypeFilterImpl.findObjectTypeDescendants(ObjectTypeFilterImpl.java:82)
        	at io.riada.insight.index.filter.ObjectTypeFilterImpl.findObjectTypeDescendants(ObjectTypeFilterImpl.java:82)
        	at io.riada.insight.index.filter.ObjectTypeFilterImpl.findObjectTypeDescendants(ObjectTypeFilterImpl.java:82)
        	at io.riada.insight.index.filter.ObjectTypeFilterImpl.findObjectTypeDescendants(ObjectTypeFilterImpl.java:82)
        	at io.riada.insight.index.filter.ObjectTypeFilterImpl.findReferencedObjectTypeIdsByObject(ObjectTypeFilterImpl.java:100)
        	at com.riadalabs.jira.plugins.insight.services.core.ObjectServiceImpl.findObjectInboundReferences(ObjectServiceImpl.java:1554)
        	at com.riadalabs.jira.plugins.insight.services.graph.GraphServiceImpl.createReferenceInNodesObj(GraphServiceImpl.java:393)
        	at com.riadalabs.jira.plugins.insight.services.graph.GraphServiceImpl.lambda$null$12(GraphServiceImpl.java:234)
        	at com.riadalabs.jira.plugins.insight.services.graph.GraphServiceImpl$$Lambda$5337/881506147.accept(Unknown Source)
        	at java.util.ArrayList.forEach(ArrayList.java:1257)
        	at com.riadalabs.jira.plugins.insight.services.graph.GraphServiceImpl.lambda$getObjectGraph$13(GraphServiceImpl.java:233)
        	at com.riadalabs.jira.plugins.insight.services.graph.GraphServiceImpl$$Lambda$5321/305885671.accept(Unknown Source)
        	at java.util.stream.Streams$RangeIntSpliterator.forEachRemaining(Streams.java:110)
        	at java.util.stream.IntPipeline$Head.forEach(IntPipeline.java:559)
        	at com.riadalabs.jira.plugins.insight.services.graph.GraphServiceImpl.getObjectGraph(GraphServiceImpl.java:212)
        	at com.riadalabs.jira.plugins.insight.channel.web.api.rest.services.graph.GraphResource.objectGraph(GraphResource.java:105)
        	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        	at java.lang.reflect.Method.invoke(Method.java:498)
        

        Thread keeps running through 22:24:12

        "http-nio-8080-exec-4" #513 daemon prio=5 os_prio=0 tid=0x00007f60f04bc000 nid=0xe9c runnable [0x00007f51f9ab7000]
           java.lang.Thread.State: RUNNABLE
        	at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:174)
        	at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382)
        	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
        	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
        	at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
        	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
        	at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
        	at io.riada.insight.index.filter.ObjectTypeFilterImpl.findObjectTypeChildrens(ObjectTypeFilterImpl.java:73)
        	at io.riada.insight.index.filter.ObjectTypeFilterImpl.findObjectTypeDescendants(ObjectTypeFilterImpl.java:80)
        	at io.riada.insight.index.filter.ObjectTypeFilterImpl.findObjectTypeDescendants(ObjectTypeFilterImpl.java:82)
        	at io.riada.insight.index.filter.ObjectTypeFilterImpl.findObjectTypeDescendants(ObjectTypeFilterImpl.java:82)
        	at io.riada.insight.index.filter.ObjectTypeFilterImpl.findReferencedObjectTypeIdsByObject(ObjectTypeFilterImpl.java:100)
        	at com.riadalabs.jira.plugins.insight.services.core.ObjectServiceImpl.findObjectInboundReferences(ObjectServiceImpl.java:1554)
        	at com.riadalabs.jira.plugins.insight.services.graph.GraphServiceImpl.createReferenceInNodesObj(GraphServiceImpl.java:393)
        	at com.riadalabs.jira.plugins.insight.services.graph.GraphServiceImpl.lambda$null$12(GraphServiceImpl.java:234)
        	at com.riadalabs.jira.plugins.insight.services.graph.GraphServiceImpl$$Lambda$5337/881506147.accept(Unknown Source)
        	at java.util.ArrayList.forEach(ArrayList.java:1257)
        	at com.riadalabs.jira.plugins.insight.services.graph.GraphServiceImpl.lambda$getObjectGraph$13(GraphServiceImpl.java:233)
        	at com.riadalabs.jira.plugins.insight.services.graph.GraphServiceImpl$$Lambda$5321/305885671.accept(Unknown Source)
        	at java.util.stream.Streams$RangeIntSpliterator.forEachRemaining(Streams.java:110)
        	at java.util.stream.IntPipeline$Head.forEach(IntPipeline.java:559)
        	at com.riadalabs.jira.plugins.insight.services.graph.GraphServiceImpl.getObjectGraph(GraphServiceImpl.java:212)
        	at com.riadalabs.jira.plugins.insight.channel.web.api.rest.services.graph.GraphResource.objectGraph(GraphResource.java:105)
        	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        	at java.lang.reflect.Method.invoke(Method.java:498)
        

      While on client side session times out, the long running thread on Jira spike the CPU and ultimately lead Jira to crash:

      https-jsse-nio-8443-exec-4
      State:RUNNABLE
      CPU usage:99.9%
      Running for: 0:44.37
      
      io.riada.insight.index.filter.ObjectTypeAttributeFilterImpl.findObjectTypeAttributes(ObjectTypeAttributeFilterImpl.java:311)
      com.riadalabs.jira.plugins.insight.services.predicate.attribute.ReferenceTypeOutboundPredicate.isMatchedAnyValue(ReferenceTypeOutboundPredicate.java:51)
      com.riadalabs.jira.plugins.insight.services.predicate.attribute.ReferenceTypeOutboundPredicate.testObjectBean(ReferenceTypeOutboundPredicate.java:42)
      com.riadalabs.jira.plugins.insight.services.predicate.ObjectBeanPredicateBase.test(ObjectBeanPredicateBase.java:21)
      com.riadalabs.jira.plugins.insight.services.predicate.ObjectBeanPredicateBase.test(ObjectBeanPredicateBase.java:11)
      com.riadalabs.jira.plugins.insight.services.predicate.CombinedPredicate.test(CombinedPredicate.java:86)
      com.riadalabs.jira.plugins.insight.services.predicate.CombinedPredicate.test(CombinedPredicate.java:16)
      com.riadalabs.jira.plugins.insight.services.core.ObjectFilterManagerImpl.filterObjectBeansInThread(ObjectFilterManagerImpl.kt:125)
      com.riadalabs.jira.plugins.insight.services.core.ObjectFilterManagerImpl.findObjectsBasedOnFilter(ObjectFilterManagerImpl.kt:84)
      com.riadalabs.jira.plugins.insight.services.core.ObjectFilterManagerImpl.findObjectsByFilter(ObjectFilterManagerImpl.kt:62)
      com.riadalabs.jira.plugins.insight.services.core.ObjectFilterManagerImpl.findObjectsByFilter(ObjectFilterManagerImpl.kt:53)
      com.riadalabs.jira.plugins.insight.services.core.ObjectServiceImpl.findInboundReferencedObjectAttributes(ObjectServiceImpl.java:2197)
      com.riadalabs.jira.plugins.insight.services.core.ObjectServiceImpl.findObjectInboundReferences(ObjectServiceImpl.java:1553)
      com.riadalabs.jira.plugins.insight.services.graph.GraphServiceImpl.createReferenceInEdgesObj(GraphServiceImpl.java:442)
      com.riadalabs.jira.plugins.insight.services.graph.GraphServiceImpl.lambda$null$12(GraphServiceImpl.java:235)
      com.riadalabs.jira.plugins.insight.services.graph.GraphServiceImpl$$Lambda$6088/0x0000000805aa6840.accept(Unknown Source)
      java.util.ArrayList.forEach(java.base@11.0.13/Unknown Source)
      

      Workaround

      Currently there is no known workaround for this behavior. A workaround will be added here when available

            [JSDSERVER-10850] Graph view fails to load when object has very large number of reference on given depth and spikes the CPU

            There are no comments yet on this issue.

              kkanojia Kunal Kanojia
              9f03f4114502 Pablo Bartolome
              Affected customers:
              2 This affects my team
              Watchers:
              4 Start watching this issue

                Created:
                Updated:
                Resolved: