Graph view fails to load when object has very large number of reference on given depth and spikes the CPU

XMLWordPrintable

    • 4
    • Severity 2 - Major

      Issue Summary

      Graph view timeout after 60s when an object has a very large number of references on depth 2 or greater.

      Steps to Reproduce

      Not all circumstances of successful reproduction are known, but the general steps to reproduce this are:

      1. Create an Insight Schema
      2. Create over 20 Objects Types
      3. In some Objects Types, create over 20 attributes that reference other objects and for second level depth, those objects have further references
      4. Go to the Object and open the Object Graph
        1. Click on Configure and change the "Reference depth" to 2
        2. View Object Graph will not load any objects nor their relations.

      Expected Results

      Object graph view should display specific objects and their relations.

      Actual Results

      1. After 60 seconds, the rest api gets cancelled and page remain blank
        1. Screenshot is from Chrome but on Safari also occurs.
      2. Thread Dumps showed that threads are busy iterating through the references. The thread eventually finished but the UI time-out way sooner than that.
        Thread dump 1 captured at 22:17:33
        "http-nio-8080-exec-4" #513 daemon prio=5 os_prio=0 tid=0x00007f60f04bc000 nid=0xe9c runnable [0x00007f51f9ab7000]
           java.lang.Thread.State: RUNNABLE
        	at io.riada.insight.index.model.ObjectTypeIndexImpl.findAllObjectTypes(ObjectTypeIndexImpl.java:49)
        	at io.riada.insight.index.filter.ObjectTypeFilterImpl.findObjectTypeChildrens(ObjectTypeFilterImpl.java:68)
        	at io.riada.insight.index.filter.ObjectTypeFilterImpl.findObjectTypeDescendants(ObjectTypeFilterImpl.java:80)
        	at io.riada.insight.index.filter.ObjectTypeFilterImpl.findObjectTypeDescendants(ObjectTypeFilterImpl.java:82)
        	at io.riada.insight.index.filter.ObjectTypeFilterImpl.findObjectTypeDescendants(ObjectTypeFilterImpl.java:82)
        	at io.riada.insight.index.filter.ObjectTypeFilterImpl.findObjectTypeDescendants(ObjectTypeFilterImpl.java:82)
        	at io.riada.insight.index.filter.ObjectTypeFilterImpl.findObjectTypeDescendants(ObjectTypeFilterImpl.java:82)
        	at io.riada.insight.index.filter.ObjectTypeFilterImpl.findReferencedObjectTypeIdsByObject(ObjectTypeFilterImpl.java:100)
        	at com.riadalabs.jira.plugins.insight.services.core.ObjectServiceImpl.findObjectInboundReferences(ObjectServiceImpl.java:1554)
        	at com.riadalabs.jira.plugins.insight.services.graph.GraphServiceImpl.createReferenceInNodesObj(GraphServiceImpl.java:393)
        	at com.riadalabs.jira.plugins.insight.services.graph.GraphServiceImpl.lambda$null$12(GraphServiceImpl.java:234)
        	at com.riadalabs.jira.plugins.insight.services.graph.GraphServiceImpl$$Lambda$5337/881506147.accept(Unknown Source)
        	at java.util.ArrayList.forEach(ArrayList.java:1257)
        	at com.riadalabs.jira.plugins.insight.services.graph.GraphServiceImpl.lambda$getObjectGraph$13(GraphServiceImpl.java:233)
        	at com.riadalabs.jira.plugins.insight.services.graph.GraphServiceImpl$$Lambda$5321/305885671.accept(Unknown Source)
        	at java.util.stream.Streams$RangeIntSpliterator.forEachRemaining(Streams.java:110)
        	at java.util.stream.IntPipeline$Head.forEach(IntPipeline.java:559)
        	at com.riadalabs.jira.plugins.insight.services.graph.GraphServiceImpl.getObjectGraph(GraphServiceImpl.java:212)
        	at com.riadalabs.jira.plugins.insight.channel.web.api.rest.services.graph.GraphResource.objectGraph(GraphResource.java:105)
        	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        	at java.lang.reflect.Method.invoke(Method.java:498)
        

        Thread keeps running through 22:24:12

        "http-nio-8080-exec-4" #513 daemon prio=5 os_prio=0 tid=0x00007f60f04bc000 nid=0xe9c runnable [0x00007f51f9ab7000]
           java.lang.Thread.State: RUNNABLE
        	at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:174)
        	at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382)
        	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
        	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
        	at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
        	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
        	at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
        	at io.riada.insight.index.filter.ObjectTypeFilterImpl.findObjectTypeChildrens(ObjectTypeFilterImpl.java:73)
        	at io.riada.insight.index.filter.ObjectTypeFilterImpl.findObjectTypeDescendants(ObjectTypeFilterImpl.java:80)
        	at io.riada.insight.index.filter.ObjectTypeFilterImpl.findObjectTypeDescendants(ObjectTypeFilterImpl.java:82)
        	at io.riada.insight.index.filter.ObjectTypeFilterImpl.findObjectTypeDescendants(ObjectTypeFilterImpl.java:82)
        	at io.riada.insight.index.filter.ObjectTypeFilterImpl.findReferencedObjectTypeIdsByObject(ObjectTypeFilterImpl.java:100)
        	at com.riadalabs.jira.plugins.insight.services.core.ObjectServiceImpl.findObjectInboundReferences(ObjectServiceImpl.java:1554)
        	at com.riadalabs.jira.plugins.insight.services.graph.GraphServiceImpl.createReferenceInNodesObj(GraphServiceImpl.java:393)
        	at com.riadalabs.jira.plugins.insight.services.graph.GraphServiceImpl.lambda$null$12(GraphServiceImpl.java:234)
        	at com.riadalabs.jira.plugins.insight.services.graph.GraphServiceImpl$$Lambda$5337/881506147.accept(Unknown Source)
        	at java.util.ArrayList.forEach(ArrayList.java:1257)
        	at com.riadalabs.jira.plugins.insight.services.graph.GraphServiceImpl.lambda$getObjectGraph$13(GraphServiceImpl.java:233)
        	at com.riadalabs.jira.plugins.insight.services.graph.GraphServiceImpl$$Lambda$5321/305885671.accept(Unknown Source)
        	at java.util.stream.Streams$RangeIntSpliterator.forEachRemaining(Streams.java:110)
        	at java.util.stream.IntPipeline$Head.forEach(IntPipeline.java:559)
        	at com.riadalabs.jira.plugins.insight.services.graph.GraphServiceImpl.getObjectGraph(GraphServiceImpl.java:212)
        	at com.riadalabs.jira.plugins.insight.channel.web.api.rest.services.graph.GraphResource.objectGraph(GraphResource.java:105)
        	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        	at java.lang.reflect.Method.invoke(Method.java:498)
        

      While on client side session times out, the long running thread on Jira spike the CPU and ultimately lead Jira to crash:

      https-jsse-nio-8443-exec-4
      State:RUNNABLE
      CPU usage:99.9%
      Running for: 0:44.37
      
      io.riada.insight.index.filter.ObjectTypeAttributeFilterImpl.findObjectTypeAttributes(ObjectTypeAttributeFilterImpl.java:311)
      com.riadalabs.jira.plugins.insight.services.predicate.attribute.ReferenceTypeOutboundPredicate.isMatchedAnyValue(ReferenceTypeOutboundPredicate.java:51)
      com.riadalabs.jira.plugins.insight.services.predicate.attribute.ReferenceTypeOutboundPredicate.testObjectBean(ReferenceTypeOutboundPredicate.java:42)
      com.riadalabs.jira.plugins.insight.services.predicate.ObjectBeanPredicateBase.test(ObjectBeanPredicateBase.java:21)
      com.riadalabs.jira.plugins.insight.services.predicate.ObjectBeanPredicateBase.test(ObjectBeanPredicateBase.java:11)
      com.riadalabs.jira.plugins.insight.services.predicate.CombinedPredicate.test(CombinedPredicate.java:86)
      com.riadalabs.jira.plugins.insight.services.predicate.CombinedPredicate.test(CombinedPredicate.java:16)
      com.riadalabs.jira.plugins.insight.services.core.ObjectFilterManagerImpl.filterObjectBeansInThread(ObjectFilterManagerImpl.kt:125)
      com.riadalabs.jira.plugins.insight.services.core.ObjectFilterManagerImpl.findObjectsBasedOnFilter(ObjectFilterManagerImpl.kt:84)
      com.riadalabs.jira.plugins.insight.services.core.ObjectFilterManagerImpl.findObjectsByFilter(ObjectFilterManagerImpl.kt:62)
      com.riadalabs.jira.plugins.insight.services.core.ObjectFilterManagerImpl.findObjectsByFilter(ObjectFilterManagerImpl.kt:53)
      com.riadalabs.jira.plugins.insight.services.core.ObjectServiceImpl.findInboundReferencedObjectAttributes(ObjectServiceImpl.java:2197)
      com.riadalabs.jira.plugins.insight.services.core.ObjectServiceImpl.findObjectInboundReferences(ObjectServiceImpl.java:1553)
      com.riadalabs.jira.plugins.insight.services.graph.GraphServiceImpl.createReferenceInEdgesObj(GraphServiceImpl.java:442)
      com.riadalabs.jira.plugins.insight.services.graph.GraphServiceImpl.lambda$null$12(GraphServiceImpl.java:235)
      com.riadalabs.jira.plugins.insight.services.graph.GraphServiceImpl$$Lambda$6088/0x0000000805aa6840.accept(Unknown Source)
      java.util.ArrayList.forEach(java.base@11.0.13/Unknown Source)
      

      Workaround

      Currently there is no known workaround for this behavior. A workaround will be added here when available

        1. Request Canceled.png
          440 kB
          Pablo Bartolome

            Assignee:
            Kunal Kanojia
            Reporter:
            Pablo Bartolome (Inactive)
            Votes:
            2 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: