Uploaded image for project: 'Jira Data Center'
  1. Jira Data Center
  2. JRASERVER-65877

JIRA crashes running out of Tomcat Threads due to broken external gadget feeds

    XMLWordPrintable

Details

    Description

      Summary

      JIRA crashed running out of Tomcat threads.
      All threads are held waiting for Gadget feed read and JIRA freezes.

      Environment

      • JIRA integration with external applications: eg: Confluence, Bamboo
      • Gadget feed subscription exists from JIRA gadgets to get feeds from the external apps.
      • Issues noticed within the connection between JIRA and the other apps, or if the other apps are having a performance issue or are not available.

      Expected Results

      • If JIRA Gadget is not able to connect, it should timeout and the threads are no longer in waiting state

      Actual Results

      Threads are stuck for ever and no timeout is happening, check below thread dump:

      http-nio-8080-exec-150" #1556 daemon prio=5 os_prio=0 tid=0x00007f614c1cb000 nid=0xe86f waiting on condition [0x00007f60c63f9000]
         java.lang.Thread.State: WAITING (parking)
              at sun.misc.Unsafe.park(Native Method)
              - parking to wait for  <0x00000006caab42f0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
              at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
              at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
              at org.codehaus.httpcache4j.cache.Mutex.acquire(Mutex.java:45)
              at org.codehaus.httpcache4j.cache.HTTPCache.doCachedRequest(HTTPCache.java:103)
              at org.codehaus.httpcache4j.cache.HTTPCache.doCachedRequest(HTTPCache.java:80)
              at com.atlassian.gadgets.directory.internal.impl.GadgetFeedReaderImpl.<init>(GadgetFeedReaderImpl.java:43)
              at com.atlassian.gadgets.directory.internal.impl.GadgetFeedReaderFactoryImpl.getFeedReader(GadgetFeedReaderFactoryImpl.java:28)
      

      Eventually all Tomcat threads are waiting and JIRA crashes.
      Also many of the following warnings are thrown in the logs:

      017-08-28 11:45:43,632 http-nio-8080-exec-483 WARN emp001627 705x16673608x10 vq636g 10.10.17.132,127.0.0.1 /plugins/servlet/gadgets/ifr [c.a.g.directory.internal.GadgetFeedsSpecProvider] Unable to parse feed from: http://<HOSTNAME>:<Port>/bamboo/rest/gadgets/1.0/g/feed
      

      Workaround

      Effort should be made to address the Confluence or Bamboo unavailability problem as that is the trigger for this bug. In the short term, the offending gadget feed may be temporarily removed from JIRA gadget configuration.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              sabdelfattah Sherif Abdelfattah (Inactive)
              Votes:
              1 Vote for this issue
              Watchers:
              18 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: