Uploaded image for project: 'Jira Cloud'
  1. Jira Cloud
  2. JSWCLOUD-13632

JIRA Software Manage Boards and viewing boards can timeout due to analysing JQL

XMLWordPrintable

      NOTE: This bug report is for JIRA Software Cloud. Using JIRA Software Server? See the corresponding bug report.

      Summary

      Environment

      Reproduced with the following instance size:

          ___ Database Statistics ____________________
      
               Issues                                        : 330776
               Projects                                      : 187
               Custom Fields                                 : 180
               Workflows                                     : 306
               Users                                         : 3719
               Groups                                        : 2060
               Attachments                                   : 118585
               Comments                                      : 265597
      

      It could be:

      • 52 Rapid Boards were using the issue in parent() function
      • Large number of boards and/or complex JQL used in the board filters.

      Steps to Reproduce

      1. Use a large instance, such as above.
      2. Install a third-party plugin that provides additional functionality. This was reproduced with J-Tricks, using issue in parent().
      3. Access the Manage Boards page, or browse to projects.

      It is important to note that this affected rapid boards which did not use that third-party JQL, as com.atlassian.greenhopper.web.sidebar.AgileBoardScopeFilterContextProvider.getContext (the JIRA Software code called from the JIRA Core sidebar) executes the same code to perform project lookups by analysing JQL queries (analysing, not executing). The problem is due to the way the third-party plugin is designed is that the operand executes when it's analysed. Specifically this is not conclusively caused by third-party plugins.

      Expected Results

      • The Manage Boards page does not time out, and the project page loads within an acceptable time.
      • Viewing Rapid Boards responds in a reasonable time frame.

      Actual Results

      The ajax timeout kicks in and the page times out whilst waiting for rest/greenhopper/1.0/rapidviews/viewsData to return results, or the board can take > 1 minute to load.

      Notes

      Accessing that endpoint outside of JIRA (and thus bypassing ajax) in this instance returned within 1.2mins.

      This appears to be caused by QueryToProjectMapper.getProjectContextIdsForQuery which uses JIRA Core code to lookup the QueryContext. Due to the way the lookup occurs it's possible that JQL may be executed and this times out the board.

      A sample snapshot of the problem has been attached.

      Verification

      The problem occurs in multiple areas of JIRA Software - it's most evident in manage boards however can occur on the Rapid Boards.

      For Manage Boards:

      1. Call /rest/greenhopper/1.0/rapidviews/viewsData will be slow (40+ seconds, in some cases will not return anything as the AJAX timeout terminates the request).
      2. Class com.atlassian.greenhopper.web.rapid.view.RapidViewListResource.getViewsAndConfigModel will be present in the stacktrace for long-running threads:
        "http-nio-80-exec-45" #8007 daemon prio=5 os_prio=0 tid=0x000000003cac0000 nid=0x19dc runnable [0x000000027765a000]
        ...
                at com.atlassian.greenhopper.web.rapid.view.RapidViewProjectHelper.getRapidViewProjectList(RapidViewProjectHelper.java:58)
        ...
                at com.atlassian.greenhopper.web.rapid.view.RapidViewListHelper.buildListAndConfigModel(RapidViewListHelper.java:105)
        ...
                at com.atlassian.greenhopper.web.rapid.view.RapidViewListResource.getViewsAndConfigModel(RapidViewListResource.java:74)
        ...
        

      For Rapid Boards:

      1. Many threads will be waiting on a response to AbstractFuture$Sync as per the attached screenshot. These threads are waiting on the populattion of the cache that FieldPossibleValues.getProjectIdsForQuery queries.
      2. There will be one executing thread in QueryToProjectMapper$RapidViewCacheLoader.load, populating the cache. Below is a sample stack trace:
        "http-bio-8081-exec-862 uri:/plugins/servlet/projects/ABC username:captain.planet" #1247 daemon prio=5 tid=0x00007f210832f000 nid=0x1aca waiting on condition [0x00007f1cea179000]
           java.lang.Thread.State: RUNNABLE
        	at com.atlassian.jira.plugins.issue.create.context.fields.helpers.CustomCFTypeInputMapper$class.getFieldType(CustomCFTypeInputMapper.scala:32)
        	at com.atlassian.jira.plugins.issue.create.context.fields.custom.GenericTextCFTypeJQLMapper.getFieldType(GenericTextCFTypeJQLMapper.scala:13)
        	at com.atlassian.jira.plugins.issue.create.context.fields.helpers.CustomCFTypeInputMapper$class.matchesClause(CustomCFTypeInputMapper.scala:49)
        	at com.atlassian.jira.plugins.issue.create.context.fields.custom.GenericTextCFTypeJQLMapper.matchesClause(GenericTextCFTypeJQLMapper.scala:22)
        	at com.atlassian.jira.plugins.issue.create.context.InputParamsBuilder$$anonfun$mapperFor$1.apply(InputParamsBuilder.scala:111)
        	at com.atlassian.jira.plugins.issue.create.context.InputParamsBuilder$$anonfun$mapperFor$1.apply(InputParamsBuilder.scala:111)
        	at scala.collection.Iterator$class.find(Iterator.scala:780)
        	at scala.collection.AbstractIterator.find(Iterator.scala:1157)
        	at scala.collection.IterableLike$class.find(IterableLike.scala:79)
        	at scala.collection.AbstractIterable.find(Iterable.scala:54)
        	at com.atlassian.jira.plugins.issue.create.context.InputParamsBuilder.mapperFor(InputParamsBuilder.scala:111)
        	at com.atlassian.jira.plugins.issue.create.context.InputParamsBuilder.mapEntry(InputParamsBuilder.scala:105)
        	at com.atlassian.jira.plugins.issue.create.context.InputParamsBuilder$$anonfun$7$$anonfun$apply$13.apply(InputParamsBuilder.scala:95)
        	at com.atlassian.jira.plugins.issue.create.context.InputParamsBuilder$$anonfun$7$$anonfun$apply$13.apply(InputParamsBuilder.scala:94)
        	at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:251)
        	at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:251)
        	at scala.collection.immutable.HashSet$HashSet1.foreach(HashSet.scala:153)
        	at scala.collection.immutable.HashSet$HashTrieSet.foreach(HashSet.scala:306)
        	at scala.collection.immutable.HashSet$HashTrieSet.foreach(HashSet.scala:306)
        	at scala.collection.TraversableLike$class.flatMap(TraversableLike.scala:251)
        	at scala.collection.AbstractTraversable.flatMap(Traversable.scala:105)
        	at com.atlassian.jira.plugins.issue.create.context.InputParamsBuilder$$anonfun$7.apply(InputParamsBuilder.scala:94)
        	at com.atlassian.jira.plugins.issue.create.context.InputParamsBuilder$$anonfun$7.apply(InputParamsBuilder.scala:93)
        	at scala.collection.TraversableLike$WithFilter$$anonfun$flatMap$2.apply(TraversableLike.scala:753)
        	at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
        	at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47)
        	at scala.collection.TraversableLike$WithFilter.flatMap(TraversableLike.scala:752)
        	at com.atlassian.jira.plugins.issue.create.context.InputParamsBuilder.contextForPossibleValues(InputParamsBuilder.scala:93)
        	at com.atlassian.jira.plugins.issue.create.rest.ContextResolutionHelper.possibleValuesForClause(ContextResolutionHelper.scala:44)
        	at com.atlassian.jira.plugins.issue.create.rest.ContextResolutionHelper.possibleValuesForIssueInput(ContextResolutionHelper.scala:49)
        	at com.atlassian.greenhopper.service.query.FieldPossibleValuesAdapterImpl.possibleValuesForIssueInput(FieldPossibleValuesAdapterImpl.java:21)
        	at com.atlassian.greenhopper.service.query.FieldPossibleValues.getProjectIdsForQuery(FieldPossibleValues.java:38)
        	at com.atlassian.greenhopper.service.query.QueryToProjectMapper$RapidViewCacheLoader.load(QueryToProjectMapper.java:145)
        	at com.atlassian.greenhopper.service.query.QueryToProjectMapper$RapidViewCacheLoader.load(QueryToProjectMapper.java:140)
        	at com.atlassian.cache.compat.delegate.DelegatingCacheLoader.load(DelegatingCacheLoader.java:23)
        	at com.atlassian.cache.memory.MemoryCacheManager$3$1.load(MemoryCacheManager.java:132)
        	at com.atlassian.cache.memory.BlockingCacheLoader.load(BlockingCacheLoader.java:51)
        	at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3573)
        	at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2350)
        	at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2313)
        	- locked <0x00007f22c68eee70> (a com.google.common.cache.LocalCache$StrongWriteEntry)
        	at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2228)
        	at com.google.common.cache.LocalCache.get(LocalCache.java:3970)
        	at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3974)
        	at com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4834)
        	at com.atlassian.cache.memory.DelegatingCache$DelegatingLoadingCache.get(DelegatingCache.java:324)
        	at com.atlassian.cache.compat.delegate.DelegatingCache.get(DelegatingCache.java:37)
        	at com.atlassian.greenhopper.service.query.QueryToProjectMapper.getProjectContextIdsForQuery(QueryToProjectMapper.java:78)
        	at com.atlassian.greenhopper.service.query.QueryServiceImpl.getProjectsForRapidView(QueryServiceImpl.java:116)
        	at com.atlassian.greenhopper.service.query.QueryServiceImpl.getExplicitProjectsForRapidViewFilterQuery(QueryServiceImpl.java:103)
        	at com.atlassian.greenhopper.service.rapid.ProjectRapidViewServiceImpl.findProjectsByRapidView(ProjectRapidViewServiceImpl.java:57)
        	at com.atlassian.greenhopper.service.rapid.ProjectRapidViewServiceImpl.findRapidViewsByProject(ProjectRapidViewServiceImpl.java:40)
        	at com.atlassian.greenhopper.web.sidebar.ProjectBoardsHelper.getProjectBoards(ProjectBoardsHelper.java:53)
        	at com.atlassian.greenhopper.web.sidebar.BoardScopeFilterContextProviderImpl.getContext(BoardScopeFilterContextProviderImpl.java:37)
        	at com.atlassian.greenhopper.web.sidebar.AgileBoardScopeFilterContextProvider.getContext(AgileBoardScopeFilterContextProvider.java:27)
        	at com.atlassian.jira.projects.context.ProjectContextPopulator.populateWithProject(ProjectContextPopulator.java:77)
        	at com.atlassian.jira.projects.context.ProjectContextPopulator.populateWithProject(ProjectContextPopulator.java:44)
        	at com.atlassian.jira.projects.page.ProjectPageServlet.getPluginProvidedDefaultPage(ProjectPageServlet.java:162)
        	at com.atlassian.jira.projects.page.ProjectPageServlet.selectItemToRender(ProjectPageServlet.java:118)
        	at com.atlassian.jira.projects.page.ProjectPageServlet.doGet(ProjectPageServlet.java:94)
        	at javax.servlet.http.HttpServlet.service(HttpServlet.java:620)
        	at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
        	at com.atlassian.plugin.servlet.DelegatingPluginServlet.service(DelegatingPluginServlet.java:42)
        	at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
        	at com.atlassian.plugin.servlet.ServletModuleContainerServlet.service(ServletModuleContainerServlet.java:52)
        	at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
        	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
        	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
        	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
        	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        	at com.atlassian.jira.web.filters.XContentTypeOptionsNoSniffFilter.doFilter(XContentTypeOptionsNoSniffFilter.java:22)
        	at com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:31)
        	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
        	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        	at com.atlassian.core.filters.HeaderSanitisingFilter.doFilter(HeaderSanitisingFilter.java:44)
        	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
        	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:46)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:70)
        	at com.atlassian.ams.shipit.servlet.filter.TomcatThreadFilter.doFilter(TomcatThreadFilter.java:70)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:78)
        	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:70)
        	at com.atlassian.jira.tzdetect.IncludeResourcesFilter.doFilter(IncludeResourcesFilter.java:40)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:78)
        	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:70)
        	at com.atlassian.jira.baseurl.IncludeResourcesFilter.doFilter(IncludeResourcesFilter.java:38)
        	at com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:31)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:78)
        	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:70)
        	at com.atlassian.applinks.core.rest.context.ContextFilter.doFilter(ContextFilter.java:25)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:78)
        	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:70)
        	at com.atlassian.prettyurls.filter.PrettyUrlsSiteMeshFixupFilter.doFilter(PrettyUrlsSiteMeshFixupFilter.java:36)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:78)
        	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:70)
        	at com.atlassian.prettyurls.filter.PrettyUrlsDispatcherFilter.doFilter(PrettyUrlsDispatcherFilter.java:60)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:78)
        	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:70)
        	at com.atlassian.prettyurls.filter.PrettyUrlsCommonFilter.preventDoubleInvocation(PrettyUrlsCommonFilter.java:38)
        	at com.atlassian.prettyurls.filter.PrettyUrlsSiteMeshFilter.doFilter(PrettyUrlsSiteMeshFilter.java:74)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:78)
        	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:70)
        	at com.atlassian.prettyurls.filter.PrettyUrlsMatcherFilter.doFilter(PrettyUrlsMatcherFilter.java:56)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:78)
        	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:70)
        	at com.atlassian.labs.botkiller.BotKillerFilter.doFilter(BotKillerFilter.java:30)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:78)
        	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42)
        	at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:77)
        	at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:63)
        	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
        	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        	at com.atlassian.jira.web.filters.pagebuilder.PageBuilderFilter.doFilter(PageBuilderFilter.java:69)
        	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
        	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:46)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:70)
        	at com.atlassian.prettyurls.filter.PrettyUrlsCombinedMatchDispatcherFilter.doFilter(PrettyUrlsCombinedMatchDispatcherFilter.java:61)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:78)
        	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42)
        	at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:77)
        	at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:63)
        	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
        	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        	at com.atlassian.seraph.filter.SecurityFilter.doFilter(SecurityFilter.java:85)
        	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
        	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        	at com.atlassian.seraph.filter.BaseLoginFilter.doFilter(BaseLoginFilter.java:172)
        	at com.atlassian.jira.web.filters.JiraLoginFilter.doFilter(JiraLoginFilter.java:70)
        	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
        	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:46)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:70)
        	at com.atlassian.oauth.serviceprovider.internal.servlet.OAuthFilter.doFilter(OAuthFilter.java:79)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:78)
        	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:70)
        	at com.atlassian.prettyurls.filter.PrettyUrlsCombinedMatchDispatcherFilter.doFilter(PrettyUrlsCombinedMatchDispatcherFilter.java:61)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:78)
        	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42)
        	at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:77)
        	at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:63)
        	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
        	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:46)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:70)
        	at com.atlassian.prettyurls.filter.PrettyUrlsCombinedMatchDispatcherFilter.doFilter(PrettyUrlsCombinedMatchDispatcherFilter.java:61)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:78)
        	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42)
        	at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:77)
        	at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:63)
        	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
        	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        	at org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcher.java:748)
        	at org.apache.catalina.core.ApplicationDispatcher.processRequest(ApplicationDispatcher.java:486)
        	at org.apache.catalina.core.ApplicationDispatcher.doForward(ApplicationDispatcher.java:411)
        	at org.apache.catalina.core.ApplicationDispatcher.forward(ApplicationDispatcher.java:338)
        	at com.atlassian.prettyurls.filter.PrettyUrlsDispatcherFilter.doFilter(PrettyUrlsDispatcherFilter.java:55)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:78)
        	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:70)
        	at com.opensymphony.sitemesh.webapp.SiteMeshFilter.obtainContent(SiteMeshFilter.java:181)
        	at com.opensymphony.sitemesh.webapp.SiteMeshFilter.doFilter(SiteMeshFilter.java:85)
        	at com.atlassian.prettyurls.filter.PrettyUrlsSiteMeshFilter.doFilter(PrettyUrlsSiteMeshFilter.java:85)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:78)
        	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:70)
        	at com.atlassian.prettyurls.filter.PrettyUrlsMatcherFilter.doFilter(PrettyUrlsMatcherFilter.java:56)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:78)
        	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:70)
        	at com.atlassian.labs.botkiller.BotKillerFilter.doFilter(BotKillerFilter.java:36)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:78)
        	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42)
        	at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:77)
        	at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:63)
        	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
        	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        	at com.atlassian.jira.web.filters.accesslog.AccessLogFilter.executeRequest(AccessLogFilter.java:105)
        	at com.atlassian.jira.web.filters.accesslog.AccessLogFilter.doFilter(AccessLogFilter.java:89)
        	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
        	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        	at com.atlassian.jira.security.xsrf.XsrfTokenAdditionRequestFilter.doFilter(XsrfTokenAdditionRequestFilter.java:54)
        	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
        	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        	at com.atlassian.jira.web.filters.pagebuilder.PageBuilderFilter.doFilter(PageBuilderFilter.java:90)
        	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
        	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        	at com.atlassian.jira.web.filters.CommittedResponseHtmlErrorRecoveryFilter.doFilter(CommittedResponseHtmlErrorRecoveryFilter.java:63)
        	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
        	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:46)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:70)
        	at com.atlassian.jira.projects.servlet.ReKeyedProjectFilter.doFilter(ReKeyedProjectFilter.java:59)
        	at com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:31)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:78)
        	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:70)
        	at com.atlassian.jira.projects.darkfeature.DarkFeatureFilter.doFilter(DarkFeatureFilter.java:38)
        	at com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:31)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:78)
        	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:70)
        	at com.atlassian.jira.projects.web.redirect.ProjectCentricNavUrlRedirectFilter.doFilter(ProjectCentricNavUrlRedirectFilter.java:47)
        	at com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:31)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:78)
        	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:70)
        	at com.atlassian.prettyurls.filter.PrettyUrlsCombinedMatchDispatcherFilter.doFilter(PrettyUrlsCombinedMatchDispatcherFilter.java:61)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:78)
        	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42)
        	at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:77)
        	at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:63)
        	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
        	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        	at com.atlassian.seraph.filter.SecurityFilter.doFilter(SecurityFilter.java:239)
        	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
        	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        	at com.atlassian.security.auth.trustedapps.filter.TrustedApplicationsFilter.doFilter(TrustedApplicationsFilter.java:100)
        	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
        	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        	at com.atlassian.seraph.filter.BaseLoginFilter.doFilter(BaseLoginFilter.java:172)
        	at com.atlassian.jira.web.filters.JiraLoginFilter.doFilter(JiraLoginFilter.java:70)
        	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
        	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:46)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:70)
        	at com.atlassian.oauth.serviceprovider.internal.servlet.OAuthFilter.doFilter(OAuthFilter.java:79)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:78)
        	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:70)
        	at com.atlassian.prettyurls.filter.PrettyUrlsCombinedMatchDispatcherFilter.doFilter(PrettyUrlsCombinedMatchDispatcherFilter.java:61)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:78)
        	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42)
        	at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:77)
        	at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:63)
        	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
        	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        	at org.tuckey.web.filters.urlrewrite.RuleChain.handleRewrite(RuleChain.java:176)
        	at org.tuckey.web.filters.urlrewrite.RuleChain.doRules(RuleChain.java:145)
        	at org.tuckey.web.filters.urlrewrite.UrlRewriter.processRequest(UrlRewriter.java:92)
        	at org.tuckey.web.filters.urlrewrite.UrlRewriteFilter.doFilter(UrlRewriteFilter.java:394)
        	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
        	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:46)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:70)
        	at com.atlassian.prettyurls.filter.PrettyUrlsCombinedMatchDispatcherFilter.doFilter(PrettyUrlsCombinedMatchDispatcherFilter.java:61)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:78)
        	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42)
        	at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:77)
        	at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:63)
        	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
        	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        	at com.atlassian.jira.web.filters.steps.ChainedFilterStepRunner.doFilter(ChainedFilterStepRunner.java:87)
        	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
        	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        	at com.atlassian.core.filters.cache.AbstractCachingFilter.doFilter(AbstractCachingFilter.java:33)
        	at com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:31)
        	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
        	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        	at com.atlassian.core.filters.encoding.AbstractEncodingFilter.doFilter(AbstractEncodingFilter.java:41)
        	at com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:31)
        	at com.atlassian.jira.web.filters.PathMatchingEncodingFilter.doFilter(PathMatchingEncodingFilter.java:49)
        	at com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:31)
        	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
        	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        	at com.atlassian.jira.startup.JiraStartupChecklistFilter.doFilter(JiraStartupChecklistFilter.java:79)
        	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
        	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        	at com.atlassian.jira.web.filters.MultipartBoundaryCheckFilter.doFilter(MultipartBoundaryCheckFilter.java:41)
        	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
        	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        	at com.atlassian.jira.web.filters.steps.ChainedFilterStepRunner.doFilter(ChainedFilterStepRunner.java:87)
        	at com.atlassian.jira.web.filters.JiraFirstFilter.doFilter(JiraFirstFilter.java:61)
        	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
        	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        	at com.atlassian.gzipfilter.GzipFilter.doFilterInternal(GzipFilter.java:115)
        	at com.atlassian.gzipfilter.GzipFilter.doFilter(GzipFilter.java:92)
        	at com.atlassian.jira.web.filters.gzip.JiraGzipFilter.doFilter(JiraGzipFilter.java:56)
        	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
        	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
        	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
        	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:501)
        	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
        	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
        	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
        	at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950)
        	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
        	at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1070)
        	at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:611)
        	at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:314)
        	- locked <0x00007f227f516490> (a org.apache.tomcat.util.net.SocketWrapper)
        	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
        	at java.lang.Thread.run(Thread.java:745)
        
           Locked ownable synchronizers:
        	- <0x00007f222fa057e8> (a java.util.concurrent.ThreadPoolExecutor$Worker)
        	- <0x00007f22c5b1fe58> (a com.atlassian.cache.impl.OneShotLatch$Sync)
        

        Once the above thread has finished executing, the cache will be populated and all other threads that are waiting on it to be available will finish execution.

      Workaround

      Disable the plugin that is causing the problem (this can be identified by using something such as jProfiler to profile the instance as per Using jProfiler to analyse a JIRA application performance).

      Alternatively, you can increase the timeout as mentioned in How to change the default AJAX timeout in JIRA.

              Unassigned Unassigned
              dcurrie@atlassian.com Dave C
              Votes:
              19 Vote for this issue
              Watchers:
              31 Start watching this issue

                Created:
                Updated:
                Resolved: