Uploaded image for project: 'Jira Software Data Center'
  1. Jira Software Data Center
  2. JSWSERVER-13632

JIRA Software Manage Boards and viewing boards can timeout due to analysing JQL

    XMLWordPrintable

Details

    Description

      NOTE: This bug report is for JIRA Software Server. Using JIRA Software Cloud? See the corresponding bug report.

      Summary

      Environment

      Reproduced with the following instance size:

          ___ Database Statistics ____________________
      
               Issues                                        : 330776
               Projects                                      : 187
               Custom Fields                                 : 180
               Workflows                                     : 306
               Users                                         : 3719
               Groups                                        : 2060
               Attachments                                   : 118585
               Comments                                      : 265597
      

      It could be:

      • 52 Rapid Boards were using the issue in parent() function
      • Large number of boards and/or complex JQL used in the board filters.

      Steps to Reproduce

      1. Use a large instance, such as above.
      2. Install a third-party plugin that provides additional functionality. This was reproduced with J-Tricks, using issue in parent().
      3. Access the Manage Boards page, or browse to projects.

      It is important to note that this affected rapid boards which did not use that third-party JQL, as com.atlassian.greenhopper.web.sidebar.AgileBoardScopeFilterContextProvider.getContext (the JIRA Software code called from the JIRA Core sidebar) executes the same code to perform project lookups by analysing JQL queries (analysing, not executing). The problem is due to the way the third-party plugin is designed is that the operand executes when it's analysed. Specifically this is not conclusively caused by third-party plugins.

      Expected Results

      • The Manage Boards page does not time out, and the project page loads within an acceptable time.
      • Viewing Rapid Boards responds in a reasonable time frame.

      Actual Results

      The ajax timeout kicks in and the page times out whilst waiting for rest/greenhopper/1.0/rapidviews/viewsData to return results, or the board can take > 1 minute to load.

      Notes

      Accessing that endpoint outside of JIRA (and thus bypassing ajax) in this instance returned within 1.2mins.

      This appears to be caused by QueryToProjectMapper.getProjectContextIdsForQuery which uses JIRA Core code to lookup the QueryContext. Due to the way the lookup occurs it's possible that JQL may be executed and this times out the board.

      A sample snapshot of the problem has been attached.

      Verification

      The problem occurs in multiple areas of JIRA Software - it's most evident in manage boards however can occur on the Rapid Boards.

      For Manage Boards:

      1. Call /rest/greenhopper/1.0/rapidviews/viewsData will be slow (40+ seconds, in some cases will not return anything as the AJAX timeout terminates the request).
      2. Class com.atlassian.greenhopper.web.rapid.view.RapidViewListResource.getViewsAndConfigModel will be present in the stacktrace for long-running threads:
        "http-nio-80-exec-45" #8007 daemon prio=5 os_prio=0 tid=0x000000003cac0000 nid=0x19dc runnable [0x000000027765a000]
        ...
                at com.atlassian.greenhopper.web.rapid.view.RapidViewProjectHelper.getRapidViewProjectList(RapidViewProjectHelper.java:58)
        ...
                at com.atlassian.greenhopper.web.rapid.view.RapidViewListHelper.buildListAndConfigModel(RapidViewListHelper.java:105)
        ...
                at com.atlassian.greenhopper.web.rapid.view.RapidViewListResource.getViewsAndConfigModel(RapidViewListResource.java:74)
        ...
        

      For Rapid Boards:

      1. Many threads will be waiting on a response to AbstractFuture$Sync as per the attached screenshot. These threads are waiting on the populattion of the cache that FieldPossibleValues.getProjectIdsForQuery queries.
      2. There will be one executing thread in QueryToProjectMapper$RapidViewCacheLoader.load, populating the cache. Below is a sample stack trace:
        "http-bio-8081-exec-862 uri:/plugins/servlet/projects/ABC username:captain.planet" #1247 daemon prio=5 tid=0x00007f210832f000 nid=0x1aca waiting on condition [0x00007f1cea179000]
           java.lang.Thread.State: RUNNABLE
        	at com.atlassian.jira.plugins.issue.create.context.fields.helpers.CustomCFTypeInputMapper$class.getFieldType(CustomCFTypeInputMapper.scala:32)
        	at com.atlassian.jira.plugins.issue.create.context.fields.custom.GenericTextCFTypeJQLMapper.getFieldType(GenericTextCFTypeJQLMapper.scala:13)
        	at com.atlassian.jira.plugins.issue.create.context.fields.helpers.CustomCFTypeInputMapper$class.matchesClause(CustomCFTypeInputMapper.scala:49)
        	at com.atlassian.jira.plugins.issue.create.context.fields.custom.GenericTextCFTypeJQLMapper.matchesClause(GenericTextCFTypeJQLMapper.scala:22)
        	at com.atlassian.jira.plugins.issue.create.context.InputParamsBuilder$$anonfun$mapperFor$1.apply(InputParamsBuilder.scala:111)
        	at com.atlassian.jira.plugins.issue.create.context.InputParamsBuilder$$anonfun$mapperFor$1.apply(InputParamsBuilder.scala:111)
        	at scala.collection.Iterator$class.find(Iterator.scala:780)
        	at scala.collection.AbstractIterator.find(Iterator.scala:1157)
        	at scala.collection.IterableLike$class.find(IterableLike.scala:79)
        	at scala.collection.AbstractIterable.find(Iterable.scala:54)
        	at com.atlassian.jira.plugins.issue.create.context.InputParamsBuilder.mapperFor(InputParamsBuilder.scala:111)
        	at com.atlassian.jira.plugins.issue.create.context.InputParamsBuilder.mapEntry(InputParamsBuilder.scala:105)
        	at com.atlassian.jira.plugins.issue.create.context.InputParamsBuilder$$anonfun$7$$anonfun$apply$13.apply(InputParamsBuilder.scala:95)
        	at com.atlassian.jira.plugins.issue.create.context.InputParamsBuilder$$anonfun$7$$anonfun$apply$13.apply(InputParamsBuilder.scala:94)
        	at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:251)
        	at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:251)
        	at scala.collection.immutable.HashSet$HashSet1.foreach(HashSet.scala:153)
        	at scala.collection.immutable.HashSet$HashTrieSet.foreach(HashSet.scala:306)
        	at scala.collection.immutable.HashSet$HashTrieSet.foreach(HashSet.scala:306)
        	at scala.collection.TraversableLike$class.flatMap(TraversableLike.scala:251)
        	at scala.collection.AbstractTraversable.flatMap(Traversable.scala:105)
        	at com.atlassian.jira.plugins.issue.create.context.InputParamsBuilder$$anonfun$7.apply(InputParamsBuilder.scala:94)
        	at com.atlassian.jira.plugins.issue.create.context.InputParamsBuilder$$anonfun$7.apply(InputParamsBuilder.scala:93)
        	at scala.collection.TraversableLike$WithFilter$$anonfun$flatMap$2.apply(TraversableLike.scala:753)
        	at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
        	at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47)
        	at scala.collection.TraversableLike$WithFilter.flatMap(TraversableLike.scala:752)
        	at com.atlassian.jira.plugins.issue.create.context.InputParamsBuilder.contextForPossibleValues(InputParamsBuilder.scala:93)
        	at com.atlassian.jira.plugins.issue.create.rest.ContextResolutionHelper.possibleValuesForClause(ContextResolutionHelper.scala:44)
        	at com.atlassian.jira.plugins.issue.create.rest.ContextResolutionHelper.possibleValuesForIssueInput(ContextResolutionHelper.scala:49)
        	at com.atlassian.greenhopper.service.query.FieldPossibleValuesAdapterImpl.possibleValuesForIssueInput(FieldPossibleValuesAdapterImpl.java:21)
        	at com.atlassian.greenhopper.service.query.FieldPossibleValues.getProjectIdsForQuery(FieldPossibleValues.java:38)
        	at com.atlassian.greenhopper.service.query.QueryToProjectMapper$RapidViewCacheLoader.load(QueryToProjectMapper.java:145)
        	at com.atlassian.greenhopper.service.query.QueryToProjectMapper$RapidViewCacheLoader.load(QueryToProjectMapper.java:140)
        	at com.atlassian.cache.compat.delegate.DelegatingCacheLoader.load(DelegatingCacheLoader.java:23)
        	at com.atlassian.cache.memory.MemoryCacheManager$3$1.load(MemoryCacheManager.java:132)
        	at com.atlassian.cache.memory.BlockingCacheLoader.load(BlockingCacheLoader.java:51)
        	at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3573)
        	at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2350)
        	at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2313)
        	- locked <0x00007f22c68eee70> (a com.google.common.cache.LocalCache$StrongWriteEntry)
        	at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2228)
        	at com.google.common.cache.LocalCache.get(LocalCache.java:3970)
        	at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3974)
        	at com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4834)
        	at com.atlassian.cache.memory.DelegatingCache$DelegatingLoadingCache.get(DelegatingCache.java:324)
        	at com.atlassian.cache.compat.delegate.DelegatingCache.get(DelegatingCache.java:37)
        	at com.atlassian.greenhopper.service.query.QueryToProjectMapper.getProjectContextIdsForQuery(QueryToProjectMapper.java:78)
        	at com.atlassian.greenhopper.service.query.QueryServiceImpl.getProjectsForRapidView(QueryServiceImpl.java:116)
        	at com.atlassian.greenhopper.service.query.QueryServiceImpl.getExplicitProjectsForRapidViewFilterQuery(QueryServiceImpl.java:103)
        	at com.atlassian.greenhopper.service.rapid.ProjectRapidViewServiceImpl.findProjectsByRapidView(ProjectRapidViewServiceImpl.java:57)
        	at com.atlassian.greenhopper.service.rapid.ProjectRapidViewServiceImpl.findRapidViewsByProject(ProjectRapidViewServiceImpl.java:40)
        	at com.atlassian.greenhopper.web.sidebar.ProjectBoardsHelper.getProjectBoards(ProjectBoardsHelper.java:53)
        	at com.atlassian.greenhopper.web.sidebar.BoardScopeFilterContextProviderImpl.getContext(BoardScopeFilterContextProviderImpl.java:37)
        	at com.atlassian.greenhopper.web.sidebar.AgileBoardScopeFilterContextProvider.getContext(AgileBoardScopeFilterContextProvider.java:27)
        	at com.atlassian.jira.projects.context.ProjectContextPopulator.populateWithProject(ProjectContextPopulator.java:77)
        	at com.atlassian.jira.projects.context.ProjectContextPopulator.populateWithProject(ProjectContextPopulator.java:44)
        	at com.atlassian.jira.projects.page.ProjectPageServlet.getPluginProvidedDefaultPage(ProjectPageServlet.java:162)
        	at com.atlassian.jira.projects.page.ProjectPageServlet.selectItemToRender(ProjectPageServlet.java:118)
        	at com.atlassian.jira.projects.page.ProjectPageServlet.doGet(ProjectPageServlet.java:94)
        	at javax.servlet.http.HttpServlet.service(HttpServlet.java:620)
        	at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
        	at com.atlassian.plugin.servlet.DelegatingPluginServlet.service(DelegatingPluginServlet.java:42)
        	at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
        	at com.atlassian.plugin.servlet.ServletModuleContainerServlet.service(ServletModuleContainerServlet.java:52)
        	at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
        	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
        	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
        	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
        	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        	at com.atlassian.jira.web.filters.XContentTypeOptionsNoSniffFilter.doFilter(XContentTypeOptionsNoSniffFilter.java:22)
        	at com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:31)
        	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
        	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        	at com.atlassian.core.filters.HeaderSanitisingFilter.doFilter(HeaderSanitisingFilter.java:44)
        	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
        	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:46)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:70)
        	at com.atlassian.ams.shipit.servlet.filter.TomcatThreadFilter.doFilter(TomcatThreadFilter.java:70)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:78)
        	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:70)
        	at com.atlassian.jira.tzdetect.IncludeResourcesFilter.doFilter(IncludeResourcesFilter.java:40)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:78)
        	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:70)
        	at com.atlassian.jira.baseurl.IncludeResourcesFilter.doFilter(IncludeResourcesFilter.java:38)
        	at com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:31)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:78)
        	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:70)
        	at com.atlassian.applinks.core.rest.context.ContextFilter.doFilter(ContextFilter.java:25)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:78)
        	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:70)
        	at com.atlassian.prettyurls.filter.PrettyUrlsSiteMeshFixupFilter.doFilter(PrettyUrlsSiteMeshFixupFilter.java:36)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:78)
        	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:70)
        	at com.atlassian.prettyurls.filter.PrettyUrlsDispatcherFilter.doFilter(PrettyUrlsDispatcherFilter.java:60)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:78)
        	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:70)
        	at com.atlassian.prettyurls.filter.PrettyUrlsCommonFilter.preventDoubleInvocation(PrettyUrlsCommonFilter.java:38)
        	at com.atlassian.prettyurls.filter.PrettyUrlsSiteMeshFilter.doFilter(PrettyUrlsSiteMeshFilter.java:74)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:78)
        	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:70)
        	at com.atlassian.prettyurls.filter.PrettyUrlsMatcherFilter.doFilter(PrettyUrlsMatcherFilter.java:56)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:78)
        	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:70)
        	at com.atlassian.labs.botkiller.BotKillerFilter.doFilter(BotKillerFilter.java:30)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:78)
        	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42)
        	at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:77)
        	at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:63)
        	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
        	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        	at com.atlassian.jira.web.filters.pagebuilder.PageBuilderFilter.doFilter(PageBuilderFilter.java:69)
        	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
        	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:46)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:70)
        	at com.atlassian.prettyurls.filter.PrettyUrlsCombinedMatchDispatcherFilter.doFilter(PrettyUrlsCombinedMatchDispatcherFilter.java:61)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:78)
        	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42)
        	at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:77)
        	at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:63)
        	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
        	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        	at com.atlassian.seraph.filter.SecurityFilter.doFilter(SecurityFilter.java:85)
        	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
        	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        	at com.atlassian.seraph.filter.BaseLoginFilter.doFilter(BaseLoginFilter.java:172)
        	at com.atlassian.jira.web.filters.JiraLoginFilter.doFilter(JiraLoginFilter.java:70)
        	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
        	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:46)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:70)
        	at com.atlassian.oauth.serviceprovider.internal.servlet.OAuthFilter.doFilter(OAuthFilter.java:79)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:78)
        	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:70)
        	at com.atlassian.prettyurls.filter.PrettyUrlsCombinedMatchDispatcherFilter.doFilter(PrettyUrlsCombinedMatchDispatcherFilter.java:61)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:78)
        	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42)
        	at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:77)
        	at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:63)
        	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
        	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:46)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:70)
        	at com.atlassian.prettyurls.filter.PrettyUrlsCombinedMatchDispatcherFilter.doFilter(PrettyUrlsCombinedMatchDispatcherFilter.java:61)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:78)
        	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42)
        	at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:77)
        	at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:63)
        	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
        	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        	at org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcher.java:748)
        	at org.apache.catalina.core.ApplicationDispatcher.processRequest(ApplicationDispatcher.java:486)
        	at org.apache.catalina.core.ApplicationDispatcher.doForward(ApplicationDispatcher.java:411)
        	at org.apache.catalina.core.ApplicationDispatcher.forward(ApplicationDispatcher.java:338)
        	at com.atlassian.prettyurls.filter.PrettyUrlsDispatcherFilter.doFilter(PrettyUrlsDispatcherFilter.java:55)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:78)
        	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:70)
        	at com.opensymphony.sitemesh.webapp.SiteMeshFilter.obtainContent(SiteMeshFilter.java:181)
        	at com.opensymphony.sitemesh.webapp.SiteMeshFilter.doFilter(SiteMeshFilter.java:85)
        	at com.atlassian.prettyurls.filter.PrettyUrlsSiteMeshFilter.doFilter(PrettyUrlsSiteMeshFilter.java:85)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:78)
        	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:70)
        	at com.atlassian.prettyurls.filter.PrettyUrlsMatcherFilter.doFilter(PrettyUrlsMatcherFilter.java:56)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:78)
        	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:70)
        	at com.atlassian.labs.botkiller.BotKillerFilter.doFilter(BotKillerFilter.java:36)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:78)
        	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42)
        	at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:77)
        	at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:63)
        	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
        	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        	at com.atlassian.jira.web.filters.accesslog.AccessLogFilter.executeRequest(AccessLogFilter.java:105)
        	at com.atlassian.jira.web.filters.accesslog.AccessLogFilter.doFilter(AccessLogFilter.java:89)
        	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
        	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        	at com.atlassian.jira.security.xsrf.XsrfTokenAdditionRequestFilter.doFilter(XsrfTokenAdditionRequestFilter.java:54)
        	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
        	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        	at com.atlassian.jira.web.filters.pagebuilder.PageBuilderFilter.doFilter(PageBuilderFilter.java:90)
        	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
        	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        	at com.atlassian.jira.web.filters.CommittedResponseHtmlErrorRecoveryFilter.doFilter(CommittedResponseHtmlErrorRecoveryFilter.java:63)
        	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
        	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:46)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:70)
        	at com.atlassian.jira.projects.servlet.ReKeyedProjectFilter.doFilter(ReKeyedProjectFilter.java:59)
        	at com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:31)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:78)
        	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:70)
        	at com.atlassian.jira.projects.darkfeature.DarkFeatureFilter.doFilter(DarkFeatureFilter.java:38)
        	at com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:31)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:78)
        	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:70)
        	at com.atlassian.jira.projects.web.redirect.ProjectCentricNavUrlRedirectFilter.doFilter(ProjectCentricNavUrlRedirectFilter.java:47)
        	at com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:31)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:78)
        	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:70)
        	at com.atlassian.prettyurls.filter.PrettyUrlsCombinedMatchDispatcherFilter.doFilter(PrettyUrlsCombinedMatchDispatcherFilter.java:61)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:78)
        	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42)
        	at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:77)
        	at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:63)
        	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
        	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        	at com.atlassian.seraph.filter.SecurityFilter.doFilter(SecurityFilter.java:239)
        	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
        	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        	at com.atlassian.security.auth.trustedapps.filter.TrustedApplicationsFilter.doFilter(TrustedApplicationsFilter.java:100)
        	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
        	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        	at com.atlassian.seraph.filter.BaseLoginFilter.doFilter(BaseLoginFilter.java:172)
        	at com.atlassian.jira.web.filters.JiraLoginFilter.doFilter(JiraLoginFilter.java:70)
        	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
        	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:46)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:70)
        	at com.atlassian.oauth.serviceprovider.internal.servlet.OAuthFilter.doFilter(OAuthFilter.java:79)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:78)
        	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:70)
        	at com.atlassian.prettyurls.filter.PrettyUrlsCombinedMatchDispatcherFilter.doFilter(PrettyUrlsCombinedMatchDispatcherFilter.java:61)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:78)
        	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42)
        	at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:77)
        	at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:63)
        	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
        	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        	at org.tuckey.web.filters.urlrewrite.RuleChain.handleRewrite(RuleChain.java:176)
        	at org.tuckey.web.filters.urlrewrite.RuleChain.doRules(RuleChain.java:145)
        	at org.tuckey.web.filters.urlrewrite.UrlRewriter.processRequest(UrlRewriter.java:92)
        	at org.tuckey.web.filters.urlrewrite.UrlRewriteFilter.doFilter(UrlRewriteFilter.java:394)
        	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
        	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:46)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:70)
        	at com.atlassian.prettyurls.filter.PrettyUrlsCombinedMatchDispatcherFilter.doFilter(PrettyUrlsCombinedMatchDispatcherFilter.java:61)
        	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:78)
        	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42)
        	at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:77)
        	at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:63)
        	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
        	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        	at com.atlassian.jira.web.filters.steps.ChainedFilterStepRunner.doFilter(ChainedFilterStepRunner.java:87)
        	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
        	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        	at com.atlassian.core.filters.cache.AbstractCachingFilter.doFilter(AbstractCachingFilter.java:33)
        	at com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:31)
        	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
        	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        	at com.atlassian.core.filters.encoding.AbstractEncodingFilter.doFilter(AbstractEncodingFilter.java:41)
        	at com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:31)
        	at com.atlassian.jira.web.filters.PathMatchingEncodingFilter.doFilter(PathMatchingEncodingFilter.java:49)
        	at com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:31)
        	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
        	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        	at com.atlassian.jira.startup.JiraStartupChecklistFilter.doFilter(JiraStartupChecklistFilter.java:79)
        	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
        	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        	at com.atlassian.jira.web.filters.MultipartBoundaryCheckFilter.doFilter(MultipartBoundaryCheckFilter.java:41)
        	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
        	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        	at com.atlassian.jira.web.filters.steps.ChainedFilterStepRunner.doFilter(ChainedFilterStepRunner.java:87)
        	at com.atlassian.jira.web.filters.JiraFirstFilter.doFilter(JiraFirstFilter.java:61)
        	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
        	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        	at com.atlassian.gzipfilter.GzipFilter.doFilterInternal(GzipFilter.java:115)
        	at com.atlassian.gzipfilter.GzipFilter.doFilter(GzipFilter.java:92)
        	at com.atlassian.jira.web.filters.gzip.JiraGzipFilter.doFilter(JiraGzipFilter.java:56)
        	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
        	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
        	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
        	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:501)
        	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
        	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
        	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
        	at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950)
        	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
        	at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1070)
        	at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:611)
        	at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:314)
        	- locked <0x00007f227f516490> (a org.apache.tomcat.util.net.SocketWrapper)
        	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
        	at java.lang.Thread.run(Thread.java:745)
        
           Locked ownable synchronizers:
        	- <0x00007f222fa057e8> (a java.util.concurrent.ThreadPoolExecutor$Worker)
        	- <0x00007f22c5b1fe58> (a com.atlassian.cache.impl.OneShotLatch$Sync)
        

        Once the above thread has finished executing, the cache will be populated and all other threads that are waiting on it to be available will finish execution.

      Workaround

      Disable the plugin that is causing the problem (this can be identified by using something such as jProfiler to profile the instance as per Using jProfiler to analyse a JIRA application performance).

      Alternatively, you can increase the timeout as mentioned in How to change the default AJAX timeout in JIRA.

      Note on fix

      This ticket is a combination of a couple of different problems related to JQL and Boards:

      We believe that the current problem is resolved at this point.

      Attachments

        1. getProjectContextIdsForQuery Congestion.jpg
          643 kB
          Dave C
        2. image-2018-02-09-11-56-39-797.png
          56 kB
          Ivan
        3. manage boards jtricks enabled.jps
          3.25 MB
          Dave C
        4. View All Boards.png
          25 kB
          Kateryna Lysova

        Issue Links

          Activity

            People

              Unassigned Unassigned
              dcurrie@atlassian.com Dave C
              Votes:
              24 Vote for this issue
              Watchers:
              39 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: