Uploaded image for project: 'Jira Software Data Center'
  1. Jira Software Data Center
  2. JSWSERVER-9611

performance very poor for kanban boards when instance has large amount of projects/issues

XMLWordPrintable

    • 6.02
    • Hide
      Atlassian Status as of Monday November 3
      • Changes have been made in JIRA Agile to improve the performance of calculations of status changes – this issue (GHS-9611) tracks the progress on this. This will improve performance on loading Kanban Boards (when Days In Column are shown) as well as charts such as Burndown, Control Chart, CFD.
      • A separate task for JIRA Agile to temporarily disable the "days in column" functionality (which will provide benefits to all supported JIRA versions) is being tracked in GHS-10520. This can be used to reduce the performance impact of this problem.
      • There is also a known issue with the unreleasedVersions() JQL, which is the default work sub-filter, that is fixed in JIRA 6.3.1 as tracked in JRA-38883.

      The fix for this bug has now been shipped in JIRA Agile 6.6.11.

      Show
      Atlassian Status as of Monday November 3 Changes have been made in JIRA Agile to improve the performance of calculations of status changes – this issue ( GHS-9611 ) tracks the progress on this. This will improve performance on loading Kanban Boards (when Days In Column are shown) as well as charts such as Burndown, Control Chart, CFD. A separate task for JIRA Agile to temporarily disable the "days in column" functionality (which will provide benefits to all supported JIRA versions) is being tracked in GHS-10520 . This can be used to reduce the performance impact of this problem. There is also a known issue with the unreleasedVersions() JQL, which is the default work sub-filter, that is fixed in JIRA 6.3.1 as tracked in JRA-38883 . The fix for this bug has now been shipped in JIRA Agile 6.6.11.

      After moving an issue on the board the board is refreshed by calling xboard/work/allData.json. On our production instance this takes 6 seconds. Updating the issue is relatively quick, refreshing the board data is not.

      This is down to com.atlassian.greenhopper.service.lucene.ChangeHistorySearchHelper#findChangeLogFilter. Specifically, computing the cross-references from two queries, one of which may contain up to half a million issues, is very slow. I'm sure there is a quicker way, even if it's just checking the permission for each issue retrieved from the change history index.

      As a workaround I am just disabling loading the days in column data in com.atlassian.greenhopper.web.rapid.list.RapidIssueEntryQueryServiceImpl#collectWorkModeIssues, which makes things passably fast.

      Areas of JIRA Agile Affected

      • Generating JIRA Agile charts - notably Burndown, Cumulative Flow Diagram & Control Chart.
      • Loading days in column for a list of issues (workaround to disable this in GHS-10520).

      Workaround for Work Sub-Filter

      This is fixed in JIRA 6.3.1 as tracked in JRA-38883, so this workaround will only apply for previous versions of JIRA.

      It's possible that on larger instances the default work sub-filter may be causing performance issues of its own. This is not related to com.atlassian.greenhopper.service.lucene.ChangeHistorySearchHelper however does present with similar symptoms. Specifically, the work sub-filter default is:

      fixVersion in unreleasedVersions() OR fixVersion is EMPTY

      This unreleasedVersions() JQL may be slow for larger instances that process a large number of projects, as it goes through all of them to find out which versions are relevant. This can be sped up by specifying a list of projects that are only relevant to the JIRA Agile board being used. For example if we wanted to only return issues for projects GHS and JRA (because they are in the board filter), we would use the following:

      fixVersion in unreleasedVersions(GHS,JRA) OR fixVersion is EMPTY

      Please make the above changes and verify if the boards are faster to render or not. If they are faster, it implies the unreleasedVersions() was slow to process due to the large number of projects. If it does not it would indicate that the com.atlassian.greenhopper.service.lucene.ChangeHistorySearchHelper is the cause of slowness.

      It's possible that both unreleasedVersions() and com.atlassian.greenhopper.service.lucene.ChangeHistorySearchHelper are slow to process and this workaround only addresses one part of the problem.

      Sample Thread Dump

      "http-bio-8080-exec-29" daemon prio=10 tid=0x00007f6bc0034000 nid=0x6705 runnable [0x00007f6bb4ba0000]
   java.lang.Thread.State: RUNNABLE
	at java.util.BitSet.nextSetBit(Unknown Source)
	at com.atlassian.greenhopper.service.lucene.BitSetRightJoinEnum.hasNext(BitSetRightJoinEnum.java:65)
	at com.atlassian.greenhopper.service.lucene.LeftJoin.computeResult(LeftJoin.java:52)
	at com.atlassian.greenhopper.service.lucene.ChangeHistorySearchHelper.findChangeLogFilter(ChangeHistorySearchHelper.java:81)
	at com.atlassian.greenhopper.service.charts.IssueStatusHistoryServiceImpl.collectStatusHistory(IssueStatusHistoryServiceImpl.java:79)
	at com.atlassian.greenhopper.web.rapid.list.RapidIssueEntryQueryServiceImpl.loadDaysInColumn(RapidIssueEntryQueryServiceImpl.java:820)
	at com.atlassian.greenhopper.web.rapid.list.RapidIssueEntryQueryServiceImpl.collectIssues(RapidIssueEntryQueryServiceImpl.java:716)
	at com.atlassian.greenhopper.web.rapid.list.RapidIssueEntryQueryServiceImpl.collectIssues(RapidIssueEntryQueryServiceImpl.java:627)
	at com.atlassian.greenhopper.web.rapid.list.RapidIssueEntryQueryServiceImpl.collectWorkModeIssues(RapidIssueEntryQueryServiceImpl.java:178)
	at com.atlassian.greenhopper.web.rapid.work.WorkDataFactory.getOrderedIssuesData(WorkDataFactory.java:511)
	at com.atlassian.greenhopper.web.rapid.work.WorkDataFactory.getAllData(WorkDataFactory.java:177)
	at com.atlassian.greenhopper.web.rapid.work.WorkResource$1.call(WorkResource.java:70)
	at com.atlassian.greenhopper.web.rapid.work.WorkResource$1.call(WorkResource.java:59)
	at com.atlassian.greenhopper.web.util.RestCall.response(RestCall.java:48)
	at com.atlassian.greenhopper.web.AbstractResource.response(AbstractResource.java:60)
	at com.atlassian.greenhopper.web.rapid.work.WorkResource.getPoolData(WorkResource.java:58)
	at sun.reflect.GeneratedMethodAccessor1993.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.lang.reflect.Method.invoke(Unknown Source)
	at com.atlassian.plugins.rest.common.interceptor.impl.DispatchProviderHelper$ResponseOutInvoker$1.invoke(DispatchProviderHelper.java:234)
	at com.atlassian.plugins.rest.common.interceptor.impl.DispatchProviderHelper$1.intercept(DispatchProviderHelper.java:100)
	at com.atlassian.plugins.rest.common.interceptor.impl.DefaultMethodInvocation.invoke(DefaultMethodInvocation.java:61)
	at com.atlassian.plugins.rest.common.expand.interceptor.ExpandInterceptor.intercept(ExpandInterceptor.java:38)
	at com.atlassian.plugins.rest.common.interceptor.impl.DefaultMethodInvocation.invoke(DefaultMethodInvocation.java:61)
	at com.atlassian.plugins.rest.common.interceptor.impl.DispatchProviderHelper.invokeMethodWithInterceptors(DispatchProviderHelper.java:132)
	at com.atlassian.plugins.rest.common.interceptor.impl.DispatchProviderHelper$ResponseOutInvoker._dispatch(DispatchProviderHelper.java:230)
	at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
	at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:288)
	at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
	at com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
	at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
	at com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
	at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1469)
	at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1400)
	at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1349)
	at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1339)
	at com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:416)
	at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:537)
	at com.atlassian.plugins.rest.module.RestDelegatingServletFilter$JerseyOsgiServletContainer.doFilter(RestDelegatingServletFilter.java:178)
	at com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:795)
	at com.atlassian.plugins.rest.module.RestDelegatingServletFilter.doFilter(RestDelegatingServletFilter.java:73)
	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:74)
	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42)
	at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:77)
	at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:63)
	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:74)
	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42)
	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:66)
	at com.atlassian.plugins.rest.module.servlet.RestServletUtilsUpdaterFilter.doFilterInternal(RestServletUtilsUpdaterFilter.java:26)
	at com.atlassian.plugins.rest.module.servlet.RestServletUtilsUpdaterFilter.doFilter(RestServletUtilsUpdaterFilter.java:40)
	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:74)
	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42)
	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:66)
	at com.atlassian.applinks.core.rest.context.ContextFilter.doFilter(ContextFilter.java:25)
	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:74)
	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42)
	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:66)
	at com.atlassian.mywork.client.filter.ServingRequestsFilter.doFilter(ServingRequestsFilter.java:37)
	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:74)
	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42)
	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:66)
	at com.atlassian.plugins.cors.CorsFilter.doFilter(CorsFilter.java:65)
	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:74)
	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42)
	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:66)
	at com.atlassian.prettyurls.filter.PrettyUrlsDispatcherFilter.doFilter(PrettyUrlsDispatcherFilter.java:60)
	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:74)
	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42)
	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:66)
	at com.atlassian.prettyurls.filter.PrettyUrlsSiteMeshFilter.doFilter(PrettyUrlsSiteMeshFilter.java:92)
	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:74)
	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42)
	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:66)
	at com.atlassian.prettyurls.filter.PrettyUrlsMatcherFilter.doFilter(PrettyUrlsMatcherFilter.java:56)
	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:74)
	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42)
	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:66)
	at com.atlassian.labs.botkiller.BotKillerFilter.doFilter(BotKillerFilter.java:36)
	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:74)
	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42)
	at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:77)
	at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:63)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
	at com.atlassian.jira.web.filters.accesslog.AccessLogFilter.executeRequest(AccessLogFilter.java:103)
	at com.atlassian.jira.web.filters.accesslog.AccessLogFilter.doFilter(AccessLogFilter.java:87)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
	at com.atlassian.jira.security.xsrf.XsrfTokenAdditionRequestFilter.doFilter(XsrfTokenAdditionRequestFilter.java:54)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
	at com.atlassian.jira.web.filters.steps.ChainedFilterStepRunner.doFilter(ChainedFilterStepRunner.java:87)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:46)
	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:66)
	at com.atlassian.prettyurls.filter.PrettyUrlsCombinedMatchDispatcherFilter.doFilter(PrettyUrlsCombinedMatchDispatcherFilter.java:61)
	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:74)
	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42)
	at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:77)
	at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:63)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
	at com.atlassian.seraph.filter.SecurityFilter.doFilter(SecurityFilter.java:234)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
	at com.atlassian.security.auth.trustedapps.filter.TrustedApplicationsFilter.doFilter(TrustedApplicationsFilter.java:100)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
	at com.atlassian.seraph.filter.BaseLoginFilter.doFilter(BaseLoginFilter.java:169)
	at com.atlassian.jira.web.filters.JiraLoginFilter.doFilter(JiraLoginFilter.java:70)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:46)
	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:66)
	at com.atlassian.oauth.serviceprovider.internal.servlet.OAuthFilter.doFilter(OAuthFilter.java:55)
	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:74)
	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42)
	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:66)
	at com.atlassian.plugins.rest.module.servlet.RestSeraphFilter.doFilter(RestSeraphFilter.java:40)
	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:74)
	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42)
	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:66)
	at com.atlassian.prettyurls.filter.PrettyUrlsCombinedMatchDispatcherFilter.doFilter(PrettyUrlsCombinedMatchDispatcherFilter.java:61)
	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:74)
	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42)
	at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:77)
	at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:63)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
	at com.atlassian.johnson.filters.AbstractJohnsonFilter.doFilter(AbstractJohnsonFilter.java:71)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
	at org.tuckey.web.filters.urlrewrite.RuleChain.handleRewrite(RuleChain.java:176)
	at org.tuckey.web.filters.urlrewrite.RuleChain.doRules(RuleChain.java:145)
	at org.tuckey.web.filters.urlrewrite.UrlRewriter.processRequest(UrlRewriter.java:92)
	at org.tuckey.web.filters.urlrewrite.UrlRewriteFilter.doFilter(UrlRewriteFilter.java:394)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
	at com.atlassian.gzipfilter.GzipFilter.doFilterInternal(GzipFilter.java:80)
	at com.atlassian.gzipfilter.GzipFilter.doFilter(GzipFilter.java:51)
	at com.atlassian.jira.web.filters.gzip.JiraGzipFilter.doFilter(JiraGzipFilter.java:55)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:46)
	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:66)
	at com.atlassian.prettyurls.filter.PrettyUrlsCombinedMatchDispatcherFilter.doFilter(PrettyUrlsCombinedMatchDispatcherFilter.java:61)
	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:74)
	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42)
	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:66)
	at com.sysbliss.jira.plugins.workflow.servlet.JWDSendRedirectFilter.doFilter(JWDSendRedirectFilter.java:25)
	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:74)
	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42)
	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:66)
	at net.bull.javamelody.MonitoringFilter.doFilter(MonitoringFilter.java:202)
	at net.bull.javamelody.MonitoringFilter.doFilter(MonitoringFilter.java:180)
	at net.bull.javamelody.PluginMonitoringFilter.doFilter(PluginMonitoringFilter.java:85)
	at net.bull.javamelody.JiraMonitoringFilter.doFilter(JiraMonitoringFilter.java:104)
	at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:74)
	at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42)
	at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:77)
	at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:63)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
	at com.atlassian.jira.web.filters.steps.ChainedFilterStepRunner.doFilter(ChainedFilterStepRunner.java:87)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
	at com.atlassian.core.filters.cache.AbstractCachingFilter.doFilter(AbstractCachingFilter.java:33)
	at com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:31)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
	at com.atlassian.core.filters.encoding.AbstractEncodingFilter.doFilter(AbstractEncodingFilter.java:41)
	at com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:31)
	at com.atlassian.jira.web.filters.PathMatchingEncodingFilter.doFilter(PathMatchingEncodingFilter.java:45)
	at com.atlassian.core.filters.AbstractHttpFilter.doFilter(AbstractHttpFilter.java:31)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
	at com.atlassian.jira.startup.JiraStartupChecklistFilter.doFilter(JiraStartupChecklistFilter.java:78)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
	at com.atlassian.jira.web.filters.steps.ChainedFilterStepRunner.doFilter(ChainedFilterStepRunner.java:87)
	at com.atlassian.jira.web.filters.JiraFirstFilter.doFilter(JiraFirstFilter.java:57)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:225)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:168)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:98)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
	at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:927)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
	at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1001)
	at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:585)
	at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:312)
	- locked <0x00007f70d75e5e70> (a org.apache.tomcat.util.net.SocketWrapper)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at java.lang.Thread.run(Unknown Source)

        1. indexQuery.patch
          4 kB
          JamieA
        2. Screen Shot 2013-08-01 at 3.00.40 PM.png
          233 kB
          Yilin
        3. Screen Shot 2014-05-20 at 5.31.56 pm.png
          315 kB
          MichaelA

            [JSWSERVER-9611] performance very poor for kanban boards when instance has large amount of projects/issues

            Pankaj Pimple added a comment -

            @Jamie and @Qualcomm support,

            Also could you please tell me which tool you have used to do Invocation trace as per attached screenshots?
            Our jira is configured on linux.

            Thanks and regards,

            Pankaj

            Pankaj Pimple added a comment - @Jamie and @Qualcomm support, Also could you please tell me which tool you have used to do Invocation trace as per attached screenshots? Our jira is configured on linux. Thanks and regards, Pankaj

            Pankaj Pimple added a comment -

            Hi All,

            We are using Agile 6.7.11 with JIRA 6.4.10 and could see same performance issue. While loading dashboards between 9-10 LDN time when all users are connecting and may be starting their standups, timeouts out errors are getting.
            We checked our Hardware and as per admins the hardware is sufficient and we need to look at the application side.
            i am aware that we have many issues on boards and many teams accessing board at the same time causing this issue.

            Hi jamie, Can I try your patch and if yes could you please tell how to deploy it?

            Thanks and regards,
            pankaj

            Pankaj Pimple added a comment - Hi All, We are using Agile 6.7.11 with JIRA 6.4.10 and could see same performance issue. While loading dashboards between 9-10 LDN time when all users are connecting and may be starting their standups, timeouts out errors are getting. We checked our Hardware and as per admins the hardware is sufficient and we need to look at the application side. i am aware that we have many issues on boards and many teams accessing board at the same time causing this issue. Hi jamie, Can I try your patch and if yes could you please tell how to deploy it? Thanks and regards, pankaj

            Michael Tokar added a comment -

            Thank you everyone for your patience. We have been investigating this issue and we believe we have a fix prepared. Please watch this issue for further updates on when the fix will be released. We expect this to be some time soon, within the next few weeks.

            Regards,
            JIRA Agile Team

            Michael Tokar added a comment - Thank you everyone for your patience. We have been investigating this issue and we believe we have a fix prepared. Please watch this issue for further updates on when the fix will be released. We expect this to be some time soon, within the next few weeks. Regards, JIRA Agile Team

            Michael Tokar added a comment -

            intersol_OLD that caching setting is only applied to the Classic Boards.

            Michael Tokar added a comment - intersol_OLD that caching setting is only applied to the Classic Boards.

            intersol_old added a comment -

            Does this has anything to do with the issue caching in Agile plugin, now is configured to 50.000 but we do have ~300.000 issues on our JIRA instance. The documentation doesn't say much about it https://confluence.atlassian.com/display/AGILE/Specifying+your+JIRA+Agile+General+Configuration?focusedCommentId=664995483&#comment-664995483

            intersol_old added a comment - Does this has anything to do with the issue caching in Agile plugin, now is configured to 50.000 but we do have ~300.000 issues on our JIRA instance. The documentation doesn't say much about it https://confluence.atlassian.com/display/AGILE/Specifying+your+JIRA+Agile+General+Configuration?focusedCommentId=664995483&#comment-664995483

            intersol_old added a comment -

            Please take a look at https://answers.atlassian.com/questions/8620924/how-to-make-agile-board-planning-loading-to-a-decent-performance-level as it presents the performance problem as being more like a design issue.

            intersol_old added a comment - Please take a look at https://answers.atlassian.com/questions/8620924/how-to-make-agile-board-planning-loading-to-a-decent-performance-level as it presents the performance problem as being more like a design issue.

            Yannick Bilodeau added a comment -

            Dieter : Same thing here: After a fresh start, Jira take about 2s to load our big board (not as big as your, but big too !). Just wait some days and you will see your 2s goes up to 4s and 6s and... after ~ 2 month our take 16s to load ! Just restart Jira and Bang : back to 2s.

            Actually on Jira 6.2.3. Planning to passe at 6.3.4 today.

            Yannick Bilodeau added a comment - Dieter : Same thing here: After a fresh start, Jira take about 2s to load our big board (not as big as your, but big too !). Just wait some days and you will see your 2s goes up to 4s and 6s and... after ~ 2 month our take 16s to load ! Just restart Jira and Bang : back to 2s. Actually on Jira 6.2.3. Planning to passe at 6.3.4 today.

            Dieter Greiner added a comment - - edited

            It's not clear to me why the same board on JIRA 6.3.4 / Agile 6.5.0 takes so much longer than on JIRA 5.2.11/Agile 6.3.6.1. It's 12s on JIRA 6 and 2s on JIRA5. Restricting the work sub filter to a project didn't help.

            The JIRA 6.3.4 is an exact copy of the JIRA 5.2.11 system (1020 projects, 2000 fields, 600000 issues). Agile 6.3.6.1 should be affected by the bug as well but it doesn't seem to.

            What could be the reason for JIRA 5.2.11 being so much faster? I can confirm that i see the mentioned stack trace while the board is being loaded.

            Update
            After i have restarted the JIRA 6 system performance is the same as in JIRA 5. Strange ...

            Dieter Greiner added a comment - - edited It's not clear to me why the same board on JIRA 6.3.4 / Agile 6.5.0 takes so much longer than on JIRA 5.2.11/Agile 6.3.6.1. It's 12s on JIRA 6 and 2s on JIRA5. Restricting the work sub filter to a project didn't help. The JIRA 6.3.4 is an exact copy of the JIRA 5.2.11 system (1020 projects, 2000 fields, 600000 issues). Agile 6.3.6.1 should be affected by the bug as well but it doesn't seem to. What could be the reason for JIRA 5.2.11 being so much faster? I can confirm that i see the mentioned stack trace while the board is being loaded. Update After i have restarted the JIRA 6 system performance is the same as in JIRA 5. Strange ...

            Ashok Manthina added a comment -

            Can we running Jira 6.3.1 and hitting severe performance issues because of this, can we please get a backport to this issue asap.
            -am

            Ashok Manthina added a comment - Can we running Jira 6.3.1 and hitting severe performance issues because of this, can we please get a backport to this issue asap. -am

            Sergey Svishchev added a comment -

            Cool. Perhaps "Manage Boards" and "Go To Board" issues will be fixed in this unknown release as well.

            Also, please consider making board refresh frequency configurable and/or make it stop refreshing after a while, like Structure plugin does – these polls take away valuable resources from the app.

            Sergey Svishchev added a comment - Cool. Perhaps "Manage Boards" and "Go To Board" issues will be fixed in this unknown release as well. Also, please consider making board refresh frequency configurable and/or make it stop refreshing after a while, like Structure plugin does – these polls take away valuable resources from the app.

              Unassigned Unassigned
              714061b26763 JamieA
              Affected customers:
              73 This affects my team
              Watchers:
              99 Start watching this issue

                Created:
                Updated:
                Resolved: