Details
-
Bug
-
Resolution: Fixed
-
Medium
-
6.5.0
-
None
-
6.05
-
-
Description
Jira Agile is building JQL queries asking all the Sprints from the DB every time and I think 3 times for every request.
There are 3 JQL functions affected:
OpenSprintsJqlFunction
FutureSprintsJqlFunction
ClosedSprintsJqlFunction
All of them inherits from AbstractSprintStateJqlFunction
where it is calling in getValues() the method SprintManagerImpl.getSprints(EnumSet<Sprint.State> states)
which calls the following method :
@Override
@Nonnull
public ServiceOutcome<Collection<Sprint>> getAllSprints()
{
final Collection<Sprint> result = Lists.newArrayList();
sprintDao.loadAll(new Consumer<SprintAO>()
{
@Override
public void consume(SprintAO sprintAO)
{
final Option<Sprint> sprint = sprintCache.get(sprintAO.getId());
if (sprint.isDefined())
{
result.add(sprint.get());
}
}
});
return ok(result);
}
As you can see here all the sprints are loaded into memory.
while inside it is using :
public void loadAll(final Consumer<SprintAO> consumer) { ao.stream(SprintAO.class, new EntityStreamCallback<SprintAO, Long>() { @Override public void onRowRead(SprintAO sprintAO) { consumer.consume(sprintAO); } }); }
AO does not implement a proper stream function with a DB iterator.
This is loading the entire ResultSet into memory and iterating over the callback.
We need to fix this.
We should load all the items in memory the first time the Object is created ( that was as it was working in 6.3.5) and move invalidate the entire cache when a Sprint is created or deleted.
I've talked with other developers here and using the SprintCacheLoader is useless here cause you need the entire Sprint info all together.
You can see a threaddump here from one of our customers:
"http-bio-8080-exec-1633" daemon prio=10 tid=0x0000000009c44000 nid=0x12b6 runnable [0x000000004aa4a000] java.lang.Thread.State: RUNNABLE at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(Unknown Source) at java.net.SocketInputStream.read(Unknown Source) at oracle.net.ns.Packet.receive(Packet.java:283) at oracle.net.ns.DataPacket.receive(DataPacket.java:103) at oracle.net.ns.NetInputStream.getNextPacket(NetInputStream.java:230) at oracle.net.ns.NetInputStream.read(NetInputStream.java:175) at oracle.net.ns.NetInputStream.read(NetInputStream.java:100) at oracle.net.ns.NetInputStream.read(NetInputStream.java:85) at oracle.jdbc.driver.T4CSocketInputStreamWrapper.readNextPacket(T4CSocketInputStreamWrapper.java:123) at oracle.jdbc.driver.T4CSocketInputStreamWrapper.read(T4CSocketInputStreamWrapper.java:79) at oracle.jdbc.driver.T4CMAREngine.unmarshalUB1(T4CMAREngine.java:1122) at oracle.jdbc.driver.T4CMAREngine.unmarshalSB1(T4CMAREngine.java:1099) at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:288) at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:191) at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:523) at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:207) at oracle.jdbc.driver.T4CPreparedStatement.fetch(T4CPreparedStatement.java:1084) at oracle.jdbc.driver.OracleResultSetImpl.close_or_fetch_from_next(OracleResultSetImpl.java:359) - locked <0x00000004e7a418a0> (a oracle.jdbc.driver.T4CConnection) at oracle.jdbc.driver.OracleResultSetImpl.next(OracleResultSetImpl.java:263) - locked <0x00000004e7a418a0> (a oracle.jdbc.driver.T4CConnection) at org.apache.commons.dbcp.DelegatingResultSet.next(DelegatingResultSet.java:207) at org.apache.commons.dbcp.DelegatingResultSet.next(DelegatingResultSet.java:207) at net.java.ao.EntityManager.stream(EntityManager.java:910) at net.java.ao.EntityManager.stream(EntityManager.java:860) at com.atlassian.activeobjects.internal.EntityManagedActiveObjects.stream(EntityManagedActiveObjects.java:223) at com.atlassian.activeobjects.osgi.DelegatingActiveObjects.stream(DelegatingActiveObjects.java:113) at sun.reflect.GeneratedMethodAccessor1287.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:307) at org.springframework.osgi.service.importer.support.internal.aop.ServiceInvoker.doInvoke(ServiceInvoker.java:58) at org.springframework.osgi.service.importer.support.internal.aop.ServiceInvoker.invoke(ServiceInvoker.java:62) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171) at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:131) at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:119) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171) at org.springframework.osgi.service.util.internal.aop.ServiceTCCLInterceptor.invokeUnprivileged(ServiceTCCLInterceptor.java:56) at org.springframework.osgi.service.util.internal.aop.ServiceTCCLInterceptor.invoke(ServiceTCCLInterceptor.java:39) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171) at org.springframework.osgi.service.importer.support.LocalBundleContextAdvice.invoke(LocalBundleContextAdvice.java:59) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171) at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:131) at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:119) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171) at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204) at com.sun.proxy.$Proxy3335.stream(Unknown Source) at com.atlassian.greenhopper.service.sprint.SprintDao.loadAll(SprintDao.java:20) at com.atlassian.greenhopper.service.sprint.SprintManagerImpl.getAllSprints(SprintManagerImpl.java:94) at com.atlassian.greenhopper.service.sprint.SprintManagerImpl.getSprints(SprintManagerImpl.java:114) at sun.reflect.GeneratedMethodAccessor1412.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at com.atlassian.activeobjects.tx.TransactionalProxy.invoke(TransactionalProxy.java:79) at com.atlassian.activeobjects.tx.TransactionalProxy.invoke(TransactionalProxy.java:39) at com.sun.proxy.$Proxy3339.getSprints(Unknown Source) at com.atlassian.greenhopper.customfield.sprint.AbstractSprintsStateJqlFunction.getValues(AbstractSprintsStateJqlFunction.java:43) at com.atlassian.jira.jql.operand.FunctionOperandHandler$3.call(FunctionOperandHandler.java:73) at com.atlassian.jira.jql.operand.FunctionOperandHandler$3.call(FunctionOperandHandler.java:69) at com.atlassian.ozymandias.SafePluginPointAccess.call(SafePluginPointAccess.java:187) at com.atlassian.jira.jql.operand.FunctionOperandHandler.getValues(FunctionOperandHandler.java:68) at com.atlassian.jira.jql.operand.FunctionOperandHandler.getValues(FunctionOperandHandler.java:27) at com.atlassian.jira.jql.operand.DefaultJqlOperandResolver.getValues(DefaultJqlOperandResolver.java:96) at com.atlassian.greenhopper.customfield.searcher.GhGenericClauseQueryFactory.getRawValues(GhGenericClauseQueryFactory.java:89) at com.atlassian.greenhopper.customfield.searcher.GhGenericClauseQueryFactory.getQuery(GhGenericClauseQueryFactory.java:62) at com.atlassian.greenhopper.customfield.sprint.SprintClauseQueryFactory.getQuery(SprintClauseQueryFactory.java:33) at com.atlassian.jira.jql.query.QueryVisitor.visit(QueryVisitor.java:155) at com.atlassian.jira.jql.query.QueryVisitor.visit(QueryVisitor.java:25) at com.atlassian.query.clause.TerminalClauseImpl.accept(TerminalClauseImpl.java:162) at com.atlassian.jira.jql.query.QueryVisitor.visit(QueryVisitor.java:91) at com.atlassian.jira.jql.query.QueryVisitor.visit(QueryVisitor.java:25) at com.atlassian.query.clause.AndClause.accept(AndClause.java:34) at com.atlassian.jira.jql.query.QueryVisitor.createQuery(QueryVisitor.java:72) at com.atlassian.jira.jql.query.DefaultLuceneQueryBuilder.createLuceneQuery(DefaultLuceneQueryBuilder.java:39) at com.atlassian.jira.issue.search.providers.LuceneSearchProvider.search(LuceneSearchProvider.java:311) at com.atlassian.jira.issue.search.providers.LuceneSearchProvider.search(LuceneSearchProvider.java:155) at sun.reflect.GeneratedMethodAccessor774.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at com.atlassian.plugin.osgi.hostcomponents.impl.DefaultComponentRegistrar$ContextClassLoaderSettingInvocationHandler.invoke(DefaultComponentRegistrar.java:129) at com.sun.proxy.$Proxy375.search(Unknown Source) at sun.reflect.GeneratedMethodAccessor774.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at com.atlassian.plugin.osgi.bridge.external.HostComponentFactoryBean$DynamicServiceInvocationHandler.invoke(HostComponentFactoryBean.java:154) at com.sun.proxy.$Proxy375.search(Unknown Source) at com.atlassian.greenhopper.service.issue.IssueDataServiceImpl.findImpl(IssueDataServiceImpl.java:166) at com.atlassian.greenhopper.service.issue.IssueDataServiceImpl.findWithServiceOutcome(IssueDataServiceImpl.java:48) at com.atlassian.greenhopper.web.rapid.work.WorkDataFactory.getWorkDataIssueCountAndLastUpdated(WorkDataFactory.java:210) at com.atlassian.greenhopper.web.rapid.work.WorkDataFactory.getAllData(WorkDataFactory.java:117) at com.atlassian.greenhopper.web.rapid.work.WorkResource$1.call(WorkResource.java:70) at com.atlassian.greenhopper.web.rapid.work.WorkResource$1.call(WorkResource.java:59) at com.atlassian.greenhopper.web.util.RestCall.response(RestCall.java:48)
This call took 10 seconds and I think most of this is taken by this method.