-
Type:
Bug
-
Resolution: Fixed
-
Priority:
Low
-
Affects Version/s: 9.2.1
-
Component/s: Performance
-
1
-
Severity 2 - Major
Issue Summary
Currently, the way that we update the plan cache after a database update has caused slowness. 2 problems has been identified:
- Threads spend a lot of time in PlanRepositoryIndexer.deindex() method because of Collection.removeAll call, which is essentially a linear search. This is particularly bad, because it happens within a 'synchronized' block.
- Threads spend a lot of time in NotificationRuleInvalidator, because identifying plans that require invalidation is a linear search to find matching Notification Rules.
Actual Results
Problem 1: you will see threads busy inside 'NotificationRuleInvalidator' class.
Regarding problem number 2, in the stack trace or threads we might see the following stack trace, as well as other thread being locked with a similar stack trace:
java.util.Collections.eq(Collections.java:5304) java.util.Collections$SingletonSet.contains(Collections.java:4764) java.util.AbstractCollection.removeAll(AbstractCollection.java:375) com.google.common.collect.Synchronized$SynchronizedCollection.removeAll(Synchronized.java:220) com.atlassian.bamboo.plan.cache.index.PlanRepositoryIndexer.lambda$deindex$2(PlanRepositoryIndexer.java:169) com.atlassian.bamboo.plan.cache.index.PlanRepositoryIndexer$$Lambda$1511/1945681902.accept(Unknown Source) java.util.Arrays$ArrayList.forEach(Arrays.java:3880) com.atlassian.bamboo.plan.cache.index.PlanRepositoryIndexer.deindex(PlanRepositoryIndexer.java:169) com.atlassian.bamboo.plan.cache.index.PlanRepositoryIndexer.index(PlanRepositoryIndexer.java:70) com.atlassian.bamboo.plan.cache.index.PlanCacheIndicesImpl.lambda$index$0(PlanCacheIndicesImpl.java:30) com.atlassian.bamboo.plan.cache.index.PlanCacheIndicesImpl$$Lambda$1500/1426985833.accept(Unknown Source) java.util.Arrays$ArrayList.forEach(Arrays.java:3880) com.atlassian.bamboo.plan.cache.index.PlanCacheIndicesImpl.index(PlanCacheIndicesImpl.java:30) com.atlassian.bamboo.plan.cache.ImmutablePlanCacheServiceImpl.lambda$indexPlan$11(ImmutablePlanCacheServiceImpl.java:551) com.atlassian.bamboo.plan.cache.ImmutablePlanCacheServiceImpl$$Lambda$1486/1393355845.run(Unknown Source) com.atlassian.bamboo.variable.CustomVariableContextImpl.withVariableSubstitutor(CustomVariableContextImpl.java:107) com.atlassian.bamboo.plan.cache.ImmutablePlanCacheServiceImpl.indexPlan(ImmutablePlanCacheServiceImpl.java:549) com.atlassian.bamboo.plan.cache.ImmutablePlanCacheServiceImpl.internalGetPlanByKey(ImmutablePlanCacheServiceImpl.java:519) com.atlassian.bamboo.plan.cache.ImmutablePlanCacheServiceImpl.getImmutablePlanByKey(ImmutablePlanCacheServiceImpl.java:233) com.atlassian.bamboo.jira.issuelink.JiraBranchLinkingServiceImpl.linkBranchToIssue(JiraBranchLinkingServiceImpl.java:86) com.atlassian.bamboo.jira.issuelink.JiraBranchLinkingServiceImpl.lambda$linkBranchToIssueIfRequiredAsync$0(JiraBranchLinkingServiceImpl.java:81) com.atlassian.bamboo.jira.issuelink.JiraBranchLinkingServiceImpl$$Lambda$4459/1290372020.run(Unknown Source) com.atlassian.bamboo.utils.BambooRunnables$1.run(BambooRunnables.java:48) com.atlassian.bamboo.security.ImpersonationHelper.runWith(ImpersonationHelper.java:26) com.atlassian.bamboo.security.ImpersonationHelper.runWithSystemAuthority(ImpersonationHelper.java:17) com.atlassian.bamboo.security.ImpersonationHelper$1.run(ImpersonationHelper.java:41) java.lang.Thread.run(Thread.java:748)
Workaround
Currently there is no workaround available.
We already have the fixes:
- we've changed the way we handle Notfication Rules, so now an update enforces cache reload by saving the plan instead of doing a linear search (Internal link BDEV-17867)
- we've reimplementad cache indices so that reindexing does not require a full scan. (Internal link BDEV-17872)
These fixes are currently being tested and we hope to be ready to backport them in the next point release cycle.
- mentioned in
-
Page Loading...