Cache invalidation and reindexing require linear searches through the cache content

XMLWordPrintable

    • 1
    • Severity 2 - Major

      Issue Summary

      Currently, the way that we update the plan cache after a database update has caused slowness. 2 problems has been identified:

      1. Threads spend a lot of time in PlanRepositoryIndexer.deindex() method because of Collection.removeAll call, which is essentially a linear search. This is particularly bad, because it happens within a 'synchronized' block.
      2. Threads spend a lot of time in NotificationRuleInvalidator, because identifying plans that require invalidation is a linear search to find matching Notification Rules.

      Actual Results

      Problem 1: you will see threads busy inside 'NotificationRuleInvalidator' class.

      Regarding problem number 2, in the stack trace or threads we might see the following stack trace, as well as other thread being locked with a similar stack trace:

      java.util.Collections.eq(Collections.java:5304)
      java.util.Collections$SingletonSet.contains(Collections.java:4764)
      java.util.AbstractCollection.removeAll(AbstractCollection.java:375)
      com.google.common.collect.Synchronized$SynchronizedCollection.removeAll(Synchronized.java:220)
      com.atlassian.bamboo.plan.cache.index.PlanRepositoryIndexer.lambda$deindex$2(PlanRepositoryIndexer.java:169)
      com.atlassian.bamboo.plan.cache.index.PlanRepositoryIndexer$$Lambda$1511/1945681902.accept(Unknown Source)
      java.util.Arrays$ArrayList.forEach(Arrays.java:3880)
      com.atlassian.bamboo.plan.cache.index.PlanRepositoryIndexer.deindex(PlanRepositoryIndexer.java:169)
      com.atlassian.bamboo.plan.cache.index.PlanRepositoryIndexer.index(PlanRepositoryIndexer.java:70)
      com.atlassian.bamboo.plan.cache.index.PlanCacheIndicesImpl.lambda$index$0(PlanCacheIndicesImpl.java:30)
      com.atlassian.bamboo.plan.cache.index.PlanCacheIndicesImpl$$Lambda$1500/1426985833.accept(Unknown Source)
      java.util.Arrays$ArrayList.forEach(Arrays.java:3880)
      com.atlassian.bamboo.plan.cache.index.PlanCacheIndicesImpl.index(PlanCacheIndicesImpl.java:30)
      com.atlassian.bamboo.plan.cache.ImmutablePlanCacheServiceImpl.lambda$indexPlan$11(ImmutablePlanCacheServiceImpl.java:551)
      com.atlassian.bamboo.plan.cache.ImmutablePlanCacheServiceImpl$$Lambda$1486/1393355845.run(Unknown Source)
      com.atlassian.bamboo.variable.CustomVariableContextImpl.withVariableSubstitutor(CustomVariableContextImpl.java:107)
      com.atlassian.bamboo.plan.cache.ImmutablePlanCacheServiceImpl.indexPlan(ImmutablePlanCacheServiceImpl.java:549)
      com.atlassian.bamboo.plan.cache.ImmutablePlanCacheServiceImpl.internalGetPlanByKey(ImmutablePlanCacheServiceImpl.java:519)
      com.atlassian.bamboo.plan.cache.ImmutablePlanCacheServiceImpl.getImmutablePlanByKey(ImmutablePlanCacheServiceImpl.java:233)
      com.atlassian.bamboo.jira.issuelink.JiraBranchLinkingServiceImpl.linkBranchToIssue(JiraBranchLinkingServiceImpl.java:86)
      com.atlassian.bamboo.jira.issuelink.JiraBranchLinkingServiceImpl.lambda$linkBranchToIssueIfRequiredAsync$0(JiraBranchLinkingServiceImpl.java:81)
      com.atlassian.bamboo.jira.issuelink.JiraBranchLinkingServiceImpl$$Lambda$4459/1290372020.run(Unknown Source)
      com.atlassian.bamboo.utils.BambooRunnables$1.run(BambooRunnables.java:48)
      com.atlassian.bamboo.security.ImpersonationHelper.runWith(ImpersonationHelper.java:26)
      com.atlassian.bamboo.security.ImpersonationHelper.runWithSystemAuthority(ImpersonationHelper.java:17)
      com.atlassian.bamboo.security.ImpersonationHelper$1.run(ImpersonationHelper.java:41)
      java.lang.Thread.run(Thread.java:748)

      Workaround

      Currently there is no workaround available.

      We already have the fixes:

      1. we've changed the way we handle Notfication Rules, so now an update enforces cache reload by saving the plan instead of doing a linear search (Internal link BDEV-17867)
      2. we've reimplementad cache indices so that reindexing does not require a full scan. (Internal link BDEV-17872)
        These fixes are currently being tested and we hope to be ready to backport them in the next point release cycle.

            Assignee:
            Marcin Gardias
            Reporter:
            Giovanna Fragoso
            Votes:
            2 Vote for this issue
            Watchers:
            7 Start watching this issue

              Created:
              Updated:
              Resolved: