Lock contentions in didyoumean code

XMLWordPrintable

      We've just experienced two outages due to lock contentions in the did you mean search code.

      The issue seems to be caused only under load, possibly thanks to some race condition.

      Turning off the feature immediately help our site to recover.

      I'm attaching thread dumps from both outages, both pointing to locks in the didyoumean code like this one:

      "service-j2ee-127" - Thread t@208
         java.lang.Thread.State: WAITING on edu.emory.mathcs.backport.java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock@7b60ad80
              at java.lang.Object.wait(Native Method)
              at java.lang.Object.wait(Object.java:485)
              at edu.emory.mathcs.backport.java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.lock(ReentrantReadWriteLock.java:415)
              at com.atlassian.bonnie.LuceneConnection.grabReadLock(LuceneConnection.java:495)
              at com.atlassian.bonnie.LuceneConnection.withReader(LuceneConnection.java:190)
              at com.atlassian.confluence.search.didyoumean.lucene.LuceneWordSuggester.isMemberOfVocabulary(LuceneWordSuggester.java:76)
              at com.atlassian.confluence.search.didyoumean.lucene.LuceneWordSuggester.suggestSimilar(LuceneWordSuggester.java:103)
              at com.atlassian.confluence.search.didyoumean.lucene.LuceneWordSuggester.suggest(LuceneWordSuggester.java:71)
              at com.atlassian.confluence.search.didyoumean.lucene.LuceneWordSuggester.suggest(LuceneWordSuggester.java:58)
              at com.atlassian.confluence.search.didyoumean.lucene.LuceneWordSuggester.suggest(LuceneWordSuggester.java:50)
              at com.atlassian.confluence.search.didyoumean.QuerySuggester.getSuggestion(QuerySuggester.java:88)
              at com.atlassian.confluence.search.actions.SearchSiteAction.execute(SearchSiteAction.java:249)
              at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:168)
              at com.opensymphony.xwork.interceptor.AroundInterceptor.intercept(AroundInterceptor.java:35)
              at com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:165)
              at com.opensymphony.xwork.interceptor.DefaultWorkflowInterceptor.intercept(DefaultWorkflowInterceptor.java:55)
              at com.atlassian.confluence.core.ConfluenceWorkflowInterceptor.intercept(ConfluenceWorkflowInterceptor.java:35)
      

        1. threaddump-1245786559419.tdump.gz
          62 kB
          Igor Minar
        2. threaddump-1245777602973.tdump.gz
          30 kB
          Igor Minar

              Assignee:
              Unassigned
              Reporter:
              Igor Minar
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Created:
                Updated:
                Resolved: