-
Type:
Bug
-
Resolution: Fixed
-
Priority:
High
-
Affects Version/s: 6.2
-
Component/s: JQL
-
6.02
When Lucene runs in high concurrent environments (more than 500K issues) and huge load we can detect this in our threaddumps
java.lang.Thread.State: BLOCKED (on object monitor)
at org.apache.lucene.index.SegmentCoreReaders.getTermsReader(SegmentCoreReaders.java:123)
- waiting to lock <0x00007f6df4420858> (a org.apache.lucene.index.SegmentCoreReaders)
at org.apache.lucene.index.SegmentTermDocs.seek(SegmentTermDocs.java:57)
at org.apache.lucene.search.MultiTermQueryWrapperFilter.getDocIdSet(MultiTermQueryWrapperFilter.java:120)
at org.apache.lucene.search.ConstantScoreQuery$ConstantWeight.scorer(ConstantScoreQuery.java:139)
at org.apache.lucene.search.BooleanQuery$BooleanWeight.scorer(BooleanQuery.java:298)
at org.apache.lucene.search.BooleanQuery$BooleanWeight.scorer(BooleanQuery.java:298)
at org.apache.lucene.search.BooleanQuery$BooleanWeight.scorer(BooleanQuery.java:298)
at org.apache.lucene.search.IndexSearcher.searchWithFilter(IndexSearcher.java:542)
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:532)
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:320)
at com.atlassian.jira.index.DelegateSearcher.search(DelegateSearcher.java:153)
at com.atlassian.jira.issue.search.providers.LuceneSearchProvider.search(LuceneSearchProvider.java:342)
at com.atlassian.jira.issue.search.providers.LuceneSearchProvider.search(LuceneSearchProvider.java:141)
at sun.reflect.GeneratedMethodAccessor843.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
And when we look what's the lock in SegmentCoreReaders we find this:
synchronized TermInfosReader getTermsReader() { if (tis != null) { return tis; } else { return tisNoIndex; } }
There are literally no computations here! So this means that this method needs to be invoked thousands of times for each search to be so much blocked. After short time of Googling we can find that we are not the only ones that suffer from these synchronization block: https://issues.apache.org/jira/browse/LUCENE-3653 - it seems to be fixed in Lucene 3.6+ there is also patch we can use on our current version of Lucene https://issues.apache.org/jira/secure/attachment/12508358/LUCENE-3653.patch
NOTE : This only affect big customers.
If you don't see any outage or problems in your instance there is no need to apply the patch.