Status: Closed (View Workflow)
Affects Version/s: 6.4.13, 6.4.14, 7.2.6
Introduced in Version:6.04
Support reference count:5
Symptom Severity:Severity 1 - Critical
Bug Fix Policy:
Lucene Search with unlimited PageFilter causes unnecessarily memory allocation for enterprise instances (1M+ issues).
In many places unlimited PageFilter is used in JQL queries (Agile board, plugins such as JQL Tricks). This leads to the allocation of an array which has number of documents items, where number of documents is typically greater than number of issues.
For 1M+ issues the object size usually is greater than G1HeapRegionSize/2 so it's humongous object. Take a look at related issue https://jira.atlassian.com/browse/JRA-63315 to see why humongous objects are painful.
- instance with 1M+ issues
- JIRA 6.4.13+
- Configure JVM to use at most 32g (exclusive, for xms and xmx). It might be easier to use less memory and then the problem should be visible with less issues.
- Enable GC logging: -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintReferenceGC -XX:+PrintAdaptiveSizePolicy
- Any search with unlimited PageFilter is fine. It can be Agile board (default filter seems to use unlimited result set) or eg. linkedIssuesInQuery function from JQL Tricks plugin.
- Single request is not enough. Try to do thousands of them within a minute.
- Observe in GC logs:
and eventually frequent Humongous Allocation will cause high memory pressure and could lead to frequent Full GC:
Search does not cause Humongous Allocation without the good reason.
Humongous Allocations are present, and also Full GC (Allocation Failure) may happen slowing down the whole instance.
As part of the fix, additional logging will be be implemented. To enable it:
- com.atlassian.jira.issue.search.providers.LuceneSearchProvider should be set to DEBUG level.
It logs only queries which try to fetch more than 1000 documents.
For some cases, increasing xms may help to fit into bigger G1HeapRegionSize.