Uploaded image for project: 'Jira Server and Data Center'
  1. Jira Server and Data Center
  2. JRASERVER-68439

Improve FieldCache memory utilization for Jira instances with large Lucene



    • UIS:
    • Support reference count:
    • Feedback Policy:
      We collect Jira feedback from various sources, and we evaluate what we've collected when planning our product roadmap. To understand how this piece of feedback will be reviewed, see our Implementation of New Features Policy.


      Problem Definition

      Lucene uses org.apache.lucene.search.FieldCache to improve sort for text fields.
      Lucene loads corresponding structures from disk into memory and stores in FieldCache.

      That means the bigger the Lucene Fields (including Jira CustomFields) size the more memory will be required for them in JVM heap

      • For large Jira instances (4+ mln issues) FieldCache might use up to 9GB+, which adds extra pressure for the JVM

      Suggested Solution

      Optimize memory usage of the FieldCache.
      According to Lucene docs, those structures are gone in latest versions, so upgrade of Lucene 7+ should fix the problem.

      Jira 8.0 no longer uses field caches to store lucene index parts in Java heap. They migrated this functionality to sth called Doc Values which does exactly the same but does not consume JVM heap space - it's all stored on disk and mmaped into process virtual memory.


      Related bog


      Since memory usage is proportional to the size of the Lucene Fields, there are a couple of workarounds here, in the order of preference:

      1. Disable Lucene search index for unused CustomFields
        • You can use the queries provided in this KB to identify the unused custom fields.
        • In order to disable the Lucene search index for these fields, you can 'Edit' the custom filed and change the 'Search Template' to 'None', the Lucene search index for this custom field will be disabled.
      2. Reduce scope for CustomFields (limit for specific projects)
        • You can find the custom fields with global contexts and then change them to specific projects by following the steps mentioned in this KB. Try applying the changes in batches and during off-peak hours, since when one of your users opens an issue that uses a certain custom field, the cache will need to be recreated.


          Issue Links



              Unassigned Unassigned
              ayakovlev@atlassian.com Andriy Yakovlev [Atlassian]
              0 Vote for this issue
              14 Start watching this issue