Uploaded image for project: 'Jira Data Center'
  1. Jira Data Center
  2. JRASERVER-66555

JIRA uses excessive amount of memory during issue export

XMLWordPrintable

    • 7.02
    • 55
    • Severity 2 - Major
    • 277
    • Hide
      Atlassian Update – 2020-02-16

      Dear Jira users,

      This issue has been largely mitigated in the 8.2 release by resolving JRASERVER-68806. The fix was also back ported to versions 7.6.14 and 7.13.4 (our Enterprise Releases). Please watch the fix versions on JRASERVER-68806 for details.

      We're leaving this issue (JRASERVER-66555) as "gathering impact", because while the greatest pain has been remedied, there is still some unwanted memory consumption during issue export. It will only manifest itself when vast amounts of data are exported.

      Note: JRASERVER-68806 was resolved by decreasing the size of the custom field values cache. While it helped the issue export, it has negatively impacted some bulk operations. For more details see JRASERVER-70397.

      Thank you,
      Grazyna Kaszkur
      Product Manager,
      Jira Server and Data Center

      Show
      Atlassian Update – 2020-02-16 Dear Jira users, This issue has been largely mitigated in the 8.2 release by resolving JRASERVER-68806 . The fix was also back ported to versions 7.6.14 and 7.13.4 (our Enterprise Releases). Please watch the fix versions on JRASERVER-68806 for details. We're leaving this issue ( JRASERVER-66555 ) as "gathering impact", because while the greatest pain has been remedied, there is still some unwanted memory consumption during issue export. It will only manifest itself when vast amounts of data are exported. Note: JRASERVER-68806 was resolved by decreasing the size of the custom field values cache. While it helped the issue export, it has negatively impacted some bulk operations. For more details see JRASERVER-70397 . Thank you, Grazyna Kaszkur Product Manager, Jira Server and Data Center

      Summary

      JIRA caches different data in JiraAuthenticationContextImpl and RequestCacheController for each request in ThreadLocal. That data is cleared after each request so there is no problem there, but during large issue export that data can stay around for a long time causing high memory pressure and later high GC pressure.

      Environment

      • JIRA instance with 1M+ issue

      Steps to Reproduce

      1. Access the issue navigator and search for large number of issues
      2. Export these issues to Excel/CSV/XML

      Expected Results

      Application streams the data and uses reasonable amount of memory (~100M)

      Actual Results

      Application caches some data for the duration of the whole request. In some case it can be 2GB+ data, which might cause OOM

      Notes

      Example of Dominator tree from heap dump after OOM

      Objects details
      • Thread ajp-nio-8009-exec-819 - 5GB
        Class Name                                                                                          | Shallow Heap | Retained Heap | Percentage
        ------------------------------------------------------------------------------------------------------------------------------------------------
        org.apache.tomcat.util.threads.TaskThread @ 0x132e509ba8  ajp-nio-8009-exec-819 Native Stack, Thread|          136 | 5,324,618,224 |     30.18%
        |- java.lang.ThreadLocal$ThreadLocalMap @ 0x136e5f3528                                              |           24 | 5,213,775,632 |     29.55%
        |  '- java.lang.ThreadLocal$ThreadLocalMap$Entry[2048] @ 0x14f984b088                               |        8,208 | 5,213,775,608 |     29.55%
        |     |- java.lang.ThreadLocal$ThreadLocalMap$Entry @ 0x12b2acec98                                  |           32 | 2,967,537,480 |     16.82%
        |     |- java.lang.ThreadLocal$ThreadLocalMap$Entry @ 0x12b2acec48                                  |           32 | 2,245,747,520 |     12.73%
        |     |- java.lang.ThreadLocal$ThreadLocalMap$Entry @ 0x135da74120                                  |           32 |        83,272 |      0.00%
        

        Note two large ThreadLocals: 2.9GB and 2.2GB
        requestUrl - /sr/jira.issueviews:searchrequest-excel-all-fields/temp/SearchRequest.xls with empty queryString

      Workaround

      Problem is largely mitigated by the following

        1. JVM_Heap_dom_tree.png
          122 kB
          Andriy Yakovlev [Atlassian]

              Unassigned Unassigned
              ayakovlev@atlassian.com Andriy Yakovlev [Atlassian]
              Votes:
              23 Vote for this issue
              Watchers:
              51 Start watching this issue

                Created:
                Updated:
                Resolved: