Uploaded image for project: 'Jira Server and Data Center'
  1. Jira Server and Data Center
  2. JRASERVER-67502

RequestCachingGroupManager may accumulate too many objects in RequestCache during processing large user sets

    XMLWordPrintable

    Details

    • Fixed in Long Term Support Release/s:
    • Introduced in Version:
      7.02
    • Support reference count:
      16
    • Symptom Severity:
      Severity 2 - Major
    • UIS:
      15
    • Current Status:
      Hide
      Atlassian Update – 13-03-2019

      Hi everyone,

      We have recently reviewed this issue and the overall interest in the problem. We acknowledge that the issue has a high impact on the software teams and we're already working on it. We plan to include this fix in our Enterprise Releases (7.6.x and 7.13.x - please watch the fix versions for details) and latest bugfix releases.

      Atlassian will continue to watch this issue for further updates, so please feel free to share your thoughts in the comments.

      Thank you,
      Grażyna Kaszkur
      Jira Server Product Management

      Show
      Atlassian Update – 13-03-2019 Hi everyone, We have recently reviewed this issue and the overall interest in the problem. We acknowledge that the issue has a high impact on the software teams and we're already working on it. We plan to include this fix in our Enterprise Releases (7.6.x and 7.13.x - please watch the fix versions for details) and latest bugfix releases. Atlassian will continue to watch this issue for further updates, so please feel free to share your thoughts in the comments. Thank you, Grażyna Kaszkur Jira Server Product Management

      Description

      Summary

      Class RequestCachingGroupManager may accumulate too many objects in RequestCache during processing large user sets

      Environment

      • Large Jira with 50k+ active users

      Steps to Reproduce

      This is one of the ways to trigger the user search

      1. Create 50k users
      2. Limit assignable permission for specific projectA for a very small subset of users
      3. Trigger assignable user search for projectA

      REST call:

      1. Trigger call to user/assignable/search (or user/assignable/multiProjectSearch)
        • Eg: rest/api/latest/user/assignable/search?username=&projectKeys=AG&issueKey=AG-13&maxResults=50&startAt=0&_=1529917579021

      Expected Results

      Memory usage by a thread serving the request a fairly low: up to 100MB

      Actual Results

      • Memory usage by a thread serving the request a quite high: 500MB+
      • If a large number of same requests are triggered at the same time, this may lead to OOM error due to exhausting of the heap.

      Notes

      • when Jira searches over user base and tries to findUsers according to specific criteria, it accumulates the different data (usernameToGroups, usernameToGroupNames,usernameToGroupNamesInLowerCase) in requestCache as a side-effect.
      • If user search base is quite large and search criteria is too specific (too strict) than a large number of Username objects will be touched, that will lead to accumulating a large number of objects in requestCache and large memory usage by a thread.
      • Note that when a thread finishes its work memory will be released, so it's not a memory leak.

      Workaround

      • None
      • Partial workarounds (if possible):
        • reduce active base
        • use more specific search

      Note about the fix

      The fix is not enabled by default. It will be applied only when setting the JVM argument:

      -Djira.cache.request.bounded.enabled=true
      

      Jira node restart is required.
      To verify that the option was enabled correctly see if the Jira application log contains message:

      2019-03-19 15:06:16,969 http-nio-8080-exec-1 INFO anonymous 906x1x1 - 0:0:0:0:0:0:0:1 / [c.a.j.cache.request.RequestCacheConfiguration] System property jira.cache.request.bounded.enabled is enabled and your requests scope caches are bounded with maxsize=1000
      

      The fix is about introducing the limit to number of objects stored by http request in it's cache. The default limit is 1000, but can be modified by setting JVM argument, e.g.:

      -Djira.cache.request.bounded.size=2000
      

      The limit won't be applied if jira.cache.request.bounded.enabled is not set.

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              tzwierzchowski Tomasz Zwierzchowski
              Reporter:
              ayakovlev@atlassian.com Andriy Yakovlev
              Votes:
              8 Vote for this issue
              Watchers:
              21 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: