RequestCachingGroupManager may accumulate too many objects in RequestCache during processing large user sets

Summary

Class RequestCachingGroupManager may accumulate too many objects in RequestCache during processing large user sets

Environment

• Large Jira with 50k+ active users

Steps to Reproduce

This is one of the ways to trigger the user search

1. Create 50k users
2. Limit assignable permission for specific projectA for a very small subset of users
3. Trigger assignable user search for projectA

REST call:

1. Trigger call to user/assignable/search (or user/assignable/multiProjectSearch)
   • Eg: rest/api/latest/user/assignable/search?username=&projectKeys=AG&issueKey=AG-13&maxResults=50&startAt=0&_=1529917579021

Expected Results

Memory usage by a thread serving the request a fairly low: up to 100MB

Actual Results

• Memory usage by a thread serving the request a quite high: 500MB+
• If a large number of same requests are triggered at the same time, this may lead to OOM error due to exhausting of the heap.

Notes

• when Jira searches over user base and tries to findUsers according to specific criteria, it accumulates the different data (usernameToGroups, usernameToGroupNames,usernameToGroupNamesInLowerCase) in requestCache as a side-effect.
• If user search base is quite large and search criteria is too specific (too strict) than a large number of Username objects will be touched, that will lead to accumulating a large number of objects in requestCache and large memory usage by a thread.
• Note that when a thread finishes its work memory will be released, so it's not a memory leak.

Workaround

• None
• Partial workarounds (if possible):
  • reduce active base
  • use more specific search

Note about the fix

The fix is not enabled by default. It will be applied only when setting the JVM argument:

-Djira.cache.request.bounded.enabled=true

Jira node restart is required.
To verify that the option was enabled correctly see if the Jira application log contains message:

2019-03-19 15:06:16,969 http-nio-8080-exec-1 INFO anonymous 906x1x1 - 0:0:0:0:0:0:0:1 / [c.a.j.cache.request.RequestCacheConfiguration] System property jira.cache.request.bounded.enabled is enabled and your requests scope caches are bounded with maxsize=1000

The fix is about introducing the limit to number of objects stored by http request in it's cache. The default limit is 1000, but can be modified by setting JVM argument, e.g.:

-Djira.cache.request.bounded.size=2000

The limit won't be applied if jira.cache.request.bounded.enabled is not set.