As reported by kavian:

We discovered scalability problems with the implementation of four of the gadgets provided by jira-gadgets-plugin:

• Two-Dimensional Filter Statistics
• Issue Statistics
• Pie Chart
• Heat Map

All four of these use a similar implementation where the internal value objects for the "count-by" field are loaded into memory for all issues returned by the filter...and this is done in a loop, so the objects will continue to be loaded into memory with no opportunity for garbage collection. In cases where there are a large number of values, this can cause JIRA to run out of memory, which causes the application to fail and require restart.

Most of our outages have corresponded to the use of one of these gadgets where the "count-by" field was a user field (e.g. Reporter, Assignee, or a custom User Picker), so this leads us to believe that there may also be scalability issues with the new ApplicationUser implementations. However, we aren't as certain about this last part, since at least one of our outages did not involve a user field.

We have mitigated the immediate impact of this problem by notifying our users of this behaviour, and asking them not to use these gadgets with large datasets. However, we would like this to be addressed properly to eliminate the risk of accidental outage, and to allow our users to use these gadgets as needed.

The following is the details of the execution path that leads to outage (this is the trace for the Issue Statistics gadget, but a similar path exists for the other three gadgets):

• the /rest/gadget/1.0/stats/generate call maps to com.atlassian.jira.gadgets.system.StatsResource.getData

• on line 145 in the getData method, a com.atlassian.jira.web.bean.StatisticAccessorBean object is created

• on line 148 in the getData method, the getWrapper method of the StatisticAccessorBean object is called, and this is what essentially pulls the data (the rest of the getData method is prepping the results for output)
  • the getData method also creates a StatisticsMapper object (line 144) and passes it to the getWrapper method
    • for counts by Assignee, this is a com.atlassian.jira.issue.statistics.AssigneeStatisticsMapper object, which is really a simple extension of com.atlassian.jira.issue.statistics.UserStatisticsMapper

• after the getWrapper method pulls the data, it iterates through the results and translates each GenericValue to a real object (line 198 in StatisticAccessorBean) using the getValueFromLuceneField method of the StatisticsMapper

• UserStatisticsManager.getValueFromLuceneField is implemented using an instance of its inner class UserManagerResolver

• UserManagerResolver.apply calls JIRA's UserManager.getUserByKey method to get the ApplicationUser instance, and passes that into ApplicationUsers.toDirectoryUser to get the embedded-Crowd User object
  • this is what is being repeated for every Assignee found in the search result
  • the embedded-Crowd User objects (and possibly the ApplicationUser objects) are not released for garbage collection until the entire user operation has completed