On large instances the createmeta REST endpoint can take a very long amount of time to return results or cause JIRA to OOM

Summary

Executing requests to /jira/rest/api/2/issue/createmeta?params on larger instances can take over a minute to complete and generate large content (500MB+). In some cases JIRA can run out of memory or have heavy GC pressure. At this stage we believe there are 2 possible problems:

1. In case of standard requests with projects.issuetypes parameters, problem is caused by duplicated data for issue-types, see Duplicate content below for details.
2. With fields added (createmeta?projects.issuetypes.fields), it's more likely that JIRA can run out of memory. The main problem is that objects representing either issue types or custom fields are not reused among projects. So theoretically this issue could be reproduced with one multi-select custom field with thousands of values and a thousand of projects.

Environment

The following data exists in the test instance:

Database Statistics
Issues          417547
Projects        1481
Custom Fields   1203
Workflows       324
Attachments     0
Comments        2921131
Users           17225
Groups          985

And 323 issue types.

For projects.issuetypes.fields paraments

Having instance with multi-select CF with large number of values magnifies the problem.
So problem is a multiplication of factors:

• Number of CustomFields values x Number of IssueTypes x Number of Projects having that

Steps to Reproduce

1. Setup or use an instance with similar data sizes.
2. Query /rest/api/2/issue/createmeta?projects.issuetypes, for example https://jira.atlassian.com/rest/api/2/issue/createmeta?projects.issuetypes.

Expected Results

The query returns in a reasonable timeframe. No JVM memory pressure.

Actual Results

The query takes ~60 secs as per the attached screenshot, or in some cases runs out of memory entirely.

Possible cause and room for improvement

We decided to remove this endpoint in Jira 9.0 see Createmeta REST endpoint to be removed

Duplicate content

We have seen that in some cases there are logical duplicates in the optionconfiguration table. This, depending on the number of affected issue types, can cause performance issues itself. To check for such, you can run the following query:

// Check duplicated issuetype, query should return 0
select optionid,fieldconfig,count(1) from optionconfiguration where fieldid = 'issuetype' group by optionid,fieldconfig having count (1) > 1;

// Check all duplicated options, query should return 0
SELECT fieldid,optionid,fieldconfig, count(*) FROM optionconfiguration group by fieldid,optionid,fieldconfig having count(*)>1

If the above returns any rows, you are likely affected by the problem. In this case, please raise a ticket on https://support.atlassian.com, referring to this bug report and ask for assistance on de-duplicating the table.
See related JRASERVER-66306

Workarounds

• Block the call to REST API at proxy level
  • Any integration using this API will be affected, (eg: create JIRA issue from Confluence will not work)
• Increasing the heap could be strategy for small heaps (< 10GB)

Specific workarounds for projects.issuetypes.fields

REST createmeta will show all data to create issue, so you remove the CF from screen it will be not present in the payload for that specific issues.

• If possible, delete unused CustomFields and/or CustomFields values.
  • Make CustomFields more specific:
    • Choose applicable issue types
    • Choose applicable context (Project)
  • Remove the CustomFields from screen which are not require it.