"availableProjects" REST endpoint requires optimization for big Jira instances

Issue Summary

With the release of Jira 9.7.0, a new enhanced UI for the configuration of customer fields context had been introduced:

• Jira Software 9.7.x release notes - Configuring contexts for custom fields: new design, enhanced functionality

The functionality of this enhancement relies on /rest/internal/2/field/<custom_field>/context/<content_id>/availableProjects endpoint which returns a lot of potentially redundant data, increasing the size of the returned response and duration of loading web page for huge Jira instances (~4K projects). While loading the context configuration page this endpoint is requested twice (with maxResults=1 and maxResults=200 parameters) which makes up to >60% of overall page loading.

For the test data set (~4K projects) during verification of this issue, the size of returned (for maxResults=200) made up ~1.8Mb. However, even for maxResults=1 duration of request execution is approximately the same.

This is reproducible on Data Center: (yes)

Steps to Reproduce

1. Create a new 9.7.0+ instance with a big dataset that includes ~4K projects (the overall number of issues doesn't play a big role here).
2. Create a custom field and edit its context.

Expected Results

Prior to the enhancement introduced at 9.7.0 simplified version of UI was used for editing custom filed context. The same instance (with the same dataset) with simplified UI can load context editing web page at least twice faster (~7s vs ~15s) for the test data set).

Actual Results

Despite greatly increasing the usability and functionality of the custom field editing field, the new UI also introduces a notable delay in page loading for the huge Jira instances (~4K+ projects)

Test Results

HAR comparison of old UI vs new UI (~7s vs ~15s):

> curl -s -w 'Establish Connection: %{time_connect}s\nTime Prestransfer: %{time_pretransfer}\nTTFB: %{time_starttransfer}s\nTotal: %{time_total}s\n' --connect-timeout 1800 -o out.txt -H 'Authorization: Bearer MTI5MzMzMzcyMjI1Ot/VjS7FQb+jNxBnyfcI+1lf/hKP' -H 'Content-type: application/json' -H "Cache-Control: no-cache" -d '{"ignoredProjectIds":[]}' https://linux-58417.prod.atl-cd.net/jira/rest/internal/2/field/customfield_10300/context/10600/availableProjects\?query\=\&maxResults\=1\&startAt\=0

Establish Connection: 0.047273s
Time Prestransfer: 0.163919
TTFB: 5.518025s
Total: 5.553788s
> ls -lah out.txt
-rw-r--r--  1 <user>  <group>   9.5K Dec  5 11:03 out.txt

> curl -s -w 'Establish Connection: %{time_connect}s\nTime Prestransfer: %{time_pretransfer}\nTTFB: %{time_starttransfer}s\nTotal: %{time_total}s\n' --connect-timeout 1800 -o out.txt -H 'Authorization: Bearer MTI5MzMzMzcyMjI1Ot/VjS7FQb+jNxBnyfcI+1lf/hKP' -H 'Content-type: application/json' -H "Cache-Control: no-cache" -d '{"ignoredProjectIds":[]}' https://linux-58417.prod.atl-cd.net/jira/rest/internal/2/field/customfield_10300/context/10600/availableProjects\?query\=\&maxResults\=200\&startAt\=0

Establish Connection: 0.052482s
Time Prestransfer: 0.195032
TTFB: 5.341751s
Total: 5.743963s
> ls -lah out.txt
-rw-r--r--  1 <user>  <group>   1.8M Dec  5 11:03 out.txt

Workaround

If the custom field context editing page loading is extremely big, you could temporarily switch to the old (pre-9.7.0) UI:

1. Navigate to the dark features configuration page <BASE_URL>/secure/admin/SiteDarkFeatures!default.jspa (How to manage dark features in Jira).
2. Add jira.customfields.dual.list.box.disabled feature there.

This option will disable the new UI for the context configuration introduced in 9.7 and revert it to the previous outlook/functionality. To revert changes, please simply delete jira.customfields.dual.list.box.disabled from the dark features configuration page (adding and removing features could be performed in runtime and doesn't require a restart of the Jira).