On one specific customer instance, the Insight Azure import takes 39 hours to complete.

The are 31,454 groups to sync from Azure, debug stats show that the majority of the time (37 hours avg) is spent reading data in Azure, and not anything such as writing the data that has been fetched to Insight (Assets).

The server CPU and Memory utilization are also very low which suggests that this is not a performance problem within Jira or Insight (Assets).

• 33 avg minutes for object type user
• 37 hours only for the object type group

Feedback from Development during DoS escalation:

This group import is slow because as mentioned above there are 31454 group to sync. When loading groups, we call external Azure API to load page by page and we load 25 group at a time. This means there must be 1259 API calls to load all the groups from Azure to sync. Call 1259 API calls to sync from azure should not take 39 hours. However, for each group `insight-azure-integration` is making an extra API call to get the list of `members` for each group. This results in at least extra 31454 API calls to sync the members of each group. This leads to minimum total 31454 + 1259 = 32173 API requests to Azure which explains why it was so slow.

In addition, 32173 API calls is just an ideal number of minimum requests that will be created, in reality, the number of requests is larger, and we can't guess as it depends on the real situation of customer Azure data.

For example, If there are 1000 groups in 31454 groups which those 1000 groups have 250 members each. When loading members for each group, we are also fetching page by page and 25 per page. This means it's will be taking 250 / 25 * 1000 = 10000 requests to get members of those 1000 groups. Then the final total number of requests to Azure needed is 1259 + 10000 + (31454 - 1000) = 41713 API requests to Azure needed to sync everything.

However, even with the minimum number of requests to be made to azure (32173 requests), it's already massive enough to be slow in progressing while the true number may even be much higher.

You may ask why the server looks to be IDLE, CPU and memory chart looks fine. It is because calling requests to Azure API when syncing does not take much memory or CPU or disk space. It is just taking time.

This suggestion has been opened to gather impact from other customers so that our JSM DC team can prioritize and plan into our roadmap when it gets enough traction.

Suggested improvement, on top of changing how we fetch data from Azure, could we build in some kind of progress bar that users can see the actual progress of the import instead of it seeming like it is stuck.