Ability to have an efficient group membership retrieval in Directory API v2 for Organization wide group membership

XMLWordPrintable

      Current State:

      With the current Directory API v2 there are only two approaches that can be taken to retrieve the complete list of group memberships of an organization. 

      Approach 1 (Atlassian-recommended):
      1. GET /directories/{directory_id}/users?includeResourcesCount=false
      2. For each user, GET /directories/{directory_id}/groups?accountIds=[userAccountId]

      Approach 2 (our alternative):
      1. GET /directories/{directory_id}/users?includeResourcesCount=false
      2. GET /directories/{directory_id}/groups?includeResourcesCount=false
      3. For each group, GET /directories/{directory_id}/users?groupIds=[groupId]&includeResourcesCount=false

      Results:
      includeResourcesCount=false resolved timeouts on the users endpoint.

      We can now paginate all users at limit=100 in ~34 minutes.

      Neither group membership strategy is viable at a large scale:

      Example:
      v1 (expand=GROUPS): ~15 min, ~2,000 calls (production baseline)
      v2 per-user group lookup: ~13+ hours, ~98K+ calls (120 req/min limit, 97,964 users)
      v2 per-group member lookup: ~22+ hours, ~10K+ calls (50 req/min limit, 8,815 groups)

      Proposed Future State:

      Efficient group membership retrieval in Directory API v2, similar to (v1 expand=GROUPS equivalent needed for migration).

       

      Alternative (e.g. Third Party Plugin):

      None

              Assignee:
              Unassigned
              Reporter:
              Angel Buendia
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated: