Uploaded image for project: 'Crowd Data Center'
  1. Crowd Data Center
  2. CWD-4267

(Embedded) Crowd can run out of memory when syncing large numbers of groups and/or users

XMLWordPrintable

      When syncing large numbers of users or groups from a remote directory, Crowd loads all updated users in memory when updating the cache, which can lead to {{OutOfMemoryError}}s.

      All CacheRefresher implementations seem to follow the same pattern:

      • retrieve all (updated) users from the remote directory
      • for each user that needs to be added - create a new Crowd user in memory
      • for each user that needs to be updated - create a new Crowd user in memory
      • bulk create / update / delete the Crowd users in database.

      A similar pattern is followed for the groups. If the remote directory has many users or groups, Crowd sync can use a lot of memory. We've seen 175 Mb in use in a Stash support case where an OutOfMemoryError was triggered.

      The logic in the various CacheRefreshers should be updated to fetch a limited number of users/groups at a time (say 5000), process those and then move on to the next batch.

              Unassigned Unassigned
              mheemskerk Michael Heemskerk (Inactive)
              Votes:
              2 Vote for this issue
              Watchers:
              12 Start watching this issue

                Created:
                Updated:
                Resolved: