Currently, when connecting an external LDAP/AD directory with JIRA, it seems JIRA attempts to make a copy of the users/groups in the identity store into the JIRA database and then periodically checks to make sure they are synchronized. This solution does not scale well with large directories.

Out-of-the-box JIRA crashed with OutOfMemory error after 15 minutes. After tripling the memory (512min/768max) JIRA got further:

[atlassian.crowd.directory.DbCachingRemoteDirectoryCache] synchronised [ 24241 ] users in [ 4713513ms ]
[directory.ldap.cache.UsnChangedCacheRefresher] found [ 57440 ] remote groups in [ 6665358ms ]

...but then crashed again with another OOM error 3 hours after that last log entry while still processing groups.
(I was trying to wait to open this issue until I had a completely successful synchronization process to be able to report final timings, but to double the memory again requires adding more physical memory to the machine which will take some time...I thought I would get this issue opened now and report back any additional findings later.)

Proposed solution: Create users in the local database "only when first needed" rather than up-front. Subsequent synchronization tasks should only be performed on existing users in the database.
I'm also using the term "only when first needed" to be intentionally vague. This would most likely mean when the user first logs on. But, there could also be other scenarios when a user should be created in the local db as well. (An admin is setting up security and needs to be able to search for and locate specific users, but since the users haven't logged in they don't exist in the db. Users should not be required to logon as the only means to get a user created in the db; instead, there should also be a way to cause a specific user to get created "on demand".)