The DefaultPermissionVoterFactory caches user permissions as PermissionVoters using a RequestLocalCache. Unfortunately, RequestLocalCache is unbounded. Certain requests may cause a large number of permission objects to be loaded into memory and cached until the request completes.

There is certainly reason for keeping the current user's PermissionVoter around until the request is completed but we may want to limit how many other user objects are kept in cache.

Workaround
Increasing the maximum Java heap size (-Xmx) may avoid an OutOfMemoryError associated with this problem