Uploaded image for project: 'Jira Data Center'
  1. Jira Data Center
  2. JRASERVER-69913

Finding membership for groups which has duplicated name can cause slowness access issues to certain pages of Jira instance

XMLWordPrintable

      Issue Summary

      Having duplicated names for the group while having a large number of groups in Jira instance will cause DB "duplicate key value violates unique constraint" error. This will cause the error while loading the data for readUsersInGroupsByDirectory method. Which will lead to different side-effects and possible instance slowness.

      Steps to Reproduce

      1. Create more than:
        • 30 000 memberships for postgresql database,
        • 2 000 MSSQL database

      Expected Results

      Jira works healthy without downtimes

      Actual Results

      A lot of threads are waiting on the activeUsersCountForLicense cache in the DefaultApplicationRoleManager class
      For instance you can find such threads:

      parking to wait for  <0x0000000223ace570> (a java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireShared(AbstractQueuedSynchronizer.java:967)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(AbstractQueuedSynchronizer.java:1283)
	at java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.lock(ReentrantReadWriteLock.java:727)
	at net.sf.ehcache.concurrent.ReadWriteLockSync.lock(ReadWriteLockSync.java:50)
	at net.sf.ehcache.constructs.blocking.BlockingCache.acquiredLockForKey(BlockingCache.java:196)
	at net.sf.ehcache.constructs.blocking.BlockingCache.get(BlockingCache.java:158)
	at com.atlassian.cache.ehcache.LoadingCache.get(LoadingCache.java:79)
	at com.atlassian.cache.ehcache.DelegatingCache.get(DelegatingCache.java:108)
	at com.atlassian.jira.cache.DeferredReplicationCache.get(DeferredReplicationCache.java:48)
	at com.atlassian.jira.application.DefaultApplicationRoleManager.lambda$getUserCountCurrentValue$16(DefaultApplicationRoleManager.java:583)
	at com.atlassian.jira.application.DefaultApplicationRoleManager$$Lambda$915/480723923.apply(Unknown Source)
	at com.atlassian.fugue.Option$Some.fold(Option.java:421)
	at com.atlassian.jira.application.DefaultApplicationRoleManager.getUserCountCurrentValue(DefaultApplicationRoleManager.java:581)
	at com.atlassian.jira.application.DefaultApplicationRoleManager.getUserCount(DefaultApplicationRoleManager.java:574)
	at com.spartez.pokerng.adapters.Jira70ApplicationRoleManager.getJiraSoftwareUserCount(Jira70ApplicationRoleManager.java:10)
	at com.spartez.pokerng.data.AnalyticsStatusDataProvider.guessTier(AnalyticsStatusDataProvider.java:78)
	at com.spartez.pokerng.data.AnalyticsStatusDataProvider.provideData(AnalyticsStatusDataProvider.java:55)
	at com.spartez.pokerng.data.SafeDataProvider$1.write(SafeDataProvider.java:29)
...

      and you can find in the log stack traces which show that we failed to refresh activeUsersCountForLicense cache for instance:

      Caused by: com.atlassian.jira.exception.DataAccessException: org.ofbiz.core.entity.GenericEntityException: Batch entry 16,143 insert into temp8 (item) values ('atlassian sysadmin') was aborted: ERROR: duplicate key value violates unique constraint "temp8_pkey"      Detail: Key (item)=(atlassian sysadmin) already exists.  Call getNextException to see other errors in the batch. (Batch entry 16,143 insert into temp8 (item) values ('atlassian sysadmin') was aborted: ERROR: duplicate key value violates unique constraint "temp8_pkey"      Detail: Key (item)=(atlassian sysadmin) already exists.  Call getNextException to see other errors in the batch.)
	at com.atlassian.jira.ofbiz.DefaultOfBizDelegator.findListIteratorByCondition(DefaultOfBizDelegator.java:401) [classes/:?]
	at com.atlassian.jira.ofbiz.WrappingOfBizDelegator.findListIteratorByCondition(WrappingOfBizDelegator.java:278) [classes/:?]
	at com.atlassian.jira.crowd.embedded.ofbiz.OfBizInternalMembershipDao.findChildrenOfGroupsGeneric(OfBizInternalMembershipDao.java:306) [classes/:?]
	at com.atlassian.jira.crowd.embedded.ofbiz.OfBizInternalMembershipDao.findUserChildrenOfGroups(OfBizInternalMembershipDao.java:289) [classes/:?]
	at com.atlassian.jira.crowd.embedded.ofbiz.OfBizInternalMembershipDao.findUserChildrenOfGroups(OfBizInternalMembershipDao.java:58) [classes/:?]
	at com.atlassian.jira.application.DefaultApplicationRoleManager.readUsersInGroupsByDirectory(DefaultApplicationRoleManager.java:1035) [classes/:?]
	at com.atlassian.jira.application.DefaultApplicationRoleManager.readUsersInGroupsExcluding(DefaultApplicationRoleManager.java:1066) [classes/:?]
	at com.atlassian.jira.application.DefaultApplicationRoleManager.readUsersInGroups(DefaultApplicationRoleManager.java:1048) [classes/:?]
	at com.atlassian.jira.application.DefaultApplicationRoleManager.countUsersInGroups(DefaultApplicationRoleManager.java:1100) [classes/:?]
	at com.atlassian.jira.application.DefaultApplicationRoleManager.countApplicationRoleUsers(DefaultApplicationRoleManager.java:955) [classes/:?]
	at com.atlassian.jira.application.DefaultApplicationRoleManager.countApplicationUsers(DefaultApplicationRoleManager.java:938) [classes/:?]
	at com.atlassian.fugue.Option.map(Option.java:272) [fugue-2.6.0.jar:?]
	at com.atlassian.jira.application.DefaultApplicationRoleManager.lambda$new$1(DefaultApplicationRoleManager.java:197) [classes/:?]
	at com.atlassian.cache.ehcache.wrapper.ValueProcessorAtlassianCacheLoaderDecorator.load(ValueProcessorAtlassianCacheLoaderDecorator.java:26) [atlassian-cache-ehcache-3.2.0.jar:?]
	at com.atlassian.cache.ehcache.LoadingCache.getFromLoader(LoadingCache.java:133) [atlassian-cache-ehcache-3.2.0.jar:?]
	at com.atlassian.cache.ehcache.SynchronizedLoadingCacheDecorator.synchronizedLoad(SynchronizedLoadingCacheDecorator.java:29) [atlassian-cache-ehcache-3.2.0.jar:?]
	at com.atlassian.cache.ehcache.LoadingCache.loadValueAndReleaseLock(LoadingCache.java:101) [atlassian-cache-ehcache-3.2.0.jar:?]
	at com.atlassian.cache.ehcache.LoadingCache.get(LoadingCache.java:80) [atlassian-cache-ehcache-3.2.0.jar:?]
	at com.atlassian.cache.ehcache.DelegatingCache.get(DelegatingCache.java:108) [atlassian-cache-ehcache-3.2.0.jar:?]	... 300 more
 

      You may also see 500 access issues when loading the Manage Apps page or Versions and Licenses:

      Workaround

      1. Decrease number of groups to less than 30 000. It can be checked how many records is in database by this sql:

      SELECT count (*),lower_child_name
FROM cwd_membership
WHERE directory_id=DIRECTORYID
AND membership_type='GROUP_GROUP'
AND lower_parent_name in (
	SELECT lower_group_name
	FROM cwd_group
	WHERE directory_id=DIRECTORYID
)

      2. Filter out unneeded groups/users from LDAP by editing LDAP filters.
      or

      3. Make sure that you do not have duplication in the lower names of the group and nested groups.

            Unassigned Unassigned
            ddudziak Stasiu
            Votes:
            2 Vote for this issue
            Watchers:
            7 Start watching this issue

              Created:
              Updated:
              Resolved: