Uploaded image for project: 'Jira Data Center'
  1. Jira Data Center
  2. JRASERVER-69913

Finding membership for groups which has duplicated name can cause slowness access issues to certain pages of Jira instance

    XMLWordPrintable

Details

    Description

      Issue Summary

      Having duplicated names for the group while having a large number of groups in Jira instance will cause DB "duplicate key value violates unique constraint" error. This will cause the error while loading the data for readUsersInGroupsByDirectory method. Which will lead to different side-effects and possible instance slowness.

      Steps to Reproduce

      1. Create more than:
        • 30 000 memberships for postgresql database,
        • 2 000 MSSQL database

      Expected Results

      Jira works healthy without downtimes

      Actual Results

      A lot of threads are waiting on the activeUsersCountForLicense cache in the DefaultApplicationRoleManager class
      For instance you can find such threads:

      parking to wait for  <0x0000000223ace570> (a java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync)
      	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
      	at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
      	at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireShared(AbstractQueuedSynchronizer.java:967)
      	at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(AbstractQueuedSynchronizer.java:1283)
      	at java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.lock(ReentrantReadWriteLock.java:727)
      	at net.sf.ehcache.concurrent.ReadWriteLockSync.lock(ReadWriteLockSync.java:50)
      	at net.sf.ehcache.constructs.blocking.BlockingCache.acquiredLockForKey(BlockingCache.java:196)
      	at net.sf.ehcache.constructs.blocking.BlockingCache.get(BlockingCache.java:158)
      	at com.atlassian.cache.ehcache.LoadingCache.get(LoadingCache.java:79)
      	at com.atlassian.cache.ehcache.DelegatingCache.get(DelegatingCache.java:108)
      	at com.atlassian.jira.cache.DeferredReplicationCache.get(DeferredReplicationCache.java:48)
      	at com.atlassian.jira.application.DefaultApplicationRoleManager.lambda$getUserCountCurrentValue$16(DefaultApplicationRoleManager.java:583)
      	at com.atlassian.jira.application.DefaultApplicationRoleManager$$Lambda$915/480723923.apply(Unknown Source)
      	at com.atlassian.fugue.Option$Some.fold(Option.java:421)
      	at com.atlassian.jira.application.DefaultApplicationRoleManager.getUserCountCurrentValue(DefaultApplicationRoleManager.java:581)
      	at com.atlassian.jira.application.DefaultApplicationRoleManager.getUserCount(DefaultApplicationRoleManager.java:574)
      	at com.spartez.pokerng.adapters.Jira70ApplicationRoleManager.getJiraSoftwareUserCount(Jira70ApplicationRoleManager.java:10)
      	at com.spartez.pokerng.data.AnalyticsStatusDataProvider.guessTier(AnalyticsStatusDataProvider.java:78)
      	at com.spartez.pokerng.data.AnalyticsStatusDataProvider.provideData(AnalyticsStatusDataProvider.java:55)
      	at com.spartez.pokerng.data.SafeDataProvider$1.write(SafeDataProvider.java:29)
      ...
      

      and you can find in the log stack traces which show that we failed to refresh activeUsersCountForLicense cache for instance:

      Caused by: com.atlassian.jira.exception.DataAccessException: org.ofbiz.core.entity.GenericEntityException: Batch entry 16,143 insert into temp8 (item) values ('atlassian sysadmin') was aborted: ERROR: duplicate key value violates unique constraint "temp8_pkey"      Detail: Key (item)=(atlassian sysadmin) already exists.  Call getNextException to see other errors in the batch. (Batch entry 16,143 insert into temp8 (item) values ('atlassian sysadmin') was aborted: ERROR: duplicate key value violates unique constraint "temp8_pkey"      Detail: Key (item)=(atlassian sysadmin) already exists.  Call getNextException to see other errors in the batch.)
      	at com.atlassian.jira.ofbiz.DefaultOfBizDelegator.findListIteratorByCondition(DefaultOfBizDelegator.java:401) [classes/:?]
      	at com.atlassian.jira.ofbiz.WrappingOfBizDelegator.findListIteratorByCondition(WrappingOfBizDelegator.java:278) [classes/:?]
      	at com.atlassian.jira.crowd.embedded.ofbiz.OfBizInternalMembershipDao.findChildrenOfGroupsGeneric(OfBizInternalMembershipDao.java:306) [classes/:?]
      	at com.atlassian.jira.crowd.embedded.ofbiz.OfBizInternalMembershipDao.findUserChildrenOfGroups(OfBizInternalMembershipDao.java:289) [classes/:?]
      	at com.atlassian.jira.crowd.embedded.ofbiz.OfBizInternalMembershipDao.findUserChildrenOfGroups(OfBizInternalMembershipDao.java:58) [classes/:?]
      	at com.atlassian.jira.application.DefaultApplicationRoleManager.readUsersInGroupsByDirectory(DefaultApplicationRoleManager.java:1035) [classes/:?]
      	at com.atlassian.jira.application.DefaultApplicationRoleManager.readUsersInGroupsExcluding(DefaultApplicationRoleManager.java:1066) [classes/:?]
      	at com.atlassian.jira.application.DefaultApplicationRoleManager.readUsersInGroups(DefaultApplicationRoleManager.java:1048) [classes/:?]
      	at com.atlassian.jira.application.DefaultApplicationRoleManager.countUsersInGroups(DefaultApplicationRoleManager.java:1100) [classes/:?]
      	at com.atlassian.jira.application.DefaultApplicationRoleManager.countApplicationRoleUsers(DefaultApplicationRoleManager.java:955) [classes/:?]
      	at com.atlassian.jira.application.DefaultApplicationRoleManager.countApplicationUsers(DefaultApplicationRoleManager.java:938) [classes/:?]
      	at com.atlassian.fugue.Option.map(Option.java:272) [fugue-2.6.0.jar:?]
      	at com.atlassian.jira.application.DefaultApplicationRoleManager.lambda$new$1(DefaultApplicationRoleManager.java:197) [classes/:?]
      	at com.atlassian.cache.ehcache.wrapper.ValueProcessorAtlassianCacheLoaderDecorator.load(ValueProcessorAtlassianCacheLoaderDecorator.java:26) [atlassian-cache-ehcache-3.2.0.jar:?]
      	at com.atlassian.cache.ehcache.LoadingCache.getFromLoader(LoadingCache.java:133) [atlassian-cache-ehcache-3.2.0.jar:?]
      	at com.atlassian.cache.ehcache.SynchronizedLoadingCacheDecorator.synchronizedLoad(SynchronizedLoadingCacheDecorator.java:29) [atlassian-cache-ehcache-3.2.0.jar:?]
      	at com.atlassian.cache.ehcache.LoadingCache.loadValueAndReleaseLock(LoadingCache.java:101) [atlassian-cache-ehcache-3.2.0.jar:?]
      	at com.atlassian.cache.ehcache.LoadingCache.get(LoadingCache.java:80) [atlassian-cache-ehcache-3.2.0.jar:?]
      	at com.atlassian.cache.ehcache.DelegatingCache.get(DelegatingCache.java:108) [atlassian-cache-ehcache-3.2.0.jar:?]	... 300 more
       

      You may also see 500 access issues when loading the Manage Apps page or Versions and Licenses:

      Workaround

      1. Decrease number of groups to less than 30 000. It can be checked how many records is in database by this sql:

      SELECT count (*),lower_child_name
      FROM cwd_membership
      WHERE directory_id=DIRECTORYID
      AND membership_type='GROUP_GROUP'
      AND lower_parent_name in (
      	SELECT lower_group_name
      	FROM cwd_group
      	WHERE directory_id=DIRECTORYID
      )

      2. Filter out unneeded groups/users from LDAP by editing LDAP filters.
      or

      3. Make sure that you do not have duplication in the lower names of the group and nested groups.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              ddudziak Stasiu
              Votes:
              2 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: