Uploaded image for project: 'Crowd'
  1. Crowd
  2. CWD-3809

ApplicationServiceGeneric#searchNestedGroupRelationships performs n+1 queries in order to shadow users

    XMLWordPrintable

Details

    Description

      Summary

      When there are multiple directories and membership aggregation is disabled for an applicatoin, group membership queries require an isCanonical check on each user. In order to do this isCanonical finds the user by name in order to shallow the user. This results in n+1 queries to the database. This can cause massive performance issues when the host application (in this case Stash) when calculating licensed user counts.

      Steps to Reproduce

      It is not entirely clear at this point how to accurately replicate the problem. It involves configuring nested groups and multiple user directories, then using the application such that it retrieves user groups.

      Expected Results

      Crowd retrieves nested groups with minimal performance impact.

      Actual Results

      There is a notable performance impact on the system and it presents with symptoms of performance problems.

      Verification

      Check for the presence of multiple threads utilising com.atlassian.crowd.manager.application.ApplicationServiceGeneric.isCanonical and/or com.atlassian.crowd.manager.application.ApplicationServiceGeneric.searchNestedGroupRelationships. For example from JIRA:

      "http-bio-8080-exec-12889" daemon prio=10 tid=0x00007f9e7cb26800 nid=0xa4a runnable [0x00007f9e3125e000]
         java.lang.Thread.State: RUNNABLE
      	at java.util.HashMap$HashIterator.nextEntry(Unknown Source)
      	at java.util.HashMap$KeyIterator.next(Unknown Source)
      	at java.util.AbstractCollection.addAll(Unknown Source)
      	at com.atlassian.crowd.model.application.DirectoryMapping.<init>(DirectoryMapping.java:70)
      	at com.atlassian.jira.crowd.embedded.ofbiz.OfBizApplication$1.apply(OfBizApplication.java:71)
      	at com.atlassian.jira.crowd.embedded.ofbiz.OfBizApplication$1.apply(OfBizApplication.java:68)
      	at com.google.common.collect.Lists$TransformingRandomAccessList.get(Lists.java:451)
      	at java.util.AbstractList$Itr.next(Unknown Source)
      	at com.google.common.collect.Iterators$8.next(Iterators.java:781)
      	at com.google.common.collect.Iterators$7.computeNext(Iterators.java:644)
      	at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141)
      	at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:136)
      	at com.google.common.collect.Iterators.getNext(Iterators.java:860)
      	at com.google.common.collect.Iterables.getFirst(Iterables.java:759)
      	at com.atlassian.crowd.manager.application.ApplicationServiceGeneric.isCanonical(ApplicationServiceGeneric.java:1638)
      	at com.atlassian.crowd.manager.application.ApplicationServiceGeneric$1.apply(ApplicationServiceGeneric.java:1587)
      	at com.google.common.collect.Iterators$7.computeNext(Iterators.java:645)
      	at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141)
      	at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:136)
      	at com.google.common.collect.Iterables$10$1.hasNext(Iterables.java:884)
      	at com.google.common.collect.Lists.newArrayList(Lists.java:138)
      	at com.google.common.collect.Lists.newArrayList(Lists.java:119)
      	at com.atlassian.crowd.manager.application.AggregatorImpl.constrainResults(ResultsAggregator.java:174)
      	at com.atlassian.crowd.manager.application.ApplicationServiceGeneric.searchNestedGroupRelationships(ApplicationServiceGeneric.java:1582)
      	at com.atlassian.crowd.embedded.core.CrowdServiceImpl.searchNestedGroupRelationships(CrowdServiceImpl.java:238)
      	at com.atlassian.crowd.embedded.core.CrowdServiceImpl.search(CrowdServiceImpl.java:158)
      	at com.atlassian.crowd.embedded.core.DelegatingCrowdService.search(DelegatingCrowdService.java:60)
      	at com.atlassian.crowd.embedded.core.FilteredCrowdServiceImpl.search(FilteredCrowdServiceImpl.java:143)
      	at com.atlassian.jira.security.groups.DefaultGroupManager.getUsersInGroup(DefaultGroupManager.java:107)
      	at com.atlassian.jira.security.roles.actor.GroupRoleActorFactory$GroupRoleActor.getUsers(GroupRoleActorFactory.java:79)
      	at com.atlassian.jira.security.roles.DefaultRoleActorsImpl.getUsers(DefaultRoleActorsImpl.java:51)
      	at com.atlassian.jira.security.roles.CachingProjectRoleAndActorStore$CachedRoleActors.getUsers(CachingProjectRoleAndActorStore.java:415)
      	at com.atlassian.jira.notification.type.ProjectRoleSecurityAndNotificationType.getUsersFromRole(ProjectRoleSecurityAndNotificationType.java:235)
      	at com.atlassian.jira.notification.type.ProjectRoleSecurityAndNotificationType.getUsers(ProjectRoleSecurityAndNotificationType.java:138)
      	at com.atlassian.jira.scheme.AbstractSchemeManager.getUsers(AbstractSchemeManager.java:819)
      	at com.atlassian.jira.permission.WorkflowBasedPermissionSchemeManager.getUsers(WorkflowBasedPermissionSchemeManager.java:73)
      	at com.atlassian.jira.bc.user.search.DefaultAssigneeService$AssignableUsers.findAllAsSet(DefaultAssigneeService.java:428)
      	at com.atlassian.jira.bc.user.search.DefaultAssigneeService$AssignableUsers.findAll(DefaultAssigneeService.java:423)
      	at com.atlassian.jira.bc.user.search.DefaultAssigneeService.getSuggestedAssignees(DefaultAssigneeService.java:70)
      	at com.atlassian.jira.issue.fields.Assignees.optionsForFrotherControl(Assignees.java:70)
      	at com.atlassian.jira.issue.fields.AssigneeSystemField.getEditHtml(AssigneeSystemField.java:168)
      	at com.atlassian.jira.issue.fields.screen.AbstractFieldScreenLayoutItem.getEditHtml(AbstractFieldScreenLayoutItem.java:78)
      	at com.atlassian.jira.issue.fields.screen.FieldScreenRenderLayoutItemImpl.getEditHtml(FieldScreenRenderLayoutItemImpl.java:62)
      	at com.atlassian.jira.issue.fields.rest.FieldHtmlFactoryImpl.getEditFields(FieldHtmlFactoryImpl.java:151)
      	at sun.reflect.GeneratedMethodAccessor1026.invoke(Unknown Source)
      

      Notes

      In JIRA, this can also cause performance problems. For example, some user actions in JIRA like com.atlassian.jira.components.issueeditor.action.AjaxIssueEditAction.doDefault and com.atlassian.jira.issue.fields.screen.FieldScreenRenderLayoutItemImpl.getEditHtml, there are many) require JIRA to calculate the permissions, which involves calling com.atlassian.jira.security.groups.DefaultGroupManager.getUsersInGroup. JIRA delegates this operation to ApplicationServiceGeneric.searchNestedGroupRelationships in its embedded Crowd. Embedded Crowd is dependent on these classes from Crowd. This can be an expensive calculation which increases with the number of users. Unfortunately the loop inside that method causes many queries to the database, many of them to fetch the directory mappings which are not cached.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              jhinch jhinch (Atlassian)
              Votes:
              7 Vote for this issue
              Watchers:
              15 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: