very poor performance of /servicedesk/customer/portals page when there are many portals

XMLWordPrintable

    • Type: Bug
    • Resolution: Fixed
    • Priority: Low
    • 3.6.1
    • Affects Version/s: 3.5.1
    • Component/s: Customer Portal
    • None
    • Severity 2 - Major

      Summary

      The GET /portals request loads too slowly on an instance with numerous ServiceDesks.

      Environment

      Reproduced on an instance with over 400 ServiceDesk projects and customer portals.

      Steps to Reproduce

      On an instance with hundreds of Service Desks, try to access <jira-base-url>/servicedesk/customer/portals as a customer who has access to only 2 or 3 of those portals.

      Expected Results

      This page should load reasonably quickly especially when the user only has access to only a few portals

      Actual Results

      The page takes several seconds to load, sometimes up to 10 seconds or more. Specifically GET /portals request alone would often take over 5 seconds to execute on the server.

      Capturing thread dumps every second or two while loading the page may reveal the following stacktrace as long running: 

      "http-nio-8096-exec-15" #190 daemon prio=5 os_prio=0 tid=0x00007fdd24018800 nid=0x2cd0 runnable [0x00007fddf175b000]
   java.lang.Thread.State: RUNNABLE
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:150)
	at java.net.SocketInputStream.read(SocketInputStream.java:121)
	at org.postgresql.core.VisibleBufferedInputStream.readMore(VisibleBufferedInputStream.java:143)
	at org.postgresql.core.VisibleBufferedInputStream.ensureBytes(VisibleBufferedInputStream.java:112)
	at org.postgresql.core.VisibleBufferedInputStream.read(VisibleBufferedInputStream.java:71)
	at org.postgresql.core.PGStream.ReceiveChar(PGStream.java:272)
	at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1659)
	at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255)
	- locked <0x00000000f61a2048> (a org.postgresql.core.v3.QueryExecutorImpl)
	at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:510)
	at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386)
	at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271)
	at org.apache.commons.dbcp2.DelegatingPreparedStatement.executeQuery(DelegatingPreparedStatement.java:83)
	at org.apache.commons.dbcp2.DelegatingPreparedStatement.executeQuery(DelegatingPreparedStatement.java:83)
	at com.querydsl.sql.AbstractSQLQuery.iterateSingle(AbstractSQLQuery.java:370)
	at com.querydsl.sql.AbstractSQLQuery.iterate(AbstractSQLQuery.java:342)
	at com.querydsl.sql.ProjectableSQLQuery.fetchOne(ProjectableSQLQuery.java:372)
	at com.querydsl.core.support.FetchableQueryBase.fetchFirst(FetchableQueryBase.java:51)
	at com.atlassian.servicedesk.internal.feature.organization.dao.CustomerOrganizationMemberQStoreImpl.lambda$isMemberOfAnyOrganizationInProject$7(CustomerOrganizationMemberQStoreImpl.java:232)
	at com.atlassian.servicedesk.internal.feature.organization.dao.CustomerOrganizationMemberQStoreImpl$$Lambda$1512/59608834.apply(Unknown Source)
	at com.atlassian.pocketknife.internal.querydsl.DatabaseAccessorImpl.lambda$execute$0(DatabaseAccessorImpl.java:68)
...
com.atlassian.servicedesk.internal.feature.organization.dao.CustomerOrganizationMemberQStoreImpl.isMemberOfAnyOrganizationInProject(CustomerOrganizationMemberQStoreImpl.java:222)
	at com.atlassian.servicedesk.internal.feature.organization.member.CustomerOrganizationMemberManagerImpl.isMemberOfAnyOrganizationsInProject(CustomerOrganizationMemberManagerImpl.java:193)
	at com.atlassian.servicedesk.internal.permission.security.type.CustomerPortalOnlyPermissionChecker.isMemberOfAnyOrganisationsInProject(CustomerPortalOnlyPermissionChecker.java:155)
	at com.atlassian.servicedesk.internal.permission.security.type.CustomerPortalOnlyPermissionChecker.allowUserToAccessPortal(CustomerPortalOnlyPermissionChecker.java:145)
	at com.atlassian.servicedesk.internal.permission.security.type.CustomerPortalOnlyPermissionChecker.hasCustomerPortalOnlyPermissionForProject(CustomerPortalOnlyPermissionChecker.java:136)
	at com.atlassian.servicedesk.internal.permission.security.type.CustomerPortalOnlyPermissionChecker.hasCustomerPortalOnlyPermission(CustomerPortalOnlyPermissionChecker.java:111)
	at com.atlassian.servicedesk.internal.permission.security.type.CustomerPortalOnlySecurityTypeImpl.hasPermission(CustomerPortalOnlySecurityTypeImpl.java:93)
	at com.atlassian.jira.permission.DefaultPermissionSchemeManager.lambda$hasSchemePermission$3(DefaultPermissionSchemeManager.java:435)
	at com.atlassian.jira.permission.DefaultPermissionSchemeManager$$Lambda$777/2140287359.hasPermission(Unknown Source)
	at com.atlassian.jira.permission.DefaultPermissionSchemeManager.hasSchemePermission(DefaultPermissionSchemeManager.java:584)
	at com.atlassian.jira.permission.DefaultPermissionSchemeManager.hasSchemePermission(DefaultPermissionSchemeManager.java:435)
	at com.atlassian.jira.security.DefaultPermissionManager.doProjectPermissionCheck(DefaultPermissionManager.java:185)
	at com.atlassian.jira.security.DefaultPermissionManager.hasPermission(DefaultPermissionManager.java:107)
	at com.atlassian.jira.security.WorkflowBasedPermissionManager.hasPermission(WorkflowBasedPermissionManager.java:110)
	at com.atlassian.jira.security.ApplicationRequiredPermissionManager.lambda$hasPermission$5(ApplicationRequiredPermissionManager.java:97)
	at com.atlassian.jira.security.ApplicationRequiredPermissionManager$$Lambda$887/1799350819.getAsBoolean(Unknown Source)
	at com.atlassian.jira.security.ApplicationRequiredPermissionManager.checkUserHasApplicationOrFalse(ApplicationRequiredPermissionManager.java:182)
	at com.atlassian.jira.security.ApplicationRequiredPermissionManager.hasPermission(ApplicationRequiredPermissionManager.java:97)
...
	at com.atlassian.servicedesk.internal.user.permission.ServiceDeskPermissions.canViewProject(ServiceDeskPermissions.scala:58)
	at com.atlassian.servicedesk.internal.user.permission.ServiceDeskPermissions.hasInternalCustomerAccess(ServiceDeskPermissions.scala:245)
	at com.atlassian.servicedesk.internal.user.permission.ServiceDeskPermissions.canCreateRequest(ServiceDeskPermissions.scala:144)
	at com.atlassian.servicedesk.internal.feature.customer.portal.InternalPortalServiceImpl.lambda$hasCustomerPortalPermission$13(InternalPortalServiceImpl.java:217)
...
com.atlassian.servicedesk.internal.feature.customer.portal.InternalPortalServiceImpl.getCustomerVisiblePortals(InternalPortalServiceImpl.java:83)
	at com.atlassian.servicedesk.internal.feature.customer.request.list.RequestListSearchQueryFactoryImpl.getAllVisiblePortals(RequestListSearchQueryFactoryImpl.java:83)
	at com.atlassian.servicedesk.internal.feature.customer.request.list.RequestListProviderImpl.getAllVisiblePortals(RequestListProviderImpl.java:151)
	at com.atlassian.servicedesk.internal.feature.customer.request.list.RequestListProviderImpl.getApprovalCount(RequestListProviderImpl.java:190)
	at com.atlassian.servicedesk.internal.feature.customer.portal.providers.user.UserResponseProviderDataHelper.getApprovalsCount(UserResponseProviderDataHelper.java:91)
...
	at com.atlassian.servicedesk.internal.rest.CustomerResponseFactory.createResponse(CustomerResponseFactory.scala:62)
	at com.atlassian.servicedesk.internal.web.CustomerPageRenderer.createPageParams(CustomerPageRenderer.scala:91)

      Notes

      The bottleneck is with permission checking for organization membership, which is done for each portal against the database by isMemberOfAnyOrganizationInProject():

      select count("AO_54307E_ORGANIZATION_MEMBER"."ID") from "public"."AO_54307E_ORGANIZATION_MEMBER" "AO_54307E_ORGANIZATION_MEMBER" inner join "public"."AO_54307E_ORGANIZATION" "AO_54307E_ORGANIZATION" on "AO_54307E_ORGANIZATION"."ID" = "AO_54307E_ORGANIZATION_MEMBER"."ORGANIZATION_ID" inner join "public"."AO_54307E_ORGANIZATION_PROJECT" "AO_54307E_ORGANIZATION_PROJECT" on "AO_54307E_ORGANIZATION_PROJECT"."ORGANIZATION_ID" = "AO_54307E_ORGANIZATION"."ID" where "AO_54307E_ORGANIZATION_PROJECT"."PROJECT_ID" = ? and "AO_54307E_ORGANIZATION_MEMBER"."USER_KEY" = ? limit ?

      While the query itself may be executed very quickly, this begins to have a significant impact with instances with large number of portals because the query is being executed multiple times against the database directly, while executing the GET /portals request to ServiceDesk.

      Workaround

      The workaround is to disable Service Desk's Customer Organization feature on the instance. Just a heads up on the main implication of disabling this feature, basically users will now need to manually list out members of their organization they would like to share their request with and cannot simply share with the entire organization once this feature is disabled.

      Steps to disable the feature

      1. Log in to the following path on your test instance: <jira-base-url>/secure/admin/SiteDarkFeatures!Default.jspa
      2. Add the following parameter sd.customer.orgs.group.participants.disabled to the Enable Dark Feature field and click Add

              Assignee:
              Sergey Zaporozhtsev (Inactive)
              Reporter:
              Taiwo Akindele (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Created:
                Updated:
                Resolved: