Loading...

Type: Bug
Resolution: Unresolved
Priority: Low
Fix Version/s: None
Affects Version/s: 9.4.0, 9.4.29, 9.12.0, 9.12.9, 10.3.0, 10.3.9
Component/s: Data Center - Apps
Labels:
None

Introduced in Version:
9.04
Support reference count:
1
Symptom Severity:
Severity 3 - Minor
Bug Fix Policy:
View Atlassian Server bug fix policy

Issue Summary

As a side effect of JRASERVER-73830 it is possible that a plugin upgrade on CMJ can cause several dependent plugins to be disabled and re-enabled. That is also visible in JRASERVER-77849.

In such situation, combined with an instance with large group membership (conditions similar to what's described in JSWSERVER-21588) it's possible that it will take several hours to update the plugin due to lack of caching while checking permission for plugin stop/start/installation life cycle, the stack trace looks like this:

"UpmAsynchronousTaskManager:thread-3" #1909 prio=5 os_prio=0 cpu=166062.50ms elapsed=1128.54s tid=0x0000029be971ded0 nid=0x2c10 runnable  [0x000000db8b3f9000]
   java.lang.Thread.State: RUNNABLE
	at sun.nio.ch.SocketDispatcher.read0(java.base@17.0.14/Native Method)
	at sun.nio.ch.SocketDispatcher.read(java.base@17.0.14/SocketDispatcher.java:46)
	at sun.nio.ch.NioSocketImpl.tryRead(java.base@17.0.14/NioSocketImpl.java:266)
	at sun.nio.ch.NioSocketImpl.implRead(java.base@17.0.14/NioSocketImpl.java:317)
	at sun.nio.ch.NioSocketImpl.read(java.base@17.0.14/NioSocketImpl.java:355)
	at sun.nio.ch.NioSocketImpl$1.read(java.base@17.0.14/NioSocketImpl.java:808)
	at java.net.Socket$SocketInputStream.read(java.base@17.0.14/Socket.java:966)
	at com.microsoft.sqlserver.jdbc.TDSChannel$ProxyInputStream.readInternal(IOBuffer.java:1019)
	at com.microsoft.sqlserver.jdbc.TDSChannel$ProxyInputStream.read(IOBuffer.java:1009)
	at sun.security.ssl.SSLSocketInputRecord.read(java.base@17.0.14/SSLSocketInputRecord.java:484)
	at sun.security.ssl.SSLSocketInputRecord.readHeader(java.base@17.0.14/SSLSocketInputRecord.java:478)
	at sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(java.base@17.0.14/SSLSocketInputRecord.java:70)
	at sun.security.ssl.SSLSocketImpl.readApplicationRecord(java.base@17.0.14/SSLSocketImpl.java:1465)
	at sun.security.ssl.SSLSocketImpl$AppInputStream.read(java.base@17.0.14/SSLSocketImpl.java:1069)
	at com.microsoft.sqlserver.jdbc.TDSChannel.read(IOBuffer.java:2058)
	at com.microsoft.sqlserver.jdbc.TDSReader.readPacket(IOBuffer.java:6617)
	- locked <0x000002919f6c3b98> (a com.microsoft.sqlserver.jdbc.TDSReader)
	at com.microsoft.sqlserver.jdbc.TDSCommand.startResponse(IOBuffer.java:7805)
	at com.microsoft.sqlserver.jdbc.TDSCommand.startResponse(IOBuffer.java:7768)
...
	at com.atlassian.jira.crowd.embedded.ofbiz.OfBizUserDao.search(OfBizUserDao.java:939)
	at com.atlassian.jira.crowd.embedded.ofbiz.DelegatingUserDao.search(DelegatingUserDao.java:124)
	at com.atlassian.jira.crowd.embedded.ofbiz.IndexedUserDao.search(IndexedUserDao.java:450)
	at com.atlassian.jira.crowd.embedded.ofbiz.DelegatingUserDao.search(DelegatingUserDao.java:124)
	at com.atlassian.jira.crowd.embedded.ofbiz.SwitchingUserDao.search(SwitchingUserDao.java:30)
	at com.atlassian.crowd.directory.AbstractInternalDirectory.searchUsers(AbstractInternalDirectory.java:733)
	at com.atlassian.crowd.manager.directory.DirectoryManagerGeneric.searchUsers(DirectoryManagerGeneric.java:376)
	at jdk.internal.reflect.GeneratedMethodAccessor618.invoke(Unknown Source)
	at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@17.0.14/DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(java.base@17.0.14/Method.java:569)
	at com.atlassian.crowd.common.util.ProxyUtil.lambda$runWithContextClassLoader$0(ProxyUtil.java:35)
	at com.atlassian.crowd.common.util.ProxyUtil$$Lambda$6400/0x0000029b84cce530.invoke(Unknown Source)
	at jdk.proxy3.$Proxy626.searchUsers(jdk.proxy3/Unknown Source)
	at com.atlassian.crowd.manager.application.search.DirectoryManagerSearchWrapper.lambda$searchUsers$0(DirectoryManagerSearchWrapper.java:46)
	at com.atlassian.crowd.manager.application.search.DirectoryManagerSearchWrapper$$Lambda$6437/0x0000029b84cde000.execute(Unknown Source)
	at com.atlassian.crowd.manager.application.search.DirectoryManagerSearchWrapper.handle(DirectoryManagerSearchWrapper.java:67)
	at com.atlassian.crowd.manager.application.search.DirectoryManagerSearchWrapper.searchUsers(DirectoryManagerSearchWrapper.java:46)
	at com.atlassian.crowd.manager.application.search.DirectoryManagerSearchWrapper.search(DirectoryManagerSearchWrapper.java:37)
	at com.atlassian.crowd.manager.application.canonicality.SimpleCanonicalityChecker.findEntitiesInternal(SimpleCanonicalityChecker.java:121)
	at com.atlassian.crowd.manager.application.canonicality.SimpleCanonicalityChecker.computeMissing(SimpleCanonicalityChecker.java:97)
	at com.atlassian.crowd.manager.application.canonicality.SimpleCanonicalityChecker.groupByCanonicalId(SimpleCanonicalityChecker.java:76)
	at com.atlassian.crowd.manager.application.search.InMemoryNonAggregatingMembershipSearchStrategy.filterByCanonical(InMemoryNonAggregatingMembershipSearchStrategy.java:51)
	at com.atlassian.crowd.manager.application.search.InMemoryNonAggregatingMembershipSearchStrategy.getQueryTransformer(InMemoryNonAggregatingMembershipSearchStrategy.java:47)
	at com.atlassian.crowd.manager.application.search.AbstractInMemoryMembershipSearchStrategy.createSearcher(AbstractInMemoryMembershipSearchStrategy.java:59)
	at com.atlassian.crowd.manager.application.search.AbstractInMemoryMembershipSearchStrategy.searchGroupRelationships(AbstractInMemoryMembershipSearchStrategy.java:50)
	at com.atlassian.crowd.manager.application.search.AbstractInMemoryMembershipSearchStrategy.searchNestedGroupRelationships(AbstractInMemoryMembershipSearchStrategy.java:46)
	at com.atlassian.crowd.manager.application.ApplicationServiceGeneric.searchNestedGroupRelationships(ApplicationServiceGeneric.java:1642)
	at com.atlassian.crowd.embedded.core.CrowdServiceImpl.searchNestedGroupRelationships(CrowdServiceImpl.java:216)
	at com.atlassian.crowd.embedded.core.CrowdServiceImpl.search(CrowdServiceImpl.java:157)
	at com.atlassian.jira.user.JiraDelegatingCrowdService.search(JiraDelegatingCrowdService.java:70)
	at com.atlassian.jira.user.JiraCrowdService.search(JiraCrowdService.java:52)
	at com.atlassian.jira.security.groups.DefaultGroupManager.getGroupsForUser(DefaultGroupManager.java:409)
	at com.atlassian.jira.security.groups.RequestCachingGroupManager.lambda$new$0(RequestCachingGroupManager.java:43)
	at com.atlassian.jira.security.groups.RequestCachingGroupManager$$Lambda$960/0x0000029b809c8840.load(Unknown Source)
	at com.atlassian.jira.cache.request.RequestCacheImpl.get(RequestCacheImpl.java:48)
	at com.atlassian.jira.security.groups.RequestCachingGroupManager.lambda$new$1(RequestCachingGroupManager.java:46)
	at com.atlassian.jira.security.groups.RequestCachingGroupManager$$Lambda$961/0x0000029b809c8a60.load(Unknown Source)
	at com.atlassian.jira.cache.request.RequestCacheImpl.get(RequestCacheImpl.java:48)
	at com.atlassian.jira.security.groups.RequestCachingGroupManager.lambda$new$2(RequestCachingGroupManager.java:51)
	at com.atlassian.jira.security.groups.RequestCachingGroupManager$$Lambda$962/0x0000029b809c8c80.load(Unknown Source)
	at com.atlassian.jira.cache.request.RequestCacheImpl.get(RequestCacheImpl.java:48)
	at com.atlassian.jira.security.groups.RequestCachingGroupManager.isUserInGroup(RequestCachingGroupManager.java:96)
	at com.atlassian.jira.security.groups.RequestCachingGroupManager.isUserInGroup(RequestCachingGroupManager.java:105)
	at com.atlassian.jira.application.DefaultApplicationRoleManager.lambda$userHasRole$22(DefaultApplicationRoleManager.java:900)
	at com.atlassian.jira.application.DefaultApplicationRoleManager$$Lambda$7571/0x0000029b85123998.test(Unknown Source)
...
	at com.atlassian.jira.application.DefaultApplicationRoleManager.userHasRole(DefaultApplicationRoleManager.java:900)
	at com.atlassian.jira.application.DefaultApplicationRoleManager.lambda$getRolesForUser$11(DefaultApplicationRoleManager.java:450)
	at com.atlassian.jira.application.DefaultApplicationRoleManager$$Lambda$8412/0x0000029b853648c0.test(Unknown Source)
...
	at com.atlassian.jira.application.DefaultApplicationRoleManager.getRolesForUser(DefaultApplicationRoleManager.java:450)
...
	at jdk.proxy18.$Proxy1070.getRolesForUser(jdk.proxy18/Unknown Source)
	at com.atlassian.analytics.client.extractor.JiraPropertyExtractor.getApplicationAccess(JiraPropertyExtractor.java:209)
	at com.atlassian.analytics.client.pipeline.serialize.properties.extractors.general.MetaPropertyExtractor.getApplicationAccess(MetaPropertyExtractor.java:32)
	at com.atlassian.analytics.client.pipeline.serialize.properties.MauAwareExtractorSupplier.oldInstant(MauAwareExtractorSupplier.java:40)
	at com.atlassian.analytics.client.pipeline.serialize.properties.MauAwareExtractorSupplier$$Lambda$6171/0x0000029b8163f9d8.populate(Unknown Source)
	at com.atlassian.analytics.client.pipeline.serialize.properties.ExtractionSupplier.createTimedSupplier(ExtractionSupplier.java:17)
	at com.atlassian.analytics.client.pipeline.serialize.properties.MauAwareExtractorSupplier.toExtractionSupplier(MauAwareExtractorSupplier.java:22)
	at com.atlassian.analytics.client.pipeline.serialize.DefaultEventSerializer.toAnalyticsEvent(DefaultEventSerializer.java:26)
	at com.atlassian.analytics.client.pipeline.DefaultAnalyticsPipeline.processEventWithTiming(DefaultAnalyticsPipeline.java:75)
	at com.atlassian.analytics.client.pipeline.DefaultAnalyticsPipeline.process(DefaultAnalyticsPipeline.java:67)
	at com.atlassian.analytics.client.pipeline.SwitchingAnalyticsPipeline.process(SwitchingAnalyticsPipeline.java:25)
	at com.atlassian.analytics.client.listener.ProductEventListener.processEvent(ProductEventListener.java:48)
	at com.atlassian.analytics.client.listener.JiraEventListener.onEvent(JiraEventListener.java:61)
	at jdk.internal.reflect.GeneratedMethodAccessor495.invoke(Unknown Source)
	at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@17.0.14/DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(java.base@17.0.14/Method.java:569)
	at com.atlassian.event.internal.SingleParameterMethodListenerInvoker.invoke(SingleParameterMethodListenerInvoker.java:42)
	at com.atlassian.diagnostics.internal.platform.monitor.event.EventSystemMonitor.invokeMonitored(EventSystemMonitor.java:105)
	at com.atlassian.diagnostics.internal.platform.monitor.event.MonitoredListenerInvoker.invoke(MonitoredListenerInvoker.java:38)
	at com.atlassian.event.internal.ComparableListenerInvoker.invoke(ComparableListenerInvoker.java:48)
	at com.atlassian.event.internal.AsynchronousAbleEventDispatcher.lambda$static$0(AsynchronousAbleEventDispatcher.java:37)
	at com.atlassian.event.internal.AsynchronousAbleEventDispatcher$$Lambda$1168/0x0000029b80b19c40.run(Unknown Source)
	at com.atlassian.event.internal.AsynchronousAbleEventDispatcher$$Lambda$527/0x0000029b8057c760.execute(Unknown Source)
	at com.atlassian.event.internal.AsynchronousAbleEventDispatcher.dispatch(AsynchronousAbleEventDispatcher.java:85)
	at com.atlassian.diagnostics.internal.platform.monitor.event.MonitoredEventDispatcher.dispatch(MonitoredEventDispatcher.java:36)
	at com.atlassian.event.internal.EventPublisherImpl.publish(EventPublisherImpl.java:102)
	at com.atlassian.plugin.event.impl.DefaultPluginEventManager.broadcast(DefaultPluginEventManager.java:91)
	at com.atlassian.plugin.manager.DefaultPluginManager.broadcastIgnoreError(DefaultPluginManager.java:2069)
...
	at com.atlassian.upm.core.install.AbstractPluginInstallHandler.installArtifacts(AbstractPluginInstallHandler.java:127)
	at com.atlassian.upm.core.install.ObrPluginInstallHandler.installResources(ObrPluginInstallHandler.java:247)
	at com.atlassian.upm.core.install.ObrPluginInstallHandler.installPluginInternal(ObrPluginInstallHandler.java:163)
	at com.atlassian.upm.core.install.AbstractPluginInstallHandler.installPlugin(AbstractPluginInstallHandler.java:54)
	at com.atlassian.upm.core.install.DefaultPluginInstallationService.execute(DefaultPluginInstallationService.java:130)
	at com.atlassian.upm.core.install.DefaultPluginInstallationService.install(DefaultPluginInstallationService.java:98)
	at com.atlassian.upm.install.UpmPluginInstallationService.install(UpmPluginInstallationService.java:114)
	at com.atlassian.upm.core.rest.resources.install.InstallTask.installFromFile(InstallTask.java:140)
	at com.atlassian.upm.core.rest.resources.install.InstallFromUriTask.lambda$executeTask$0(InstallFromUriTask.java:50)
	at com.atlassian.upm.core.rest.resources.install.InstallFromUriTask$$Lambda$10872/0x0000029b85b62428.apply(Unknown Source)
	at com.atlassian.upm.api.util.Either$Right.fold(Either.java:165)
	at com.atlassian.upm.core.rest.resources.install.InstallFromUriTask.executeTask(InstallFromUriTask.java:43)
	at com.atlassian.upm.core.rest.resources.install.InstallTask.run(InstallTask.java:78)
...

Steps to Reproduce

Steps to reproduce JSWSERVER-21588
Steps to reproduce JRASERVER-73830 or JRASERVER-77849

Expected Results

The plugin upgrade will run in a reasonable time, even if it includes stopping the dependent plugins.

Actual Results

The plugin upgrade with take from 30m to hours to complete.

Workaround

Currently there is no known workaround for this behavior. A workaround will be added here when available

Details

Description

Issue Summary

Steps to Reproduce

Expected Results

Actual Results

Workaround

Attachments

Forms

Activity

People

Dates