Loading...

XML

Word

Printable

Details

Type: Bug
Resolution: Fixed
Priority: High
Fix Version/s: 8.22.1, 8.20.13
Affects Version/s: 6.4.13, 7.0.10, 7.1.0, 7.2.1, 7.4.2, 7.5.0, 7.2.4, 7.13.1, 7.13.5, 7.13.8, 8.5.6, 8.13.4
Component/s: Upgrade
Labels:

Introduced in Version:
6.04
Support reference count:
72
Symptom Severity:
Severity 2 - Major
UIS:
180
Bug Fix Policy:
View Atlassian Server bug fix policy

Description

Summary

Any simultaneous calls to <T> T getComponent(final Class<T> componentType) and to <T> List<T> getComponents(final Class<T> componentType) in DefaultPicoContainer from parallel threads can lead to deadlock.

A more specific version of the problem covering scenario when localhost-startStop-1 and localq-reader threads are deadlocking during DC node startup, see ~~JRASERVER-70816~~.

When DefaultPicoContainer has <T> List<T> getComponents(final Class<T> componentType) invoked it enters into synchronised block over itself and proceeds further and eventually ends up in Locked#getComponentInstance where it tries to acquire ReetrantLock. Unfortunately, this lock is already acquired by another thread which invoked <T> T getComponent(final Class<T> componentType) and this second threads proceeds further and perform back-call to DefaultPicoContainer#getConverters which is "synchronized", so it deadlocks, as first threads owns "synchronized".

It happens during import:

Found one Java-level deadlock:
=============================
"JiraImportTaskExecutionThread-1":
  waiting for ownable synchronizer 0x000000076f409dc8, (a java.util.concurrent.locks.ReentrantLock$NonfairSync),
  which is held by "http-nio-8040-exec-17"
"http-nio-8040-exec-17":
  waiting to lock monitor 0x00007f5d60bfddd8 (object 0x000000076f0f8768, a org.picocontainer.DefaultPicoContainer),
  which is held by "JiraImportTaskExecutionThread-1"

Java stack information for the threads listed above:
===================================================
"JiraImportTaskExecutionThread-1":
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x000000076f409dc8> (a java.util.concurrent.locks.ReentrantLock$NonfairSync)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)
	at java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:209)
	at java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:285)
	at org.picocontainer.behaviors.Locked.getComponentInstance(Locked.java:38)
	at org.picocontainer.DefaultPicoContainer.getInstance(DefaultPicoContainer.java:698)
	at org.picocontainer.DefaultPicoContainer.getComponent(DefaultPicoContainer.java:646)
	at org.picocontainer.DefaultPicoContainer.getComponent(DefaultPicoContainer.java:631)
	at org.picocontainer.parameters.BasicComponentParameter$1.resolveInstance(BasicComponentParameter.java:118)
	at org.picocontainer.parameters.ComponentParameter$1.resolveInstance(ComponentParameter.java:136)
	at org.picocontainer.injectors.SingleMemberInjector.getParameter(SingleMemberInjector.java:78)
	at org.picocontainer.injectors.ConstructorInjector$CtorAndAdapters.getParameterArguments(ConstructorInjector.java:309)
	at org.picocontainer.injectors.ConstructorInjector$1.run(ConstructorInjector.java:335)
	at org.picocontainer.injectors.AbstractInjector$ThreadLocalCyclicDependencyGuard.observe(AbstractInjector.java:270)
	at org.picocontainer.injectors.ConstructorInjector.getComponentInstance(ConstructorInjector.java:364)
	at org.picocontainer.behaviors.AbstractBehavior.getComponentInstance(AbstractBehavior.java:64)
	at org.picocontainer.behaviors.Stored.getComponentInstance(Stored.java:91)
	at org.picocontainer.behaviors.AbstractBehavior.getComponentInstance(AbstractBehavior.java:64)
	at org.picocontainer.behaviors.Locked.getComponentInstance(Locked.java:40)
	at org.picocontainer.DefaultPicoContainer.getLocalInstance(DefaultPicoContainer.java:605)
	at org.picocontainer.DefaultPicoContainer.getComponents(DefaultPicoContainer.java:586)
	- locked <0x000000076f0f8768> (a org.picocontainer.DefaultPicoContainer)
	at com.atlassian.jira.component.CachingMutablePicoContainer.getComponents(CachingMutablePicoContainer.java:191)
	at com.atlassian.jira.ComponentManager.runInitializingComponents(ComponentManager.java:245)
	at com.atlassian.jira.ComponentManager.initialise(ComponentManager.java:138)
	at com.atlassian.jira.ComponentManager.initialise(ComponentManager.java:146)
	at com.atlassian.jira.ManagerFactory.globalRefresh(ManagerFactory.java:50)
	- locked <0x00000006cd3ae048> (a java.lang.Class for com.atlassian.jira.ManagerFactory)
	at com.atlassian.jira.bc.dataimport.DataImportProductionDependencies.globalRefresh(DataImportProductionDependencies.java:97)
	at com.atlassian.jira.bc.dataimport.DefaultDataImportService.globalRefresh(DefaultDataImportService.java:893)
	at com.atlassian.jira.bc.dataimport.DefaultDataImportService.restartJira(DefaultDataImportService.java:850)
	at com.atlassian.jira.bc.dataimport.DefaultDataImportService.performImport(DefaultDataImportService.java:718)
	at com.atlassian.jira.bc.dataimport.DefaultDataImportService.doImport(DefaultDataImportService.java:312)
	at com.atlassian.jira.web.action.setup.DataImportAsyncCommand.call(DataImportAsyncCommand.java:60)
	at com.atlassian.jira.web.action.setup.DataImportAsyncCommand.call(DataImportAsyncCommand.java:28)
	at com.atlassian.jira.task.ImportTaskManagerImpl$TaskCallableDecorator.call(ImportTaskManagerImpl.java:157)
	at com.atlassian.jira.task.ImportTaskManagerImpl$TaskCallableDecorator.call(ImportTaskManagerImpl.java:133)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

"http-nio-8040-exec-17":
	at org.picocontainer.DefaultPicoContainer.getConverters(DefaultPicoContainer.java:1106)
	- waiting to lock <0x000000076f0f8768> (a org.picocontainer.DefaultPicoContainer)
	at org.picocontainer.parameters.BasicComponentParameter.getConverters(BasicComponentParameter.java:129)
	at org.picocontainer.parameters.BasicComponentParameter.access$000(BasicComponentParameter.java:51)
	at org.picocontainer.parameters.BasicComponentParameter$1.resolveInstance(BasicComponentParameter.java:118)
	at org.picocontainer.parameters.ComponentParameter$1.resolveInstance(ComponentParameter.java:136)
	at org.picocontainer.injectors.SingleMemberInjector.getParameter(SingleMemberInjector.java:78)
	at org.picocontainer.injectors.ConstructorInjector$CtorAndAdapters.getParameterArguments(ConstructorInjector.java:309)
	at org.picocontainer.injectors.ConstructorInjector$1.run(ConstructorInjector.java:335)
	at org.picocontainer.injectors.AbstractInjector$ThreadLocalCyclicDependencyGuard.observe(AbstractInjector.java:270)
	at org.picocontainer.injectors.ConstructorInjector.getComponentInstance(ConstructorInjector.java:364)
	at org.picocontainer.behaviors.AbstractBehavior.getComponentInstance(AbstractBehavior.java:64)
	at org.picocontainer.behaviors.Stored.getComponentInstance(Stored.java:91)
	at org.picocontainer.behaviors.AbstractBehavior.getComponentInstance(AbstractBehavior.java:64)
	at org.picocontainer.behaviors.Locked.getComponentInstance(Locked.java:40)
	at org.picocontainer.DefaultPicoContainer.getInstance(DefaultPicoContainer.java:698)
	at org.picocontainer.DefaultPicoContainer.getComponent(DefaultPicoContainer.java:646)
	at org.picocontainer.DefaultPicoContainer.getComponent(DefaultPicoContainer.java:677)
	at com.atlassian.jira.component.CachingMutablePicoContainer.getComponent(CachingMutablePicoContainer.java:137)
	at com.atlassian.jira.ComponentManager.getComponent(ComponentManager.java:484)
	at com.atlassian.jira.component.ComponentAccessorWorker.getComponent(ComponentAccessorWorker.java:25)
	at com.atlassian.jira.component.ComponentAccessor.getComponent(ComponentAccessor.java:160)
	at com.atlassian.jira.component.ComponentAccessor.getPluginAccessor(ComponentAccessor.java:477)
	at com.atlassian.jira.web.servlet.InternalServerErrorHelper.getFooterContent(InternalServerErrorHelper.java:121)
	at com.atlassian.jira.web.servlet.InternalServerErrorHelper.render500ResponsePage(InternalServerErrorHelper.java:45)
	at com.atlassian.jira.web.servlet.InternalServerErrorServlet.doRequest(InternalServerErrorServlet.java:19)
	at com.atlassian.jira.web.servlet.InternalServerErrorServlet.doGet(InternalServerErrorServlet.java:25)
...

Found 1 deadlock.

This causes the XML import to hang indefinitely:

JiraImportTaskExecutionThread-1 INFO scmadm 587x82x1 USER 111.111.111.111 /secure/admin/XmlRestore.jspa [jira.config.database.SystemDatabaseConfigurationLoader] Reading database configuration from /opt/app/scm/app/apfc-app/home/dbconfig.xml

Workaround in this specific scenario is to try restoring the XML backup and then completely closing the browser.

Also it happens on JIRA's startup:

The following error can be found in the logs :

2016-10-02 21:01:49,724 DeadlockDetection:thread-1 ERROR      [c.a.jira.startup.LauncherContextListener] A deadlock has been detected on JIRA startup for the following threads: ["Structure-throttled-clear-caches #2" Id=82 BLOCKED on org.picocontainer.DefaultPicoContainer@7dec9655 owned by "JIRA-Bootstrap" Id=16, "JIRA-Bootstrap" Id=16 WAITING on java.util.concurrent.locks.ReentrantLock$NonfairSync@732b7798 owned by "Structure-throttled-clear-caches #2" Id=82]
2016-10-02 21:01:49,726 DeadlockDetection:thread-1 ERROR      [c.a.jira.startup.LauncherContextListener] "Structure-throttled-clear-caches #2" Id=82 BLOCKED on org.picocontainer.DefaultPicoContainer@7dec9655 owned by "JIRA-Bootstrap" Id=16
    	at org.picocontainer.DefaultPicoContainer.getConverters(DefaultPicoContainer.java:1106)
    	-  blocked on org.picocontainer.DefaultPicoContainer@7dec9655
    	at org.picocontainer.parameters.BasicComponentParameter.getConverters(BasicComponentParameter.java:129)
    	at org.picocontainer.parameters.BasicComponentParameter.access$000(BasicComponentParameter.java:51)
    	at org.picocontainer.parameters.BasicComponentParameter$1.resolveInstance(BasicComponentParameter.java:118)
    	at org.picocontainer.parameters.ComponentParameter$1.resolveInstance(ComponentParameter.java:136)
    	at org.picocontainer.injectors.SingleMemberInjector.getParameter(SingleMemberInjector.java:78)
    	at org.picocontainer.injectors.ConstructorInjector$CtorAndAdapters.getParameterArguments(ConstructorInjector.java:309)
    	at org.picocontainer.injectors.ConstructorInjector$1.run(ConstructorInjector.java:335)
    	...:
    	org.picocontainer.DefaultPicoContainer.getConverters(DefaultPicoContainer.java:1106)
    	org.picocontainer.parameters.BasicComponentParameter.getConverters(BasicComponentParameter.java:129)
    	org.picocontainer.parameters.BasicComponentParameter.access$000(BasicComponentParameter.java:51)
    	org.picocontainer.parameters.BasicComponentParameter$1.resolveInstance(BasicComponentParameter.java:118)
    	org.picocontainer.parameters.ComponentParameter$1.resolveInstance(ComponentParameter.java:136)
    	org.picocontainer.injectors.SingleMemberInjector.getParameter(SingleMemberInjector.java:78)
    	org.picocontainer.injectors.ConstructorInjector$CtorAndAdapters.getParameterArguments(ConstructorInjector.java:309)
    	org.picocontainer.injectors.ConstructorInjector$1.run(ConstructorInjector.java:335)
    	org.picocontainer.injectors.AbstractInjector$ThreadLocalCyclicDependencyGuard.observe(AbstractInjector.java:270)
    	org.picocontainer.injectors.ConstructorInjector.getComponentInstance(ConstructorInjector.java:364)
    	org.picocontainer.behaviors.AbstractBehavior.getComponentInstance(AbstractBehavior.java:64)
    	org.picocontainer.behaviors.Stored.getComponentInstance(Stored.java:91)
    	org.picocontainer.behaviors.AbstractBehavior.getComponentInstance(AbstractBehavior.java:64)
    	org.picocontainer.behaviors.Locked.getComponentInstance(Locked.java:40)
    	org.picocontainer.DefaultPicoContainer.getInstance(DefaultPicoContainer.java:698)
    	org.picocontainer.DefaultPicoContainer.getComponent(DefaultPicoContainer.java:646)
    	org.picocontainer.DefaultPicoContainer.getComponent(DefaultPicoContainer.java:677)
    	com.atlassian.jira.component.CachingMutablePicoContainer.getComponent(CachingMutablePicoContainer.java:137)
    	com.atlassian.jira.ComponentManager.getComponent(ComponentManager.java:513)
    	com.atlassian.jira.component.ComponentAccessorWorker.getComponent(ComponentAccessorWorker.java:30)
    	com.atlassian.jira.component.ComponentAccessorWorker.getComponentSafely(ComponentAccessorWorker.java:22)
    	com.atlassian.jira.component.ComponentAccessor.getComponentSafely(ComponentAccessor.java:140)
    	com.atlassian.jira.instrumentation.DefaultInstrumentationListenerManager.endContext(DefaultInstrumentationListenerManager.java:184)
    	com.atlassian.jira.instrumentation.DefaultInstrumentationListenerManager.endContext(DefaultInstrumentationListenerManager.java:168)
    	com.atlassian.jira.util.thread.JiraThreadLocalUtils.postCall(JiraThreadLocalUtils.java:116)
    	com.almworks.jira.structure.services.JiraThreadEnv.clear(JiraThreadEnv.java:24)
    	com.almworks.jira.structure.util.ThrottledRunnableImpl$DelayedTask.run(ThrottledRunnableImpl.java:100)
    	java.lang.Thread.run(Thread.java:745)
    
2016-10-02 21:01:49,726 DeadlockDetection:thread-1 ERROR      [c.a.jira.startup.LauncherContextListener] "JIRA-Bootstrap" Id=16 WAITING on java.util.concurrent.locks.ReentrantLock$NonfairSync@732b7798 owned by "Structure-throttled-clear-caches #2" Id=82
    	at sun.misc.Unsafe.park(Native Method)
    	-  waiting on java.util.concurrent.locks.ReentrantLock$NonfairSync@732b7798
    	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
    	at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
    	at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)
    	at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)
    	at java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:209)
    	at java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:285)
    	at org.picocontainer.behaviors.Locked.getComponentInstance(Locked.java:38)
    	...:
    	sun.misc.Unsafe.park(Native Method)
    	java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
    	java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
    	java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)
    	java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)
    	java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:209)
    	java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:285)
    	org.picocontainer.behaviors.Locked.getComponentInstance(Locked.java:38)
    	org.picocontainer.DefaultPicoContainer.getLocalInstance(DefaultPicoContainer.java:605)
    	org.picocontainer.DefaultPicoContainer.getComponents(DefaultPicoContainer.java:586)
    	org.picocontainer.DefaultPicoContainer.getComponents(DefaultPicoContainer.java:572)
    	com.atlassian.jira.component.CachingMutablePicoContainer.getComponents(CachingMutablePicoContainer.java:151)
    	com.atlassian.jira.ComponentContainer.initializeEagerComponents(ComponentContainer.java:214)
    	com.atlassian.jira.ComponentManager.eagerlyInstantiate(ComponentManager.java:372)
    	com.atlassian.jira.ComponentManager.quickStart(ComponentManager.java:207)
    	com.atlassian.jira.ComponentManager.start(ComponentManager.java:157)
    	com.atlassian.jira.upgrade.PluginSystemLauncher.start(PluginSystemLauncher.java:43)
    	com.atlassian.jira.startup.DefaultJiraLauncher.lambda$postDbLaunch$2(DefaultJiraLauncher.java:144)
    	com.atlassian.jira.startup.DefaultJiraLauncher$$Lambda$35/1345937687.run(Unknown Source)
    	com.atlassian.jira.config.database.DatabaseConfigurationManagerImpl.doNowOrEnqueue(DatabaseConfigurationManagerImpl.java:298)
    	com.atlassian.jira.config.database.DatabaseConfigurationManagerImpl.doNowOrWhenDatabaseActivated(DatabaseConfigurationManagerImpl.java:194)
    	com.atlassian.jira.startup.DefaultJiraLauncher.postDbLaunch(DefaultJiraLauncher.java:135)
    	com.atlassian.jira.startup.DefaultJiraLauncher.lambda$start$0(DefaultJiraLauncher.java:101)
    	com.atlassian.jira.startup.DefaultJiraLauncher$$Lambda$2/1448589397.run(Unknown Source)
    	com.atlassian.jira.util.devspeed.JiraDevSpeedTimer.run(JiraDevSpeedTimer.java:31)
    	com.atlassian.jira.startup.DefaultJiraLauncher.start(DefaultJiraLauncher.java:99)
    	com.atlassian.jira.startup.LauncherContextListener.initSlowStuff(LauncherContextListener.java:149)
    	com.atlassian.jira.startup.LauncherContextListener$$Lambda$1/1736489687.run(Unknown Source)
    	java.lang.Thread.run(Thread.java:745)

The problem can also be caused by User Management requests during startup

In one instance, we noticed a rest call from a downstream application (normal user requests are routed to the "Jira is Starting" page in modern versions of Jira. This was spotted very early in the application logs, before the instance had even started up.

2018-09-26 23:37:15,444 https-jsse-nio-443-exec-19 ERROR anonymous 1417x1x1 - 111.111.111.111 /rest/usermanagement/1/authentication [c.atlassian.instrumentation.Instrument] Unable to snapshot thread local operations (implementation of OpTimerFactory is not a ThreadLocalOpTimerFactory): null

This lead to a NullPointerException in that (and other HTTP threads). Eventually, the thread failed being unable to resolve the loginManager component with an java.lang.IllegalStateException: Unable to resolve component: interface com.atlassian.jira.security.login.LoginManager error.

Later in the logs, the deadlock detection message appeared for another thread:

2018-09-26 23:37:33,488 DeadlockDetection:thread-1 ERROR      [c.a.jira.startup.LauncherContextListener] A deadlock has been detected on JIRA startup for the following threads: ["https-jsse-nio-443-exec-36" Id=64 WAITING on java.util.concurrent.locks.ReentrantLock$NonfairSync@60ae2f1e owned by "https-jsse-nio-443-exec-2" Id=29, "https-jsse-nio-443-exec-2" Id=29 BLOCKED on org.picocontainer.DefaultPicoContainer@6e69cd7b owned by "JIRA-Bootstrap" Id=17, "JIRA-Bootstrap" Id=17 WAITING on java.util.concurrent.locks.ReentrantLock$NonfairSync@6a56446e owned by "https-jsse-nio-443-exec-2" Id=29]

Thread 36 was waiting on 2, which was blocked by JIRA-Bootstrap. Checking on Thread 2's messages in the log, we can see that it was also unable to resolve the LoginManagerComponent:

2018-09-26 23:37:26,703 https-jsse-nio-443-exec-2 ERROR      [o.a.c.c.C.[.[localhost].[/].[default]] Servlet.service() for servlet [default] in context with path [] threw exception
java.lang.IllegalStateException: Unable to resolve component: interface com.atlassian.jira.security.login.LoginManager
	at com.atlassian.jira.component.ComponentAccessorWorker.getComponentOfType(ComponentAccessorWorker.java:41)
	at com.atlassian.jira.component.ComponentAccessor.getComponentOfType(ComponentAccessor.java:255)
	at com.atlassian.jira.security.login.JiraElevatedSecurityGuard.getLoginManager(JiraElevatedSecurityGuard.java:39)
	at com.atlassian.jira.security.login.JiraElevatedSecurityGuard.performElevatedSecurityCheck(JiraElevatedSecurityGuard.java:22)
	at com.atlassian.seraph.filter.PasswordBasedLoginFilter.runAuthentication(PasswordBasedLoginFilter.java:117)
	... 19 filtered
	at com.atlassian.jira.servermetrics.CorrelationIdPopulatorFilter.doFilter(CorrelationIdPopulatorFilter.java:30)
	... 23 filtered
	at com.atlassian.jira.servermetrics.MetricsCollectorFilter.doFilter(MetricsCollectorFilter.java:25)
	... 24 filtered
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.util.ConcurrentModificationException
	at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:901)
	at java.util.ArrayList$Itr.next(ArrayList.java:851)
	at java.util.Collections$UnmodifiableCollection$1.next(Collections.java:1042)
	at org.picocontainer.DefaultPicoContainer.getComponentAdapters(DefaultPicoContainer.java:422)
	at org.picocontainer.DefaultPicoContainer.getComponentAdapters(DefaultPicoContainer.java:413)
	at org.picocontainer.DefaultPicoContainer.getComponentAdapter(DefaultPicoContainer.java:369)
	at org.picocontainer.DefaultPicoContainer.getComponent(DefaultPicoContainer.java:645)
	at org.picocontainer.DefaultPicoContainer.getComponent(DefaultPicoContainer.java:677)
	at com.atlassian.jira.component.CachingMutablePicoContainer.getComponent(CachingMutablePicoContainer.java:137)
	at com.atlassian.jira.ComponentManager.getComponent(ComponentManager.java:546)
	at com.atlassian.jira.ComponentManager.getComponentInstanceOfType(ComponentManager.java:498)
	at com.atlassian.jira.component.ComponentAccessorWorker.getComponentOfType(ComponentAccessorWorker.java:39)
	... 76 more

Blocking access to the instance from external consumers allowed the instance to restart successfully.

Diagnosis Steps

We see the following line in DefaultInstrumentationListenerManager.endContext(Optional<Long>):

            Optional<InstrumentationListenerManager> manager = ComponentAccessor.getComponentSafely(InstrumentationListenerManager.class);

However, if one looks at DefaultInstrumentationListenerManager.startContext(), there is a guarding if() around retrieving this same component:

//we do not want this to run if JIRA is not fully started as it may lead to deadlock in Pico
        if (ComponentManager.getInstance().getState().isStarted()) {
            ComponentAccessor.getComponentSafely(InstrumentationListenerManager.class) //... using the component
}

It looks like endContext() lacks the same precaution, which causes the deadlock.

Workaround

The exact workaround will depend on your specific problem. This can be caused by a third party plugin, restoring getting HTTP access during the backup restore; or remote user management.

Try starting Jira with an empty installed-plugins folder - this helps to eliminate third party plugins
If you're importing, close the browser after kicking off the restore process; this might help
If you're not importing, and are seeing requests to Jira's User Management API, try using a reverse proxy or firewall to block access to Jira while it starts up (then re-enable it later.

Attachments

- Sort By Name
- Sort By Date
- Ascending
- Descending
- Thumbnails
- List

JIRA_support_2016-10-14-17-31-23.zip
3.72 MB
14/Oct/2016 3:55 PM

Issue Links

was cloned as

JRASERVER-70816 JIRA deadlocks in PicoСontainer while start-up and replaying events by localq-reader

Closed

is duplicated by: JSEV-2569 Loading...

mentioned in: Page Loading...; Page Loading...; Page Loading...; Page Loading...; Page Loading...; Page Loading...; Page Loading...; Page Loading...; Page Loading...; Page Loading...; Page Loading...; Page Loading...; Page Loading...; Page Loading...

relates to: PS-52577 Loading...; HL-2388 Loading...; JC-518 Loading...; JSEV-2125 Loading...; JSEV-564 Loading...; JSEV-698 Loading...

(11 mentioned in, 6 relates to)

Activity

People

Assignee:: Mikolaj Grzaslewicz

Reporter:: Adrian Stephen

Votes:: 44 Vote for this issue

Watchers:: 58 Start watching this issue

Dates

Created:: 12/Oct/2016 1:57 PM

Updated:: 02/Nov/2023 7:00 AM

Resolved:: 07/Apr/2022 12:03 PM