[CWD-3915] APPARENT DEADLOCK exception

Type: Bug
Resolution: Not a bug
Priority: High
Fix Version/s: None
Affects Version/s: 2.7.2
Component/s: Database
Labels:
None
Environment:

Ubuntu 14.04, OpenJDK 7, MySQL

Bug Fix Policy:
View Atlassian Server bug fix policy

I see lots of exceptions like the one below.
System seems very unstable. And it seems it's affecting the other systems (JIRA, Confluence, Bamboo, Stash).
I turned token database off, and configured in memory.

WARN  05:05:41,147 nTaskTimer ThreadPoolAsynchronousRunner - com.mchange.v2.async.ThreadPoolAsynchronousRunner$DeadlockDetector@554eadfe -- APPARENT DEADLOCK!!! Creating emergency threads for unassigned pending tasks!
WARN  05:05:41,150 nTaskTimer ThreadPoolAsynchronousRunner - com.mchange.v2.async.ThreadPoolAsynchronousRunner$DeadlockDetector@554eadfe -- APPARENT DEADLOCK!!! Complete Status:
        Managed Threads: 3
        Active Threads: 3
        Active Tasks:
                com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask@34dddfbe
                        on thread: C3P0PooledConnectionPoolManager[identityToken->itn5ok911wmgal2jy18l3|4cdfdf9a]-HelperThread-#1
                com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask@528c4e0f
                        on thread: C3P0PooledConnectionPoolManager[identityToken->itn5ok911wmgal2jy18l3|4cdfdf9a]-HelperThread-#2
                com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask@7958e51f
                        on thread: C3P0PooledConnectionPoolManager[identityToken->itn5ok911wmgal2jy18l3|4cdfdf9a]-HelperThread-#0
        Pending Tasks:
                com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask@3b2a9476
                com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask@6878ed52
                com.mchange.v2.resourcepool.BasicResourcePool$1RefurbishCheckinResourceTask@5b24d813
                com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask@4fe5f624
                com.mchange.v2.resourcepool.BasicResourcePool$1RefurbishCheckinResourceTask@282de77
                com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask@5ddb57ea
                com.mchange.v2.resourcepool.BasicResourcePool$1RefurbishCheckinResourceTask@6b68f3c9
                com.mchange.v2.resourcepool.BasicResourcePool$1RefurbishCheckinResourceTask@375d4836
                com.mchange.v2.resourcepool.BasicResourcePool$1RefurbishCheckinResourceTask@4d2fed9f
                com.mchange.v2.resourcepool.BasicResourcePool$1RefurbishCheckinResourceTask@5998950
                com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask@2305e60f
                com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask@5842f4bf
                com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask@13d254de
                com.mchange.v2.resourcepool.BasicResourcePool$1RefurbishCheckinResourceTask@55c1c4fc
                com.mchange.v2.resourcepool.BasicResourcePool$1RefurbishCheckinResourceTask@246ba860
                com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask@17fb494e
                com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask@7289ac2e
Pool thread stack traces:
        Thread[C3P0PooledConnectionPoolManager[identityToken->itn5ok911wmgal2jy18l3|4cdfdf9a]-HelperThread-#0,5,main]
                java.net.SocketInputStream.socketRead0(Native Method)
                java.net.SocketInputStream.read(SocketInputStream.java:152)
                java.net.SocketInputStream.read(SocketInputStream.java:122)
                com.mysql.jdbc.util.ReadAheadInputStream.fill(ReadAheadInputStream.java:112)
                com.mysql.jdbc.util.ReadAheadInputStream.readFromUnderlyingStreamIfNecessary(ReadAheadInputStream.java:159)
                com.mysql.jdbc.util.ReadAheadInputStream.read(ReadAheadInputStream.java:187)
                com.mysql.jdbc.MysqlIO.readFully(MysqlIO.java:3158)
                com.mysql.jdbc.MysqlIO.readPacket(MysqlIO.java:598)
                com.mysql.jdbc.MysqlIO.doHandshake(MysqlIO.java:1086)
                com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2506)
                com.mysql.jdbc.ConnectionImpl.connectWithRetries(ConnectionImpl.java:2344)
                com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2326)
                com.mysql.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:832)
                com.mysql.jdbc.JDBC4Connection.<init>(JDBC4Connection.java:46)
                sun.reflect.GeneratedConstructorAccessor131.newInstance(Unknown Source)
                sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
                java.lang.reflect.Constructor.newInstance(Constructor.java:526)
                com.mysql.jdbc.Util.handleNewInstance(Util.java:409)
                com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:417)
                com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:344)
                com.mchange.v2.c3p0.DriverManagerDataSource.getConnection(DriverManagerDataSource.java:146)
                com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:195)
                com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:184)
                com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.acquireResource(C3P0PooledConnectionPool.java:200)
                com.mchange.v2.resourcepool.BasicResourcePool.doAcquire(BasicResourcePool.java:1086)
                com.mchange.v2.resourcepool.BasicResourcePool.doAcquireAndDecrementPendingAcquiresWithinLockOnSuccess(BasicResourcePool.java:1073)
                com.mchange.v2.resourcepool.BasicResourcePool.access$800(BasicResourcePool.java:44)
                com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask.run(BasicResourcePool.java:1810)
                com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:648)
        Thread[C3P0PooledConnectionPoolManager[identityToken->itn5ok911wmgal2jy18l3|4cdfdf9a]-HelperThread-#1,5,main]
                java.net.SocketInputStream.socketRead0(Native Method)
                java.net.SocketInputStream.read(SocketInputStream.java:152)
                java.net.SocketInputStream.read(SocketInputStream.java:122)
                com.mysql.jdbc.util.ReadAheadInputStream.fill(ReadAheadInputStream.java:112)
                com.mysql.jdbc.util.ReadAheadInputStream.readFromUnderlyingStreamIfNecessary(ReadAheadInputStream.java:159)
                com.mysql.jdbc.util.ReadAheadInputStream.read(ReadAheadInputStream.java:187)
                com.mysql.jdbc.MysqlIO.readFully(MysqlIO.java:3158)
                com.mysql.jdbc.MysqlIO.readPacket(MysqlIO.java:598)
                com.mysql.jdbc.MysqlIO.doHandshake(MysqlIO.java:1086)
                com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2506)
                com.mysql.jdbc.ConnectionImpl.connectWithRetries(ConnectionImpl.java:2344)
                com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2326)
                com.mysql.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:832)
                com.mysql.jdbc.JDBC4Connection.<init>(JDBC4Connection.java:46)
                sun.reflect.GeneratedConstructorAccessor131.newInstance(Unknown Source)
                sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
                java.lang.reflect.Constructor.newInstance(Constructor.java:526)
                com.mysql.jdbc.Util.handleNewInstance(Util.java:409)
                com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:417)
                com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:344)
                com.mchange.v2.c3p0.DriverManagerDataSource.getConnection(DriverManagerDataSource.java:146)
                com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:195)
                com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:184)
                com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.acquireResource(C3P0PooledConnectionPool.java:200)
                com.mchange.v2.resourcepool.BasicResourcePool.doAcquire(BasicResourcePool.java:1086)
                com.mchange.v2.resourcepool.BasicResourcePool.doAcquireAndDecrementPendingAcquiresWithinLockOnSuccess(BasicResourcePool.java:1073)
                com.mchange.v2.resourcepool.BasicResourcePool.access$800(BasicResourcePool.java:44)
                com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask.run(BasicResourcePool.java:1810)
                com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:648)
        Thread[C3P0PooledConnectionPoolManager[identityToken->itn5ok911wmgal2jy18l3|4cdfdf9a]-HelperThread-#2,5,main]
                java.net.SocketInputStream.socketRead0(Native Method)
                java.net.SocketInputStream.read(SocketInputStream.java:152)
                java.net.SocketInputStream.read(SocketInputStream.java:122)
                com.mysql.jdbc.util.ReadAheadInputStream.fill(ReadAheadInputStream.java:112)
                com.mysql.jdbc.util.ReadAheadInputStream.readFromUnderlyingStreamIfNecessary(ReadAheadInputStream.java:159)
                com.mysql.jdbc.util.ReadAheadInputStream.read(ReadAheadInputStream.java:187)
                com.mysql.jdbc.MysqlIO.readFully(MysqlIO.java:3158)
                com.mysql.jdbc.MysqlIO.readPacket(MysqlIO.java:598)
                com.mysql.jdbc.MysqlIO.doHandshake(MysqlIO.java:1086)
                com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2506)
                com.mysql.jdbc.ConnectionImpl.connectWithRetries(ConnectionImpl.java:2344)
                com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2326)
                com.mysql.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:832)
                com.mysql.jdbc.JDBC4Connection.<init>(JDBC4Connection.java:46)
                sun.reflect.GeneratedConstructorAccessor131.newInstance(Unknown Source)
                sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
                java.lang.reflect.Constructor.newInstance(Constructor.java:526)
                com.mysql.jdbc.Util.handleNewInstance(Util.java:409)
                com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:417)
                com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:344)
                com.mchange.v2.c3p0.DriverManagerDataSource.getConnection(DriverManagerDataSource.java:146)
                com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:195)
                com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:184)
                com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.acquireResource(C3P0PooledConnectionPool.java:200)
                com.mchange.v2.resourcepool.BasicResourcePool.doAcquire(BasicResourcePool.java:1086)
                com.mchange.v2.resourcepool.BasicResourcePool.doAcquireAndDecrementPendingAcquiresWithinLockOnSuccess(BasicResourcePool.java:1073)
                com.mchange.v2.resourcepool.BasicResourcePool.access$800(BasicResourcePool.java:44)
                com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask.run(BasicResourcePool.java:1810)
                com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:648)

mentioned in: Page Failed to load

Form Name

No work has yet been logged on this issue.

Details

Description

Attachments

Issue Links

Forms

Activity

People

Dates