-
Bug
-
Resolution: Not a bug
-
High
-
None
-
2.7.2
-
None
-
Ubuntu 14.04, OpenJDK 7, MySQL
I see lots of exceptions like the one below.
System seems very unstable. And it seems it's affecting the other systems (JIRA, Confluence, Bamboo, Stash).
I turned token database off, and configured in memory.
WARN 05:05:41,147 nTaskTimer ThreadPoolAsynchronousRunner - com.mchange.v2.async.ThreadPoolAsynchronousRunner$DeadlockDetector@554eadfe -- APPARENT DEADLOCK!!! Creating emergency threads for unassigned pending tasks! WARN 05:05:41,150 nTaskTimer ThreadPoolAsynchronousRunner - com.mchange.v2.async.ThreadPoolAsynchronousRunner$DeadlockDetector@554eadfe -- APPARENT DEADLOCK!!! Complete Status: Managed Threads: 3 Active Threads: 3 Active Tasks: com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask@34dddfbe on thread: C3P0PooledConnectionPoolManager[identityToken->itn5ok911wmgal2jy18l3|4cdfdf9a]-HelperThread-#1 com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask@528c4e0f on thread: C3P0PooledConnectionPoolManager[identityToken->itn5ok911wmgal2jy18l3|4cdfdf9a]-HelperThread-#2 com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask@7958e51f on thread: C3P0PooledConnectionPoolManager[identityToken->itn5ok911wmgal2jy18l3|4cdfdf9a]-HelperThread-#0 Pending Tasks: com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask@3b2a9476 com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask@6878ed52 com.mchange.v2.resourcepool.BasicResourcePool$1RefurbishCheckinResourceTask@5b24d813 com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask@4fe5f624 com.mchange.v2.resourcepool.BasicResourcePool$1RefurbishCheckinResourceTask@282de77 com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask@5ddb57ea com.mchange.v2.resourcepool.BasicResourcePool$1RefurbishCheckinResourceTask@6b68f3c9 com.mchange.v2.resourcepool.BasicResourcePool$1RefurbishCheckinResourceTask@375d4836 com.mchange.v2.resourcepool.BasicResourcePool$1RefurbishCheckinResourceTask@4d2fed9f com.mchange.v2.resourcepool.BasicResourcePool$1RefurbishCheckinResourceTask@5998950 com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask@2305e60f com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask@5842f4bf com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask@13d254de com.mchange.v2.resourcepool.BasicResourcePool$1RefurbishCheckinResourceTask@55c1c4fc com.mchange.v2.resourcepool.BasicResourcePool$1RefurbishCheckinResourceTask@246ba860 com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask@17fb494e com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask@7289ac2e Pool thread stack traces: Thread[C3P0PooledConnectionPoolManager[identityToken->itn5ok911wmgal2jy18l3|4cdfdf9a]-HelperThread-#0,5,main] java.net.SocketInputStream.socketRead0(Native Method) java.net.SocketInputStream.read(SocketInputStream.java:152) java.net.SocketInputStream.read(SocketInputStream.java:122) com.mysql.jdbc.util.ReadAheadInputStream.fill(ReadAheadInputStream.java:112) com.mysql.jdbc.util.ReadAheadInputStream.readFromUnderlyingStreamIfNecessary(ReadAheadInputStream.java:159) com.mysql.jdbc.util.ReadAheadInputStream.read(ReadAheadInputStream.java:187) com.mysql.jdbc.MysqlIO.readFully(MysqlIO.java:3158) com.mysql.jdbc.MysqlIO.readPacket(MysqlIO.java:598) com.mysql.jdbc.MysqlIO.doHandshake(MysqlIO.java:1086) com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2506) com.mysql.jdbc.ConnectionImpl.connectWithRetries(ConnectionImpl.java:2344) com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2326) com.mysql.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:832) com.mysql.jdbc.JDBC4Connection.<init>(JDBC4Connection.java:46) sun.reflect.GeneratedConstructorAccessor131.newInstance(Unknown Source) sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) java.lang.reflect.Constructor.newInstance(Constructor.java:526) com.mysql.jdbc.Util.handleNewInstance(Util.java:409) com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:417) com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:344) com.mchange.v2.c3p0.DriverManagerDataSource.getConnection(DriverManagerDataSource.java:146) com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:195) com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:184) com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.acquireResource(C3P0PooledConnectionPool.java:200) com.mchange.v2.resourcepool.BasicResourcePool.doAcquire(BasicResourcePool.java:1086) com.mchange.v2.resourcepool.BasicResourcePool.doAcquireAndDecrementPendingAcquiresWithinLockOnSuccess(BasicResourcePool.java:1073) com.mchange.v2.resourcepool.BasicResourcePool.access$800(BasicResourcePool.java:44) com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask.run(BasicResourcePool.java:1810) com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:648) Thread[C3P0PooledConnectionPoolManager[identityToken->itn5ok911wmgal2jy18l3|4cdfdf9a]-HelperThread-#1,5,main] java.net.SocketInputStream.socketRead0(Native Method) java.net.SocketInputStream.read(SocketInputStream.java:152) java.net.SocketInputStream.read(SocketInputStream.java:122) com.mysql.jdbc.util.ReadAheadInputStream.fill(ReadAheadInputStream.java:112) com.mysql.jdbc.util.ReadAheadInputStream.readFromUnderlyingStreamIfNecessary(ReadAheadInputStream.java:159) com.mysql.jdbc.util.ReadAheadInputStream.read(ReadAheadInputStream.java:187) com.mysql.jdbc.MysqlIO.readFully(MysqlIO.java:3158) com.mysql.jdbc.MysqlIO.readPacket(MysqlIO.java:598) com.mysql.jdbc.MysqlIO.doHandshake(MysqlIO.java:1086) com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2506) com.mysql.jdbc.ConnectionImpl.connectWithRetries(ConnectionImpl.java:2344) com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2326) com.mysql.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:832) com.mysql.jdbc.JDBC4Connection.<init>(JDBC4Connection.java:46) sun.reflect.GeneratedConstructorAccessor131.newInstance(Unknown Source) sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) java.lang.reflect.Constructor.newInstance(Constructor.java:526) com.mysql.jdbc.Util.handleNewInstance(Util.java:409) com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:417) com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:344) com.mchange.v2.c3p0.DriverManagerDataSource.getConnection(DriverManagerDataSource.java:146) com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:195) com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:184) com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.acquireResource(C3P0PooledConnectionPool.java:200) com.mchange.v2.resourcepool.BasicResourcePool.doAcquire(BasicResourcePool.java:1086) com.mchange.v2.resourcepool.BasicResourcePool.doAcquireAndDecrementPendingAcquiresWithinLockOnSuccess(BasicResourcePool.java:1073) com.mchange.v2.resourcepool.BasicResourcePool.access$800(BasicResourcePool.java:44) com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask.run(BasicResourcePool.java:1810) com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:648) Thread[C3P0PooledConnectionPoolManager[identityToken->itn5ok911wmgal2jy18l3|4cdfdf9a]-HelperThread-#2,5,main] java.net.SocketInputStream.socketRead0(Native Method) java.net.SocketInputStream.read(SocketInputStream.java:152) java.net.SocketInputStream.read(SocketInputStream.java:122) com.mysql.jdbc.util.ReadAheadInputStream.fill(ReadAheadInputStream.java:112) com.mysql.jdbc.util.ReadAheadInputStream.readFromUnderlyingStreamIfNecessary(ReadAheadInputStream.java:159) com.mysql.jdbc.util.ReadAheadInputStream.read(ReadAheadInputStream.java:187) com.mysql.jdbc.MysqlIO.readFully(MysqlIO.java:3158) com.mysql.jdbc.MysqlIO.readPacket(MysqlIO.java:598) com.mysql.jdbc.MysqlIO.doHandshake(MysqlIO.java:1086) com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2506) com.mysql.jdbc.ConnectionImpl.connectWithRetries(ConnectionImpl.java:2344) com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2326) com.mysql.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:832) com.mysql.jdbc.JDBC4Connection.<init>(JDBC4Connection.java:46) sun.reflect.GeneratedConstructorAccessor131.newInstance(Unknown Source) sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) java.lang.reflect.Constructor.newInstance(Constructor.java:526) com.mysql.jdbc.Util.handleNewInstance(Util.java:409) com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:417) com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:344) com.mchange.v2.c3p0.DriverManagerDataSource.getConnection(DriverManagerDataSource.java:146) com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:195) com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:184) com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.acquireResource(C3P0PooledConnectionPool.java:200) com.mchange.v2.resourcepool.BasicResourcePool.doAcquire(BasicResourcePool.java:1086) com.mchange.v2.resourcepool.BasicResourcePool.doAcquireAndDecrementPendingAcquiresWithinLockOnSuccess(BasicResourcePool.java:1073) com.mchange.v2.resourcepool.BasicResourcePool.access$800(BasicResourcePool.java:44) com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask.run(BasicResourcePool.java:1810) com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:648)
- mentioned in
-
Page Failed to load
[CWD-3915] APPARENT DEADLOCK exception
Workflow | Original: Simplified Crowd Development Workflow v2 - restricted [ 1509743 ] | New: JAC Bug Workflow v3 [ 3364036 ] |
Status | Original: Resolved [ 5 ] | New: Closed [ 6 ] |
Workflow | Original: Simplified Crowd Development Workflow v2 [ 1392045 ] | New: Simplified Crowd Development Workflow v2 - restricted [ 1509743 ] |
Workflow | Original: Crowd Development Workflow v2 [ 686667 ] | New: Simplified Crowd Development Workflow v2 [ 1392045 ] |
Remote Link | Original: This issue links to "Page (Extranet)" [ 65533 ] | New: This issue links to "Page (Extranet)" [ 65533 ] |
Remote Link | Original: This issue links to "Page (Extranet)" [ 65533 ] | New: This issue links to "Page (Extranet)" [ 65533 ] |
Remote Link | Original: This issue links to "Page (Extranet)" [ 65533 ] | New: This issue links to "Page (Extranet)" [ 65533 ] |
Resolution | New: Not a bug [ 12 ] | |
Status | Original: Open [ 1 ] | New: Resolved [ 5 ] |
Remote Link | New: This issue links to "Page (Extranet)" [ 65533 ] |
Description |
Original:
I see lots of exceptions like the one below.
System seems very unstable. And it seems it's affecting the other systems (JIRA, Confluence, Bamboo, Stash). I turned token database off, and configured in memory. WARN 05:05:41,147 nTaskTimer ThreadPoolAsynchronousRunner - com.mchange.v2.async.ThreadPoolAsynchronousRunner$DeadlockDetector@554eadfe -- APPARENT DEADLOCK!!! Creating emergency threads for unassigned pending tasks! WARN 05:05:41,150 nTaskTimer ThreadPoolAsynchronousRunner - com.mchange.v2.async.ThreadPoolAsynchronousRunner$DeadlockDetector@554eadfe -- APPARENT DEADLOCK!!! Complete Status: Managed Threads: 3 Active Threads: 3 Active Tasks: com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask@34dddfbe on thread: C3P0PooledConnectionPoolManager[identityToken->itn5ok911wmgal2jy18l3|4cdfdf9a]-HelperThread-#1 com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask@528c4e0f on thread: C3P0PooledConnectionPoolManager[identityToken->itn5ok911wmgal2jy18l3|4cdfdf9a]-HelperThread-#2 com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask@7958e51f on thread: C3P0PooledConnectionPoolManager[identityToken->itn5ok911wmgal2jy18l3|4cdfdf9a]-HelperThread-#0 Pending Tasks: com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask@3b2a9476 com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask@6878ed52 com.mchange.v2.resourcepool.BasicResourcePool$1RefurbishCheckinResourceTask@5b24d813 com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask@4fe5f624 com.mchange.v2.resourcepool.BasicResourcePool$1RefurbishCheckinResourceTask@282de77 com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask@5ddb57ea com.mchange.v2.resourcepool.BasicResourcePool$1RefurbishCheckinResourceTask@6b68f3c9 com.mchange.v2.resourcepool.BasicResourcePool$1RefurbishCheckinResourceTask@375d4836 com.mchange.v2.resourcepool.BasicResourcePool$1RefurbishCheckinResourceTask@4d2fed9f com.mchange.v2.resourcepool.BasicResourcePool$1RefurbishCheckinResourceTask@5998950 com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask@2305e60f com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask@5842f4bf com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask@13d254de com.mchange.v2.resourcepool.BasicResourcePool$1RefurbishCheckinResourceTask@55c1c4fc com.mchange.v2.resourcepool.BasicResourcePool$1RefurbishCheckinResourceTask@246ba860 com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask@17fb494e com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask@7289ac2e Pool thread stack traces: Thread[C3P0PooledConnectionPoolManager[identityToken->itn5ok911wmgal2jy18l3|4cdfdf9a]-HelperThread-#0,5,main] java.net.SocketInputStream.socketRead0(Native Method) java.net.SocketInputStream.read(SocketInputStream.java:152) java.net.SocketInputStream.read(SocketInputStream.java:122) com.mysql.jdbc.util.ReadAheadInputStream.fill(ReadAheadInputStream.java:112) com.mysql.jdbc.util.ReadAheadInputStream.readFromUnderlyingStreamIfNecessary(ReadAheadInputStream.java:159) com.mysql.jdbc.util.ReadAheadInputStream.read(ReadAheadInputStream.java:187) com.mysql.jdbc.MysqlIO.readFully(MysqlIO.java:3158) com.mysql.jdbc.MysqlIO.readPacket(MysqlIO.java:598) com.mysql.jdbc.MysqlIO.doHandshake(MysqlIO.java:1086) com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2506) com.mysql.jdbc.ConnectionImpl.connectWithRetries(ConnectionImpl.java:2344) com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2326) com.mysql.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:832) com.mysql.jdbc.JDBC4Connection.<init>(JDBC4Connection.java:46) sun.reflect.GeneratedConstructorAccessor131.newInstance(Unknown Source) sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) java.lang.reflect.Constructor.newInstance(Constructor.java:526) com.mysql.jdbc.Util.handleNewInstance(Util.java:409) com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:417) com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:344) com.mchange.v2.c3p0.DriverManagerDataSource.getConnection(DriverManagerDataSource.java:146) com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:195) com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:184) com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.acquireResource(C3P0PooledConnectionPool.java:200) com.mchange.v2.resourcepool.BasicResourcePool.doAcquire(BasicResourcePool.java:1086) com.mchange.v2.resourcepool.BasicResourcePool.doAcquireAndDecrementPendingAcquiresWithinLockOnSuccess(BasicResourcePool.java:1073) com.mchange.v2.resourcepool.BasicResourcePool.access$800(BasicResourcePool.java:44) com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask.run(BasicResourcePool.java:1810) com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:648) Thread[C3P0PooledConnectionPoolManager[identityToken->itn5ok911wmgal2jy18l3|4cdfdf9a]-HelperThread-#1,5,main] java.net.SocketInputStream.socketRead0(Native Method) java.net.SocketInputStream.read(SocketInputStream.java:152) java.net.SocketInputStream.read(SocketInputStream.java:122) com.mysql.jdbc.util.ReadAheadInputStream.fill(ReadAheadInputStream.java:112) com.mysql.jdbc.util.ReadAheadInputStream.readFromUnderlyingStreamIfNecessary(ReadAheadInputStream.java:159) com.mysql.jdbc.util.ReadAheadInputStream.read(ReadAheadInputStream.java:187) com.mysql.jdbc.MysqlIO.readFully(MysqlIO.java:3158) com.mysql.jdbc.MysqlIO.readPacket(MysqlIO.java:598) com.mysql.jdbc.MysqlIO.doHandshake(MysqlIO.java:1086) com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2506) com.mysql.jdbc.ConnectionImpl.connectWithRetries(ConnectionImpl.java:2344) com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2326) com.mysql.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:832) com.mysql.jdbc.JDBC4Connection.<init>(JDBC4Connection.java:46) sun.reflect.GeneratedConstructorAccessor131.newInstance(Unknown Source) sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) java.lang.reflect.Constructor.newInstance(Constructor.java:526) com.mysql.jdbc.Util.handleNewInstance(Util.java:409) com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:417) com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:344) com.mchange.v2.c3p0.DriverManagerDataSource.getConnection(DriverManagerDataSource.java:146) com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:195) com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:184) com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.acquireResource(C3P0PooledConnectionPool.java:200) com.mchange.v2.resourcepool.BasicResourcePool.doAcquire(BasicResourcePool.java:1086) com.mchange.v2.resourcepool.BasicResourcePool.doAcquireAndDecrementPendingAcquiresWithinLockOnSuccess(BasicResourcePool.java:1073) com.mchange.v2.resourcepool.BasicResourcePool.access$800(BasicResourcePool.java:44) com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask.run(BasicResourcePool.java:1810) com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:648) Thread[C3P0PooledConnectionPoolManager[identityToken->itn5ok911wmgal2jy18l3|4cdfdf9a]-HelperThread-#2,5,main] java.net.SocketInputStream.socketRead0(Native Method) java.net.SocketInputStream.read(SocketInputStream.java:152) java.net.SocketInputStream.read(SocketInputStream.java:122) com.mysql.jdbc.util.ReadAheadInputStream.fill(ReadAheadInputStream.java:112) com.mysql.jdbc.util.ReadAheadInputStream.readFromUnderlyingStreamIfNecessary(ReadAheadInputStream.java:159) com.mysql.jdbc.util.ReadAheadInputStream.read(ReadAheadInputStream.java:187) com.mysql.jdbc.MysqlIO.readFully(MysqlIO.java:3158) com.mysql.jdbc.MysqlIO.readPacket(MysqlIO.java:598) com.mysql.jdbc.MysqlIO.doHandshake(MysqlIO.java:1086) com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2506) com.mysql.jdbc.ConnectionImpl.connectWithRetries(ConnectionImpl.java:2344) com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2326) com.mysql.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:832) com.mysql.jdbc.JDBC4Connection.<init>(JDBC4Connection.java:46) sun.reflect.GeneratedConstructorAccessor131.newInstance(Unknown Source) sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) java.lang.reflect.Constructor.newInstance(Constructor.java:526) com.mysql.jdbc.Util.handleNewInstance(Util.java:409) com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:417) com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:344) com.mchange.v2.c3p0.DriverManagerDataSource.getConnection(DriverManagerDataSource.java:146) com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:195) com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:184) com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.acquireResource(C3P0PooledConnectionPool.java:200) com.mchange.v2.resourcepool.BasicResourcePool.doAcquire(BasicResourcePool.java:1086) com.mchange.v2.resourcepool.BasicResourcePool.doAcquireAndDecrementPendingAcquiresWithinLockOnSuccess(BasicResourcePool.java:1073) com.mchange.v2.resourcepool.BasicResourcePool.access$800(BasicResourcePool.java:44) com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask.run(BasicResourcePool.java:1810) com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:648) |
New:
I see lots of exceptions like the one below.
System seems very unstable. And it seems it's affecting the other systems (JIRA, Confluence, Bamboo, Stash). I turned token database off, and configured in memory. {noformat} WARN 05:05:41,147 nTaskTimer ThreadPoolAsynchronousRunner - com.mchange.v2.async.ThreadPoolAsynchronousRunner$DeadlockDetector@554eadfe -- APPARENT DEADLOCK!!! Creating emergency threads for unassigned pending tasks! WARN 05:05:41,150 nTaskTimer ThreadPoolAsynchronousRunner - com.mchange.v2.async.ThreadPoolAsynchronousRunner$DeadlockDetector@554eadfe -- APPARENT DEADLOCK!!! Complete Status: Managed Threads: 3 Active Threads: 3 Active Tasks: com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask@34dddfbe on thread: C3P0PooledConnectionPoolManager[identityToken->itn5ok911wmgal2jy18l3|4cdfdf9a]-HelperThread-#1 com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask@528c4e0f on thread: C3P0PooledConnectionPoolManager[identityToken->itn5ok911wmgal2jy18l3|4cdfdf9a]-HelperThread-#2 com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask@7958e51f on thread: C3P0PooledConnectionPoolManager[identityToken->itn5ok911wmgal2jy18l3|4cdfdf9a]-HelperThread-#0 Pending Tasks: com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask@3b2a9476 com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask@6878ed52 com.mchange.v2.resourcepool.BasicResourcePool$1RefurbishCheckinResourceTask@5b24d813 com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask@4fe5f624 com.mchange.v2.resourcepool.BasicResourcePool$1RefurbishCheckinResourceTask@282de77 com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask@5ddb57ea com.mchange.v2.resourcepool.BasicResourcePool$1RefurbishCheckinResourceTask@6b68f3c9 com.mchange.v2.resourcepool.BasicResourcePool$1RefurbishCheckinResourceTask@375d4836 com.mchange.v2.resourcepool.BasicResourcePool$1RefurbishCheckinResourceTask@4d2fed9f com.mchange.v2.resourcepool.BasicResourcePool$1RefurbishCheckinResourceTask@5998950 com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask@2305e60f com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask@5842f4bf com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask@13d254de com.mchange.v2.resourcepool.BasicResourcePool$1RefurbishCheckinResourceTask@55c1c4fc com.mchange.v2.resourcepool.BasicResourcePool$1RefurbishCheckinResourceTask@246ba860 com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask@17fb494e com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask@7289ac2e Pool thread stack traces: Thread[C3P0PooledConnectionPoolManager[identityToken->itn5ok911wmgal2jy18l3|4cdfdf9a]-HelperThread-#0,5,main] java.net.SocketInputStream.socketRead0(Native Method) java.net.SocketInputStream.read(SocketInputStream.java:152) java.net.SocketInputStream.read(SocketInputStream.java:122) com.mysql.jdbc.util.ReadAheadInputStream.fill(ReadAheadInputStream.java:112) com.mysql.jdbc.util.ReadAheadInputStream.readFromUnderlyingStreamIfNecessary(ReadAheadInputStream.java:159) com.mysql.jdbc.util.ReadAheadInputStream.read(ReadAheadInputStream.java:187) com.mysql.jdbc.MysqlIO.readFully(MysqlIO.java:3158) com.mysql.jdbc.MysqlIO.readPacket(MysqlIO.java:598) com.mysql.jdbc.MysqlIO.doHandshake(MysqlIO.java:1086) com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2506) com.mysql.jdbc.ConnectionImpl.connectWithRetries(ConnectionImpl.java:2344) com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2326) com.mysql.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:832) com.mysql.jdbc.JDBC4Connection.<init>(JDBC4Connection.java:46) sun.reflect.GeneratedConstructorAccessor131.newInstance(Unknown Source) sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) java.lang.reflect.Constructor.newInstance(Constructor.java:526) com.mysql.jdbc.Util.handleNewInstance(Util.java:409) com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:417) com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:344) com.mchange.v2.c3p0.DriverManagerDataSource.getConnection(DriverManagerDataSource.java:146) com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:195) com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:184) com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.acquireResource(C3P0PooledConnectionPool.java:200) com.mchange.v2.resourcepool.BasicResourcePool.doAcquire(BasicResourcePool.java:1086) com.mchange.v2.resourcepool.BasicResourcePool.doAcquireAndDecrementPendingAcquiresWithinLockOnSuccess(BasicResourcePool.java:1073) com.mchange.v2.resourcepool.BasicResourcePool.access$800(BasicResourcePool.java:44) com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask.run(BasicResourcePool.java:1810) com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:648) Thread[C3P0PooledConnectionPoolManager[identityToken->itn5ok911wmgal2jy18l3|4cdfdf9a]-HelperThread-#1,5,main] java.net.SocketInputStream.socketRead0(Native Method) java.net.SocketInputStream.read(SocketInputStream.java:152) java.net.SocketInputStream.read(SocketInputStream.java:122) com.mysql.jdbc.util.ReadAheadInputStream.fill(ReadAheadInputStream.java:112) com.mysql.jdbc.util.ReadAheadInputStream.readFromUnderlyingStreamIfNecessary(ReadAheadInputStream.java:159) com.mysql.jdbc.util.ReadAheadInputStream.read(ReadAheadInputStream.java:187) com.mysql.jdbc.MysqlIO.readFully(MysqlIO.java:3158) com.mysql.jdbc.MysqlIO.readPacket(MysqlIO.java:598) com.mysql.jdbc.MysqlIO.doHandshake(MysqlIO.java:1086) com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2506) com.mysql.jdbc.ConnectionImpl.connectWithRetries(ConnectionImpl.java:2344) com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2326) com.mysql.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:832) com.mysql.jdbc.JDBC4Connection.<init>(JDBC4Connection.java:46) sun.reflect.GeneratedConstructorAccessor131.newInstance(Unknown Source) sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) java.lang.reflect.Constructor.newInstance(Constructor.java:526) com.mysql.jdbc.Util.handleNewInstance(Util.java:409) com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:417) com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:344) com.mchange.v2.c3p0.DriverManagerDataSource.getConnection(DriverManagerDataSource.java:146) com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:195) com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:184) com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.acquireResource(C3P0PooledConnectionPool.java:200) com.mchange.v2.resourcepool.BasicResourcePool.doAcquire(BasicResourcePool.java:1086) com.mchange.v2.resourcepool.BasicResourcePool.doAcquireAndDecrementPendingAcquiresWithinLockOnSuccess(BasicResourcePool.java:1073) com.mchange.v2.resourcepool.BasicResourcePool.access$800(BasicResourcePool.java:44) com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask.run(BasicResourcePool.java:1810) com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:648) Thread[C3P0PooledConnectionPoolManager[identityToken->itn5ok911wmgal2jy18l3|4cdfdf9a]-HelperThread-#2,5,main] java.net.SocketInputStream.socketRead0(Native Method) java.net.SocketInputStream.read(SocketInputStream.java:152) java.net.SocketInputStream.read(SocketInputStream.java:122) com.mysql.jdbc.util.ReadAheadInputStream.fill(ReadAheadInputStream.java:112) com.mysql.jdbc.util.ReadAheadInputStream.readFromUnderlyingStreamIfNecessary(ReadAheadInputStream.java:159) com.mysql.jdbc.util.ReadAheadInputStream.read(ReadAheadInputStream.java:187) com.mysql.jdbc.MysqlIO.readFully(MysqlIO.java:3158) com.mysql.jdbc.MysqlIO.readPacket(MysqlIO.java:598) com.mysql.jdbc.MysqlIO.doHandshake(MysqlIO.java:1086) com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2506) com.mysql.jdbc.ConnectionImpl.connectWithRetries(ConnectionImpl.java:2344) com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2326) com.mysql.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:832) com.mysql.jdbc.JDBC4Connection.<init>(JDBC4Connection.java:46) sun.reflect.GeneratedConstructorAccessor131.newInstance(Unknown Source) sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) java.lang.reflect.Constructor.newInstance(Constructor.java:526) com.mysql.jdbc.Util.handleNewInstance(Util.java:409) com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:417) com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:344) com.mchange.v2.c3p0.DriverManagerDataSource.getConnection(DriverManagerDataSource.java:146) com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:195) com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:184) com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.acquireResource(C3P0PooledConnectionPool.java:200) com.mchange.v2.resourcepool.BasicResourcePool.doAcquire(BasicResourcePool.java:1086) com.mchange.v2.resourcepool.BasicResourcePool.doAcquireAndDecrementPendingAcquiresWithinLockOnSuccess(BasicResourcePool.java:1073) com.mchange.v2.resourcepool.BasicResourcePool.access$800(BasicResourcePool.java:44) com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask.run(BasicResourcePool.java:1810) com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:648) {noformat} |