Details
-
Bug
-
Resolution: Fixed
-
Low
-
3.5.13, 5.1.5
-
None
-
MySQL 5.5
JDK 1.6.0_26
Centos 5.8
MySQL JDBC Connector/J 5.1.17
Description
Today we experienced a DB deadlock which took our Confluence instance offline. The deadlock appeared to stem from c3p0, and we'd seem something similar a couple of months back in our Bamboo instance. At that time an Atlassian developer provided a patched version of c3p0 that eliminated the deadlock. I believe Confluence's own version of c3p0 needs to include the same patches, and the confluence.cfg.xml needs to be updated to include the relevant new property values.
The details of the deadlock are:
Found one Java-level deadlock:
=============================
"com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#2":
waiting to lock monitor 0x00002aaaf402aad0 (object 0x00000005afb836c0, a com.mysql.jdbc.JDBC4Connection),
which is held by "com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#1"
"com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#1":
waiting to lock monitor 0x00002aaaf402aa28 (object 0x00000005afb95110, a com.mysql.jdbc.JDBC4ResultSet),
which is held by "com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#2"
Java stack information for the threads listed above:
===================================================
"com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#2":
at com.mysql.jdbc.ConnectionImpl.getCharacterSetMetadata(ConnectionImpl.java:2799)
- waiting to lock <0x00000005afb836c0> (a com.mysql.jdbc.JDBC4Connection)
at com.mysql.jdbc.Field.getStringFromBytes(Field.java:710)
at com.mysql.jdbc.Field.getOriginalName(Field.java:652)
at com.mysql.jdbc.ResultSetImpl.buildIndexMapping(ResultSetImpl.java:749)
at com.mysql.jdbc.ResultSetImpl.findColumn(ResultSetImpl.java:1108)
- locked <0x00000005afb95110> (a com.mysql.jdbc.JDBC4ResultSet)
at com.mysql.jdbc.DatabaseMetaData$9.forEach(DatabaseMetaData.java:5030)
at com.mysql.jdbc.IterateBlock.doForAll(IterateBlock.java:51)
at com.mysql.jdbc.DatabaseMetaData.getTables(DatabaseMetaData.java:4962)
at com.mchange.v2.c3p0.impl.DefaultConnectionTester.activeCheckConnectionNoQuery(DefaultConnectionTester.java:185)
at com.mchange.v2.c3p0.impl.DefaultConnectionTester.activeCheckConnection(DefaultConnectionTester.java:62)
at com.mchange.v2.c3p0.AbstractConnectionTester.activeCheckConnection(AbstractConnectionTester.java:67)
at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.testPooledConnection(C3P0PooledConnectionPool.java:368)
at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.refurbishIdleResource(C3P0PooledConnectionPool.java:310)
at com.mchange.v2.resourcepool.BasicResourcePool$AsyncTestIdleResourceTask.run(BasicResourcePool.java:1999)
at com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:547)
"com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#1":
at com.mysql.jdbc.ResultSetImpl.realClose(ResultSetImpl.java:7477)
- waiting to lock <0x00000005afb95110> (a com.mysql.jdbc.JDBC4ResultSet)
at com.mysql.jdbc.ResultSetImpl.close(ResultSetImpl.java:907)
at com.mysql.jdbc.StatementImpl.realClose(StatementImpl.java:2430)
- locked <0x00000005afb95268> (a com.mysql.jdbc.StatementImpl)
at com.mysql.jdbc.ConnectionImpl.closeAllOpenStatements(ConnectionImpl.java:1585)
at com.mysql.jdbc.ConnectionImpl.realClose(ConnectionImpl.java:4361)
at com.mysql.jdbc.ConnectionImpl.close(ConnectionImpl.java:1557)
- locked <0x00000005afb836c0> (a com.mysql.jdbc.JDBC4Connection)
at com.mchange.v2.c3p0.impl.NewPooledConnection.close(NewPooledConnection.java:549)
at com.mchange.v2.c3p0.impl.NewPooledConnection.close(NewPooledConnection.java:234)
- locked <0x00000005afb941e8> (a com.mchange.v2.c3p0.impl.NewPooledConnection)
at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.destroyResource(C3P0PooledConnectionPool.java:470)
at com.mchange.v2.resourcepool.BasicResourcePool$1DestroyResourceTask.run(BasicResourcePool.java:964)
at com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:547)
Found 1 deadlock.
The original Bamboo ticket is BSP-6755, and it has all the relevant details and a patch of the C3P0 library to workaround the problem.
Workaround
You can workaround this issue in versions under 5.9 by manually dropping the latest c3p0 and mchange-commons jars into your installation directoy:
- Shut down Confluence
- Rename <install-directory>/confluence/WEB-INF/lib/c3p0-0.9.1.2.jar to c3p0-0.9.1.2.jar
- Download c3p0-0.9.5.1.jar
- Download mchange-commons-java-0.2.10.jar
- Copy them both into <install-directory>/confluence/WEB-INF/lib/
- Restart Confluence
Attachments
Issue Links
- is related to
-
-
- Closed
-
- mentioned in
-
Page Loading...