[BSERV-9743] Pull request rescoping deadlocks on Bitbucket Server 5.0.0

Type: Bug
Resolution: Fixed
Priority: Highest
Fix Version/s: 5.1.0, 5.0.1
Affects Version/s: 5.0.0
Component/s: None
Labels:
None

Symptom Severity:
Severity 1 - Critical
Bug Fix Policy:
View Atlassian Server bug fix policy

Notice: Although this bug report was raised against an environment involving Microsoft SQL Server, the problem itself is not specific to this environment. It is known to impact MySQL 5.7 and likely impacts other databases. Testing on PostgreSQL suggests it does not experience a deadlock, however all customers running version 5.0.0 should upgrade to 5.0.1 or later.

Summary

After upgrading to 5.0.0 on Windows connected to SQL Server pull request operations (decline, merge, add comment, etc) never complete.

Looking at the thread dumps, we see several threads that are stuck trying to flush requests to the DB for example:

"http-nio-8085-exec-10" daemon prio=5 tid=0x000000000000001c nid=0 runnable 
   java.lang.Thread.State: RUNNABLE
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
	at java.net.SocketInputStream.read(SocketInputStream.java:171)
	at java.net.SocketInputStream.read(SocketInputStream.java:141)
	at com.microsoft.sqlserver.jdbc.TDSChannel.read(IOBuffer.java:1877)
	at com.microsoft.sqlserver.jdbc.TDSReader.readPacket(IOBuffer.java:6642)
	- locked <0x0000000048d652ab> (a com.microsoft.sqlserver.jdbc.TDSReader)
	at com.microsoft.sqlserver.jdbc.TDSCommand.startResponse(IOBuffer.java:7959)
	at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.doExecutePreparedStatement(SQLServerPreparedStatement.java:438)
	at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement$PrepStmtExecCmd.doExecute(SQLServerPreparedStatement.java:385)
	at com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:7505)
	at com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLServerConnection.java:2444)
	- locked <0x0000000015cc4a77> (a java.lang.Object)
	at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeCommand(SQLServerStatement.java:191)
	at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeStatement(SQLServerStatement.java:166)
	at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.executeUpdate(SQLServerPreparedStatement.java:328)
	at com.zaxxer.hikari.pool.ProxyPreparedStatement.executeUpdate(ProxyPreparedStatement.java:61)
	at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.executeUpdate(HikariProxyPreparedStatement.java)
	at org.hibernate.engine.jdbc.internal.ResultSetReturnImpl.executeUpdate(ResultSetReturnImpl.java:205)
	at org.hibernate.persister.entity.AbstractEntityPersister.insert(AbstractEntityPersister.java:3009)
	at org.hibernate.persister.entity.AbstractEntityPersister.insert(AbstractEntityPersister.java:3503)
	at org.hibernate.action.internal.EntityInsertAction.execute(EntityInsertAction.java:89)
	at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:589)
	at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:463)
	at org.hibernate.event.internal.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:337)
	at org.hibernate.event.internal.DefaultAutoFlushEventListener.onAutoFlush(DefaultAutoFlushEventListener.java:50)
	at org.hibernate.internal.SessionImpl.autoFlushIfRequired(SessionImpl.java:1396)
	at org.hibernate.internal.SessionImpl.list(SessionImpl.java:1889)
	at org.hibernate.internal.CriteriaImpl.list(CriteriaImpl.java:366)
	at org.hibernate.internal.CriteriaImpl.uniqueResult(CriteriaImpl.java:388)
	at com.atlassian.stash.internal.pull.HibernatePullRequestParticipantDao.findByPullRequestAndUser(HibernatePullRequestParticipantDao.java:55)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:333)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
	at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:85)
...

and

"pull-request-rescoping:thread-1" daemon prio=5 tid=0x00000000000000e7 nid=0 runnable 
   java.lang.Thread.State: RUNNABLE
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
	at java.net.SocketInputStream.read(SocketInputStream.java:171)
	at java.net.SocketInputStream.read(SocketInputStream.java:141)
	at com.microsoft.sqlserver.jdbc.TDSChannel.read(IOBuffer.java:1877)
	at com.microsoft.sqlserver.jdbc.TDSReader.readPacket(IOBuffer.java:6642)
	- locked <0x000000000dec746f> (a com.microsoft.sqlserver.jdbc.TDSReader)
	at com.microsoft.sqlserver.jdbc.TDSCommand.startResponse(IOBuffer.java:7959)
	at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.doExecutePreparedStatementBatch(SQLServerPreparedStatement.java:2072)
	at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement$PrepStmtBatchExecCmd.doExecute(SQLServerPreparedStatement.java:1967)
	at com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:7505)
	at com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLServerConnection.java:2444)
	- locked <0x0000000060b639fd> (a java.lang.Object)
	at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeCommand(SQLServerStatement.java:191)
	at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeStatement(SQLServerStatement.java:166)
	at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.executeBatch(SQLServerPreparedStatement.java:1858)
	at com.zaxxer.hikari.pool.ProxyStatement.executeBatch(ProxyStatement.java:125)
	at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.executeBatch(HikariProxyPreparedStatement.java)
	at org.hibernate.engine.jdbc.batch.internal.BatchingBatch.performExecution(BatchingBatch.java:111)
	at org.hibernate.engine.jdbc.batch.internal.BatchingBatch.doExecuteBatch(BatchingBatch.java:97)
	at org.hibernate.engine.jdbc.batch.internal.AbstractBatchImpl.execute(AbstractBatchImpl.java:147)
	at org.hibernate.engine.jdbc.internal.JdbcCoordinatorImpl.executeBatch(JdbcCoordinatorImpl.java:206)
	at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:618)
	at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:463)
	at org.hibernate.event.internal.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:337)
	at org.hibernate.event.internal.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:39)
	at org.hibernate.internal.SessionImpl.doFlush(SessionImpl.java:1435)
	at org.hibernate.internal.SessionImpl.managedFlush(SessionImpl.java:491)
	at org.hibernate.internal.SessionImpl.flushBeforeTransactionCompletion(SessionImpl.java:3201)
	at org.hibernate.internal.SessionImpl.beforeTransactionCompletion(SessionImpl.java:2411)
	at org.hibernate.engine.jdbc.internal.JdbcCoordinatorImpl.beforeTransactionCompletion(JdbcCoordinatorImpl.java:467)
	at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl.beforeCompletionCallback(JdbcResourceLocalTransactionCoordinatorImpl.java:146)
	at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl.access$100(JdbcResourceLocalTransactionCoordinatorImpl.java:38)
	at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl$TransactionDriverControlImpl.commit(JdbcResourceLocalTransactionCoordinatorImpl.java:220)
	at org.hibernate.engine.transaction.internal.TransactionImpl.commit(TransactionImpl.java:68)
	at org.springframework.orm.hibernate5.HibernateTransactionManager.doCommit(HibernateTransactionManager.java:582)
	at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:761)
	at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:730)
	at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:150)
	at com.atlassian.stash.internal.pull.comment.drift.DriftCommentUpdateProcessor.process(DriftCommentUpdateProcessor.java:192)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:333)
	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:207)
	at com.sun.proxy.$Proxy280.process(Unknown Source)
	at com.atlassian.stash.internal.pull.DefaultPullRequestService.updateComments(DefaultPullRequestService.java:1851)
	at com.atlassian.stash.internal.pull.DefaultPullRequestService.rescope(DefaultPullRequestService.java:746)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
...

Environment

Windows 2012 R2
SQL Server
Upgraded from 4.14.x

Workaround

None. The only fix for this issue is to rollback the 5.0.0 upgrade (it is not possible to downgrade) or upgrade to 5.0.1.

This issue only appears to affect SQL Server. Testing on PostgreSQL, for example, has shown no issues.

mentioned in: Page Failed to load; Page Failed to load; Page Failed to load

Owen made changes - 10/Apr/2019 6:47 AM

Workflow

Original: Stash Workflow - Restricted [ 2016091 ]

New: JAC Bug Workflow v3 [ 3137275 ]

Owen made changes - 24/Oct/2018 5:01 AM

Symptom Severity

Original: Critical [ 14430 ]

New: Severity 1 - Critical [ 15830 ]

Mark A. made changes - 17/Jul/2018 3:05 PM

Remote Link

Original: This issue links to "Page (Extranet)" [ 288006 ]

Mark A. made changes - 17/Jul/2018 2:56 PM

Remote Link

New: This issue links to "Page (Extranet)" [ 377267 ]

Renan Battaglin made changes - 26/May/2017 12:12 AM

Remote Link

Original: This issue links to "Page (Extranet)" [ 293083 ]

Michael Andreacchio made changes - 25/May/2017 9:33 AM

Remote Link

New: This issue links to "Page (Extranet)" [ 293083 ]

Ben Humphreys made changes - 15/May/2017 11:47 PM

Fix Version/s

New: 5.1.0 [ 71491 ]

Ben Humphreys made changes - 15/May/2017 6:42 AM

Summary

Original: Pull request rescoping deadlocks on Bitbucket Server 5.0 on SQL Server

New: Pull request rescoping deadlocks on Bitbucket Server 5.0.0

Ben Humphreys made changes - 15/May/2017 6:42 AM

Description

Original: h3. {color:#d04437}Notice:{color} Although this bug report was raised against an environment involving Microsoft SQL Server, the problem itself is not specific to this environment. It is known to impact MySQL 5.7 and likely impacts other databases. Testing on PostgreSQL suggests it is not impacted, however all customers running version 5.0.0 should upgrade to 5.0.1 or later.

h4. Summary

After upgrading to 5.0.0 on Windows connected to SQL Server pull request operations (decline, merge, add comment, etc) never complete.

Looking at the thread dumps, we see several threads that are stuck trying to flush requests to the DB for example:

{code}"http-nio-8085-exec-10" daemon prio=5 tid=0x000000000000001c nid=0 runnable
java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:171)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at com.microsoft.sqlserver.jdbc.TDSChannel.read(IOBuffer.java:1877)
at com.microsoft.sqlserver.jdbc.TDSReader.readPacket(IOBuffer.java:6642)
- locked <0x0000000048d652ab> (a com.microsoft.sqlserver.jdbc.TDSReader)
at com.microsoft.sqlserver.jdbc.TDSCommand.startResponse(IOBuffer.java:7959)
at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.doExecutePreparedStatement(SQLServerPreparedStatement.java:438)
at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement$PrepStmtExecCmd.doExecute(SQLServerPreparedStatement.java:385)
at com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:7505)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLServerConnection.java:2444)
- locked <0x0000000015cc4a77> (a java.lang.Object)
at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeCommand(SQLServerStatement.java:191)
at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeStatement(SQLServerStatement.java:166)
at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.executeUpdate(SQLServerPreparedStatement.java:328)
at com.zaxxer.hikari.pool.ProxyPreparedStatement.executeUpdate(ProxyPreparedStatement.java:61)
at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.executeUpdate(HikariProxyPreparedStatement.java)
at org.hibernate.engine.jdbc.internal.ResultSetReturnImpl.executeUpdate(ResultSetReturnImpl.java:205)
at org.hibernate.persister.entity.AbstractEntityPersister.insert(AbstractEntityPersister.java:3009)
at org.hibernate.persister.entity.AbstractEntityPersister.insert(AbstractEntityPersister.java:3503)
at org.hibernate.action.internal.EntityInsertAction.execute(EntityInsertAction.java:89)
at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:589)
at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:463)
at org.hibernate.event.internal.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:337)
at org.hibernate.event.internal.DefaultAutoFlushEventListener.onAutoFlush(DefaultAutoFlushEventListener.java:50)
at org.hibernate.internal.SessionImpl.autoFlushIfRequired(SessionImpl.java:1396)
at org.hibernate.internal.SessionImpl.list(SessionImpl.java:1889)
at org.hibernate.internal.CriteriaImpl.list(CriteriaImpl.java:366)
at org.hibernate.internal.CriteriaImpl.uniqueResult(CriteriaImpl.java:388)
at com.atlassian.stash.internal.pull.HibernatePullRequestParticipantDao.findByPullRequestAndUser(HibernatePullRequestParticipantDao.java:55)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:333)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:85)
...{code}

and

{code}"pull-request-rescoping:thread-1" daemon prio=5 tid=0x00000000000000e7 nid=0 runnable
java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:171)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at com.microsoft.sqlserver.jdbc.TDSChannel.read(IOBuffer.java:1877)
at com.microsoft.sqlserver.jdbc.TDSReader.readPacket(IOBuffer.java:6642)
- locked <0x000000000dec746f> (a com.microsoft.sqlserver.jdbc.TDSReader)
at com.microsoft.sqlserver.jdbc.TDSCommand.startResponse(IOBuffer.java:7959)
at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.doExecutePreparedStatementBatch(SQLServerPreparedStatement.java:2072)
at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement$PrepStmtBatchExecCmd.doExecute(SQLServerPreparedStatement.java:1967)
at com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:7505)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLServerConnection.java:2444)
- locked <0x0000000060b639fd> (a java.lang.Object)
at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeCommand(SQLServerStatement.java:191)
at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeStatement(SQLServerStatement.java:166)
at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.executeBatch(SQLServerPreparedStatement.java:1858)
at com.zaxxer.hikari.pool.ProxyStatement.executeBatch(ProxyStatement.java:125)
at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.executeBatch(HikariProxyPreparedStatement.java)
at org.hibernate.engine.jdbc.batch.internal.BatchingBatch.performExecution(BatchingBatch.java:111)
at org.hibernate.engine.jdbc.batch.internal.BatchingBatch.doExecuteBatch(BatchingBatch.java:97)
at org.hibernate.engine.jdbc.batch.internal.AbstractBatchImpl.execute(AbstractBatchImpl.java:147)
at org.hibernate.engine.jdbc.internal.JdbcCoordinatorImpl.executeBatch(JdbcCoordinatorImpl.java:206)
at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:618)
at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:463)
at org.hibernate.event.internal.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:337)
at org.hibernate.event.internal.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:39)
at org.hibernate.internal.SessionImpl.doFlush(SessionImpl.java:1435)
at org.hibernate.internal.SessionImpl.managedFlush(SessionImpl.java:491)
at org.hibernate.internal.SessionImpl.flushBeforeTransactionCompletion(SessionImpl.java:3201)
at org.hibernate.internal.SessionImpl.beforeTransactionCompletion(SessionImpl.java:2411)
at org.hibernate.engine.jdbc.internal.JdbcCoordinatorImpl.beforeTransactionCompletion(JdbcCoordinatorImpl.java:467)
at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl.beforeCompletionCallback(JdbcResourceLocalTransactionCoordinatorImpl.java:146)
at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl.access$100(JdbcResourceLocalTransactionCoordinatorImpl.java:38)
at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl$TransactionDriverControlImpl.commit(JdbcResourceLocalTransactionCoordinatorImpl.java:220)
at org.hibernate.engine.transaction.internal.TransactionImpl.commit(TransactionImpl.java:68)
at org.springframework.orm.hibernate5.HibernateTransactionManager.doCommit(HibernateTransactionManager.java:582)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:761)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:730)
at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:150)
at com.atlassian.stash.internal.pull.comment.drift.DriftCommentUpdateProcessor.process(DriftCommentUpdateProcessor.java:192)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:333)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:207)
at com.sun.proxy.$Proxy280.process(Unknown Source)
at com.atlassian.stash.internal.pull.DefaultPullRequestService.updateComments(DefaultPullRequestService.java:1851)
at com.atlassian.stash.internal.pull.DefaultPullRequestService.rescope(DefaultPullRequestService.java:746)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
...{code}

h4. Environment

* Windows 2012 R2
* SQL Server
* Upgraded from 4.14.x

h4. Workaround

None. The only fix for this issue is to rollback the 5.0.0 upgrade (it is _not_ possible to downgrade) or upgrade to 5.0.1.

This issue only appears to affect SQL Server. Testing on PostgreSQL, for example, has shown no issues.

New: h3. {color:#d04437}Notice:{color} Although this bug report was raised against an environment involving Microsoft SQL Server, the problem itself is not specific to this environment. It is known to impact MySQL 5.7 and likely impacts other databases. Testing on PostgreSQL suggests it does not experience a deadlock, however all customers running version 5.0.0 should upgrade to 5.0.1 or later.

h4. Summary

After upgrading to 5.0.0 on Windows connected to SQL Server pull request operations (decline, merge, add comment, etc) never complete.

Looking at the thread dumps, we see several threads that are stuck trying to flush requests to the DB for example:

{code}"http-nio-8085-exec-10" daemon prio=5 tid=0x000000000000001c nid=0 runnable
java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:171)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at com.microsoft.sqlserver.jdbc.TDSChannel.read(IOBuffer.java:1877)
at com.microsoft.sqlserver.jdbc.TDSReader.readPacket(IOBuffer.java:6642)
- locked <0x0000000048d652ab> (a com.microsoft.sqlserver.jdbc.TDSReader)
at com.microsoft.sqlserver.jdbc.TDSCommand.startResponse(IOBuffer.java:7959)
at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.doExecutePreparedStatement(SQLServerPreparedStatement.java:438)
at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement$PrepStmtExecCmd.doExecute(SQLServerPreparedStatement.java:385)
at com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:7505)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLServerConnection.java:2444)
- locked <0x0000000015cc4a77> (a java.lang.Object)
at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeCommand(SQLServerStatement.java:191)
at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeStatement(SQLServerStatement.java:166)
at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.executeUpdate(SQLServerPreparedStatement.java:328)
at com.zaxxer.hikari.pool.ProxyPreparedStatement.executeUpdate(ProxyPreparedStatement.java:61)
at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.executeUpdate(HikariProxyPreparedStatement.java)
at org.hibernate.engine.jdbc.internal.ResultSetReturnImpl.executeUpdate(ResultSetReturnImpl.java:205)
at org.hibernate.persister.entity.AbstractEntityPersister.insert(AbstractEntityPersister.java:3009)
at org.hibernate.persister.entity.AbstractEntityPersister.insert(AbstractEntityPersister.java:3503)
at org.hibernate.action.internal.EntityInsertAction.execute(EntityInsertAction.java:89)
at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:589)
at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:463)
at org.hibernate.event.internal.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:337)
at org.hibernate.event.internal.DefaultAutoFlushEventListener.onAutoFlush(DefaultAutoFlushEventListener.java:50)
at org.hibernate.internal.SessionImpl.autoFlushIfRequired(SessionImpl.java:1396)
at org.hibernate.internal.SessionImpl.list(SessionImpl.java:1889)
at org.hibernate.internal.CriteriaImpl.list(CriteriaImpl.java:366)
at org.hibernate.internal.CriteriaImpl.uniqueResult(CriteriaImpl.java:388)
at com.atlassian.stash.internal.pull.HibernatePullRequestParticipantDao.findByPullRequestAndUser(HibernatePullRequestParticipantDao.java:55)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:333)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:85)
...{code}

and

{code}"pull-request-rescoping:thread-1" daemon prio=5 tid=0x00000000000000e7 nid=0 runnable
java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:171)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at com.microsoft.sqlserver.jdbc.TDSChannel.read(IOBuffer.java:1877)
at com.microsoft.sqlserver.jdbc.TDSReader.readPacket(IOBuffer.java:6642)
- locked <0x000000000dec746f> (a com.microsoft.sqlserver.jdbc.TDSReader)
at com.microsoft.sqlserver.jdbc.TDSCommand.startResponse(IOBuffer.java:7959)
at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.doExecutePreparedStatementBatch(SQLServerPreparedStatement.java:2072)
at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement$PrepStmtBatchExecCmd.doExecute(SQLServerPreparedStatement.java:1967)
at com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:7505)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLServerConnection.java:2444)
- locked <0x0000000060b639fd> (a java.lang.Object)
at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeCommand(SQLServerStatement.java:191)
at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeStatement(SQLServerStatement.java:166)
at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.executeBatch(SQLServerPreparedStatement.java:1858)
at com.zaxxer.hikari.pool.ProxyStatement.executeBatch(ProxyStatement.java:125)
at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.executeBatch(HikariProxyPreparedStatement.java)
at org.hibernate.engine.jdbc.batch.internal.BatchingBatch.performExecution(BatchingBatch.java:111)
at org.hibernate.engine.jdbc.batch.internal.BatchingBatch.doExecuteBatch(BatchingBatch.java:97)
at org.hibernate.engine.jdbc.batch.internal.AbstractBatchImpl.execute(AbstractBatchImpl.java:147)
at org.hibernate.engine.jdbc.internal.JdbcCoordinatorImpl.executeBatch(JdbcCoordinatorImpl.java:206)
at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:618)
at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:463)
at org.hibernate.event.internal.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:337)
at org.hibernate.event.internal.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:39)
at org.hibernate.internal.SessionImpl.doFlush(SessionImpl.java:1435)
at org.hibernate.internal.SessionImpl.managedFlush(SessionImpl.java:491)
at org.hibernate.internal.SessionImpl.flushBeforeTransactionCompletion(SessionImpl.java:3201)
at org.hibernate.internal.SessionImpl.beforeTransactionCompletion(SessionImpl.java:2411)
at org.hibernate.engine.jdbc.internal.JdbcCoordinatorImpl.beforeTransactionCompletion(JdbcCoordinatorImpl.java:467)
at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl.beforeCompletionCallback(JdbcResourceLocalTransactionCoordinatorImpl.java:146)
at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl.access$100(JdbcResourceLocalTransactionCoordinatorImpl.java:38)
at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl$TransactionDriverControlImpl.commit(JdbcResourceLocalTransactionCoordinatorImpl.java:220)
at org.hibernate.engine.transaction.internal.TransactionImpl.commit(TransactionImpl.java:68)
at org.springframework.orm.hibernate5.HibernateTransactionManager.doCommit(HibernateTransactionManager.java:582)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:761)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:730)
at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:150)
at com.atlassian.stash.internal.pull.comment.drift.DriftCommentUpdateProcessor.process(DriftCommentUpdateProcessor.java:192)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:333)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:207)
at com.sun.proxy.$Proxy280.process(Unknown Source)
at com.atlassian.stash.internal.pull.DefaultPullRequestService.updateComments(DefaultPullRequestService.java:1851)
at com.atlassian.stash.internal.pull.DefaultPullRequestService.rescope(DefaultPullRequestService.java:746)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
...{code}

h4. Environment

* Windows 2012 R2
* SQL Server
* Upgraded from 4.14.x

h4. Workaround

None. The only fix for this issue is to rollback the 5.0.0 upgrade (it is _not_ possible to downgrade) or upgrade to 5.0.1.

This issue only appears to affect SQL Server. Testing on PostgreSQL, for example, has shown no issues.

Ben Humphreys made changes - 15/May/2017 6:41 AM

Description

Original: h3. {color:#205081}Notice:{color} Although this bug report was raised against an environment involving Microsoft SQL Server, the problem itself is not specific to this environment. It is known to impact MySQL 5.7 and likely impacts other databases. Testing on PostgreSQL suggests it is not impacted, however all customers running version 5.0.0 should upgrade to 5.0.1 or later.

h4. Summary

After upgrading to 5.0.0 on Windows connected to SQL Server pull request operations (decline, merge, add comment, etc) never complete.

Looking at the thread dumps, we see several threads that are stuck trying to flush requests to the DB for example:

{code}"http-nio-8085-exec-10" daemon prio=5 tid=0x000000000000001c nid=0 runnable
java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:171)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at com.microsoft.sqlserver.jdbc.TDSChannel.read(IOBuffer.java:1877)
at com.microsoft.sqlserver.jdbc.TDSReader.readPacket(IOBuffer.java:6642)
- locked <0x0000000048d652ab> (a com.microsoft.sqlserver.jdbc.TDSReader)
at com.microsoft.sqlserver.jdbc.TDSCommand.startResponse(IOBuffer.java:7959)
at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.doExecutePreparedStatement(SQLServerPreparedStatement.java:438)
at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement$PrepStmtExecCmd.doExecute(SQLServerPreparedStatement.java:385)
at com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:7505)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLServerConnection.java:2444)
- locked <0x0000000015cc4a77> (a java.lang.Object)
at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeCommand(SQLServerStatement.java:191)
at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeStatement(SQLServerStatement.java:166)
at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.executeUpdate(SQLServerPreparedStatement.java:328)
at com.zaxxer.hikari.pool.ProxyPreparedStatement.executeUpdate(ProxyPreparedStatement.java:61)
at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.executeUpdate(HikariProxyPreparedStatement.java)
at org.hibernate.engine.jdbc.internal.ResultSetReturnImpl.executeUpdate(ResultSetReturnImpl.java:205)
at org.hibernate.persister.entity.AbstractEntityPersister.insert(AbstractEntityPersister.java:3009)
at org.hibernate.persister.entity.AbstractEntityPersister.insert(AbstractEntityPersister.java:3503)
at org.hibernate.action.internal.EntityInsertAction.execute(EntityInsertAction.java:89)
at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:589)
at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:463)
at org.hibernate.event.internal.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:337)
at org.hibernate.event.internal.DefaultAutoFlushEventListener.onAutoFlush(DefaultAutoFlushEventListener.java:50)
at org.hibernate.internal.SessionImpl.autoFlushIfRequired(SessionImpl.java:1396)
at org.hibernate.internal.SessionImpl.list(SessionImpl.java:1889)
at org.hibernate.internal.CriteriaImpl.list(CriteriaImpl.java:366)
at org.hibernate.internal.CriteriaImpl.uniqueResult(CriteriaImpl.java:388)
at com.atlassian.stash.internal.pull.HibernatePullRequestParticipantDao.findByPullRequestAndUser(HibernatePullRequestParticipantDao.java:55)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:333)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:85)
...{code}

and

{code}"pull-request-rescoping:thread-1" daemon prio=5 tid=0x00000000000000e7 nid=0 runnable
java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:171)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at com.microsoft.sqlserver.jdbc.TDSChannel.read(IOBuffer.java:1877)
at com.microsoft.sqlserver.jdbc.TDSReader.readPacket(IOBuffer.java:6642)
- locked <0x000000000dec746f> (a com.microsoft.sqlserver.jdbc.TDSReader)
at com.microsoft.sqlserver.jdbc.TDSCommand.startResponse(IOBuffer.java:7959)
at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.doExecutePreparedStatementBatch(SQLServerPreparedStatement.java:2072)
at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement$PrepStmtBatchExecCmd.doExecute(SQLServerPreparedStatement.java:1967)
at com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:7505)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLServerConnection.java:2444)
- locked <0x0000000060b639fd> (a java.lang.Object)
at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeCommand(SQLServerStatement.java:191)
at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeStatement(SQLServerStatement.java:166)
at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.executeBatch(SQLServerPreparedStatement.java:1858)
at com.zaxxer.hikari.pool.ProxyStatement.executeBatch(ProxyStatement.java:125)
at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.executeBatch(HikariProxyPreparedStatement.java)
at org.hibernate.engine.jdbc.batch.internal.BatchingBatch.performExecution(BatchingBatch.java:111)
at org.hibernate.engine.jdbc.batch.internal.BatchingBatch.doExecuteBatch(BatchingBatch.java:97)
at org.hibernate.engine.jdbc.batch.internal.AbstractBatchImpl.execute(AbstractBatchImpl.java:147)
at org.hibernate.engine.jdbc.internal.JdbcCoordinatorImpl.executeBatch(JdbcCoordinatorImpl.java:206)
at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:618)
at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:463)
at org.hibernate.event.internal.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:337)
at org.hibernate.event.internal.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:39)
at org.hibernate.internal.SessionImpl.doFlush(SessionImpl.java:1435)
at org.hibernate.internal.SessionImpl.managedFlush(SessionImpl.java:491)
at org.hibernate.internal.SessionImpl.flushBeforeTransactionCompletion(SessionImpl.java:3201)
at org.hibernate.internal.SessionImpl.beforeTransactionCompletion(SessionImpl.java:2411)
at org.hibernate.engine.jdbc.internal.JdbcCoordinatorImpl.beforeTransactionCompletion(JdbcCoordinatorImpl.java:467)
at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl.beforeCompletionCallback(JdbcResourceLocalTransactionCoordinatorImpl.java:146)
at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl.access$100(JdbcResourceLocalTransactionCoordinatorImpl.java:38)
at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl$TransactionDriverControlImpl.commit(JdbcResourceLocalTransactionCoordinatorImpl.java:220)
at org.hibernate.engine.transaction.internal.TransactionImpl.commit(TransactionImpl.java:68)
at org.springframework.orm.hibernate5.HibernateTransactionManager.doCommit(HibernateTransactionManager.java:582)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:761)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:730)
at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:150)
at com.atlassian.stash.internal.pull.comment.drift.DriftCommentUpdateProcessor.process(DriftCommentUpdateProcessor.java:192)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:333)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:207)
at com.sun.proxy.$Proxy280.process(Unknown Source)
at com.atlassian.stash.internal.pull.DefaultPullRequestService.updateComments(DefaultPullRequestService.java:1851)
at com.atlassian.stash.internal.pull.DefaultPullRequestService.rescope(DefaultPullRequestService.java:746)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
...{code}

h4. Environment

* Windows 2012 R2
* SQL Server
* Upgraded from 4.14.x

h4. Workaround

None. The only fix for this issue is to rollback the 5.0.0 upgrade (it is _not_ possible to downgrade) or upgrade to 5.0.1.

This issue only appears to affect SQL Server. Testing on PostgreSQL, for example, has shown no issues.

New: h3. {color:#d04437}Notice:{color} Although this bug report was raised against an environment involving Microsoft SQL Server, the problem itself is not specific to this environment. It is known to impact MySQL 5.7 and likely impacts other databases. Testing on PostgreSQL suggests it is not impacted, however all customers running version 5.0.0 should upgrade to 5.0.1 or later.

h4. Summary

After upgrading to 5.0.0 on Windows connected to SQL Server pull request operations (decline, merge, add comment, etc) never complete.

Looking at the thread dumps, we see several threads that are stuck trying to flush requests to the DB for example:

{code}"http-nio-8085-exec-10" daemon prio=5 tid=0x000000000000001c nid=0 runnable
java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:171)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at com.microsoft.sqlserver.jdbc.TDSChannel.read(IOBuffer.java:1877)
at com.microsoft.sqlserver.jdbc.TDSReader.readPacket(IOBuffer.java:6642)
- locked <0x0000000048d652ab> (a com.microsoft.sqlserver.jdbc.TDSReader)
at com.microsoft.sqlserver.jdbc.TDSCommand.startResponse(IOBuffer.java:7959)
at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.doExecutePreparedStatement(SQLServerPreparedStatement.java:438)
at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement$PrepStmtExecCmd.doExecute(SQLServerPreparedStatement.java:385)
at com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:7505)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLServerConnection.java:2444)
- locked <0x0000000015cc4a77> (a java.lang.Object)
at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeCommand(SQLServerStatement.java:191)
at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeStatement(SQLServerStatement.java:166)
at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.executeUpdate(SQLServerPreparedStatement.java:328)
at com.zaxxer.hikari.pool.ProxyPreparedStatement.executeUpdate(ProxyPreparedStatement.java:61)
at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.executeUpdate(HikariProxyPreparedStatement.java)
at org.hibernate.engine.jdbc.internal.ResultSetReturnImpl.executeUpdate(ResultSetReturnImpl.java:205)
at org.hibernate.persister.entity.AbstractEntityPersister.insert(AbstractEntityPersister.java:3009)
at org.hibernate.persister.entity.AbstractEntityPersister.insert(AbstractEntityPersister.java:3503)
at org.hibernate.action.internal.EntityInsertAction.execute(EntityInsertAction.java:89)
at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:589)
at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:463)
at org.hibernate.event.internal.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:337)
at org.hibernate.event.internal.DefaultAutoFlushEventListener.onAutoFlush(DefaultAutoFlushEventListener.java:50)
at org.hibernate.internal.SessionImpl.autoFlushIfRequired(SessionImpl.java:1396)
at org.hibernate.internal.SessionImpl.list(SessionImpl.java:1889)
at org.hibernate.internal.CriteriaImpl.list(CriteriaImpl.java:366)
at org.hibernate.internal.CriteriaImpl.uniqueResult(CriteriaImpl.java:388)
at com.atlassian.stash.internal.pull.HibernatePullRequestParticipantDao.findByPullRequestAndUser(HibernatePullRequestParticipantDao.java:55)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:333)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:85)
...{code}

and

{code}"pull-request-rescoping:thread-1" daemon prio=5 tid=0x00000000000000e7 nid=0 runnable
java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:171)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at com.microsoft.sqlserver.jdbc.TDSChannel.read(IOBuffer.java:1877)
at com.microsoft.sqlserver.jdbc.TDSReader.readPacket(IOBuffer.java:6642)
- locked <0x000000000dec746f> (a com.microsoft.sqlserver.jdbc.TDSReader)
at com.microsoft.sqlserver.jdbc.TDSCommand.startResponse(IOBuffer.java:7959)
at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.doExecutePreparedStatementBatch(SQLServerPreparedStatement.java:2072)
at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement$PrepStmtBatchExecCmd.doExecute(SQLServerPreparedStatement.java:1967)
at com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:7505)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLServerConnection.java:2444)
- locked <0x0000000060b639fd> (a java.lang.Object)
at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeCommand(SQLServerStatement.java:191)
at com.microsoft.sqlserver.jdbc.SQLServerStatement.executeStatement(SQLServerStatement.java:166)
at com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.executeBatch(SQLServerPreparedStatement.java:1858)
at com.zaxxer.hikari.pool.ProxyStatement.executeBatch(ProxyStatement.java:125)
at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.executeBatch(HikariProxyPreparedStatement.java)
at org.hibernate.engine.jdbc.batch.internal.BatchingBatch.performExecution(BatchingBatch.java:111)
at org.hibernate.engine.jdbc.batch.internal.BatchingBatch.doExecuteBatch(BatchingBatch.java:97)
at org.hibernate.engine.jdbc.batch.internal.AbstractBatchImpl.execute(AbstractBatchImpl.java:147)
at org.hibernate.engine.jdbc.internal.JdbcCoordinatorImpl.executeBatch(JdbcCoordinatorImpl.java:206)
at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:618)
at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:463)
at org.hibernate.event.internal.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:337)
at org.hibernate.event.internal.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:39)
at org.hibernate.internal.SessionImpl.doFlush(SessionImpl.java:1435)
at org.hibernate.internal.SessionImpl.managedFlush(SessionImpl.java:491)
at org.hibernate.internal.SessionImpl.flushBeforeTransactionCompletion(SessionImpl.java:3201)
at org.hibernate.internal.SessionImpl.beforeTransactionCompletion(SessionImpl.java:2411)
at org.hibernate.engine.jdbc.internal.JdbcCoordinatorImpl.beforeTransactionCompletion(JdbcCoordinatorImpl.java:467)
at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl.beforeCompletionCallback(JdbcResourceLocalTransactionCoordinatorImpl.java:146)
at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl.access$100(JdbcResourceLocalTransactionCoordinatorImpl.java:38)
at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl$TransactionDriverControlImpl.commit(JdbcResourceLocalTransactionCoordinatorImpl.java:220)
at org.hibernate.engine.transaction.internal.TransactionImpl.commit(TransactionImpl.java:68)
at org.springframework.orm.hibernate5.HibernateTransactionManager.doCommit(HibernateTransactionManager.java:582)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:761)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:730)
at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:150)
at com.atlassian.stash.internal.pull.comment.drift.DriftCommentUpdateProcessor.process(DriftCommentUpdateProcessor.java:192)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:333)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:207)
at com.sun.proxy.$Proxy280.process(Unknown Source)
at com.atlassian.stash.internal.pull.DefaultPullRequestService.updateComments(DefaultPullRequestService.java:1851)
at com.atlassian.stash.internal.pull.DefaultPullRequestService.rescope(DefaultPullRequestService.java:746)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
...{code}

h4. Environment

* Windows 2012 R2
* SQL Server
* Upgraded from 4.14.x

h4. Workaround

None. The only fix for this issue is to rollback the 5.0.0 upgrade (it is _not_ possible to downgrade) or upgrade to 5.0.1.

This issue only appears to affect SQL Server. Testing on PostgreSQL, for example, has shown no issues.

Assignee:: Juan Palacios (Inactive)

Reporter:: Jeff Thomas

Affected customers:: 1 This affects my team

Watchers:: 7 Start watching this issue

Created:: 09/May/2017 11:45 PM

Updated:: 24/Oct/2018 5:01 AM

Resolved:: 10/May/2017 5:05 AM

Bitbucket Data Center

Details

Description

Summary

Environment

Workaround

Attachments

Issue Links

Forms

Activity

People

Dates