Uploaded image for project: 'Bitbucket Data Center'
  1. Bitbucket Data Center
  2. BSERV-3460

During directory synchronization, commit errors trigger cascading rollback failures

    XMLWordPrintable

Details

    Description

      During directory synchronization, if a batch is processed successfully and then fails to be committed, it results in cascading rollback failures which completely block directory synchronization from that point forward.

      The most likely cause for commit failure is constraint violations. For example, if Crowd synchronizes down two groups whose names vary only by case, that can result in an exception similar to the following:

      2013-05-07 17:18:50,983 ERROR [scheduler_Worker-1]  o.h.e.j.batch.internal.BatchingBatch HHH000315: Exception executing batch [integrity constraint violation: unique constraint or index violation: SYS_IDX_SYS_CT_10149_10151]
      2013-05-07 17:18:50,996 WARN  [scheduler_Worker-1]  c.a.c.u.p.h.b.AbstractBatchProcessor batch failed falling back to individual processing
      org.springframework.dao.DataIntegrityViolationException: integrity constraint violation: unique constraint or index violation: SYS_IDX_SYS_CT_10149_10151; SQL [n/a]; constraint [null]; nested exception is org.hibernate.exception.ConstraintViolationException: integrity constraint violation: unique constraint or index violation: SYS_IDX_SYS_CT_10149_10151
      	at org.springframework.orm.hibernate4.SessionFactoryUtils.convertHibernateAccessException(SessionFactoryUtils.java:138) ~[spring-orm-3.2.0.RELEASE.jar:3.2.0.RELEASE]
      	at org.springframework.orm.hibernate4.HibernateTransactionManager.convertHibernateAccessException(HibernateTransactionManager.java:594) ~[spring-orm-3.2.0.RELEASE.jar:3.2.0.RELEASE]
      	at org.springframework.orm.hibernate4.HibernateTransactionManager.doCommit(HibernateTransactionManager.java:476) ~[spring-orm-3.2.0.RELEASE.jar:3.2.0.RELEASE]
      	at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:754) ~[spring-tx-3.2.0.RELEASE.jar:3.2.0.RELEASE]
      	at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:723) ~[spring-tx-3.2.0.RELEASE.jar:3.2.0.RELEASE]
      	at com.atlassian.stash.internal.crowd.TransactionManagerBatchProcessor.commitTransaction(TransactionManagerBatchProcessor.java:45) ~[stash-dao-impl-2.1.2.jar:na]
      	at com.atlassian.crowd.util.persistence.hibernate.batch.hibernate4.Hibernate4BatchProcessor.afterProcessBatch(Hibernate4BatchProcessor.java:31) ~[crowd-persistence-hibernate4-2.5.3-m1.jar:na]
      	at com.atlassian.crowd.util.persistence.hibernate.batch.AbstractBatchProcessor.processBatch(AbstractBatchProcessor.java:160) [crowd-persistence-2.5.3-m1.jar:na]
      	at com.atlassian.crowd.util.persistence.hibernate.batch.AbstractBatchProcessor.execute(AbstractBatchProcessor.java:121) [crowd-persistence-2.5.3-m1.jar:na]
      

      Spring's TransactionManager automatically rolls back the transaction in its exception handling for `AbstractPlatformTransactionManager.processCommit`. Before Crowd attempts to start processing individual rows, it first attempts to rollback the transaction. That results in a failure like this:

      2013-05-07 17:18:51,009 ERROR [scheduler_Worker-1]  c.a.c.d.DbCachingDirectoryPoller Error occurred while refreshing the cache for directory [ 32770 ].
      org.springframework.transaction.IllegalTransactionStateException: Transaction is already completed - do not call commit or rollback more than once per transaction
      	at org.springframework.transaction.support.AbstractPlatformTransactionManager.rollback(AbstractPlatformTransactionManager.java:817) ~[spring-tx-3.2.0.RELEASE.jar:3.2.0.RELEASE]
      	at com.atlassian.stash.internal.crowd.TransactionManagerBatchProcessor.rollbackTransaction(TransactionManagerBatchProcessor.java:52) ~[stash-dao-impl-2.1.2.jar:na]
      	at com.atlassian.crowd.util.persistence.hibernate.batch.hibernate4.Hibernate4BatchProcessor.rollbackProcessBatch(Hibernate4BatchProcessor.java:74) ~[crowd-persistence-hibernate4-2.5.3-m1.jar:na]
      	at com.atlassian.crowd.util.persistence.hibernate.batch.AbstractBatchProcessor.processBatch(AbstractBatchProcessor.java:166) ~[crowd-persistence-2.5.3-m1.jar:na]
      	at com.atlassian.crowd.util.persistence.hibernate.batch.AbstractBatchProcessor.execute(AbstractBatchProcessor.java:121) ~[crowd-persistence-2.5.3-m1.jar:na]
      

      The rollback failure is not handled by the batch processor, allowing it to propagate up the stack where it is handled by the Crowd DbCachingDirectoryPoller and terminates synchronization.

      Attachments

        Activity

          People

            bturner Bryan Turner (Inactive)
            bturner Bryan Turner (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: