Uploaded image for project: 'Bamboo'
  1. Bamboo
  2. BAM-14215

Branches not marked for deletion when master plan/job is - causes deletion to fail

    XMLWordPrintable

Details

    Description

      Please note that this bug doesn't have any major impact on your instance (i.e. it doesn't make your instance freeze or become unusable).

      Summary

      When you delete a plan, Bamboo actually marks it and its plan branches for deletion as opposed to performing the operation right away. A daily task is run to perform the removal of plans in that state later in the day.

      This bug is about an inconsistency that happens in the database while marking "Plans" and its "Plan Branches" for deletion. The bug is that sometimes the "Plan" is marked for deletion but it children "Plan Branches" are not.

      When the daily tasks is run, Bamboo fails to remove that plan (or perhaps this could block deletion of other plans as well as the one with the inconsistency).

      Steps to Reproduce

      The problem happens intermittently and we don't have the steps to reproduce it.

      Expected Results

      It is expected that there is no inconsistency while marking plans for deletion.

      Actual Results

      The below exception is thrown in the atlassian-bamboo.log file:

      2014-01-15 08:50:47,707 WARN [QuartzScheduler_Worker-9] [JDBCExceptionReporter] SQL Error: 0, SQLState: 23503
      2014-01-15 08:50:47,708 ERROR [QuartzScheduler_Worker-9] [JDBCExceptionReporter] Batch entry 0 delete from BUILD where BUILD_ID='512561036' was aborted.  Call getNextException to see the cause.
      2014-01-15 08:50:47,708 WARN [QuartzScheduler_Worker-9] [JDBCExceptionReporter] SQL Error: 0, SQLState: 23503
      2014-01-15 08:50:47,708 ERROR [QuartzScheduler_Worker-9] [JDBCExceptionReporter] ERROR: update or delete on table "build" violates foreign key constraint "fk3c9ce4eb2b11c18" on table "build"
        Detail: Key (build_id)=(512561036) is still referenced from table "build".
      2014-01-15 08:50:47,708 WARN [QuartzScheduler_Worker-9] [JDBCExceptionReporter] SQL Error: 0, SQLState: 23503
      2014-01-15 08:50:47,708 ERROR [QuartzScheduler_Worker-9] [JDBCExceptionReporter] Batch entry 0 delete from BUILD where BUILD_ID='512561036' was aborted.  Call getNextException to see the cause.
      2014-01-15 08:50:47,708 WARN [QuartzScheduler_Worker-9] [JDBCExceptionReporter] SQL Error: 0, SQLState: 23503
      2014-01-15 08:50:47,708 ERROR [QuartzScheduler_Worker-9] [JDBCExceptionReporter] ERROR: update or delete on table "build" violates foreign key constraint "fk3c9ce4eb2b11c18" on table "build"
        Detail: Key (build_id)=(512561036) is still referenced from table "build".
      2014-01-15 08:50:47,708 ERROR [QuartzScheduler_Worker-9] [SessionImpl] Could not synchronize database state with session
      2014-01-15 08:50:47,708 ERROR [QuartzScheduler_Worker-9] [DeletionServiceImpl] Unable to complete delayed deletion:
      org.springframework.dao.DataIntegrityViolationException: Hibernate operation: Could not execute JDBC batch update; SQL []; Batch entry 0 delete from BUILD where BUILD_ID='512561036' was aborted.  Call getNextException to see the cause.; nested exception is java.sql.BatchUpdateException: Batch entry 0 delete from BUILD where BUILD_ID='512561036' was aborted.  Call getNextException to see the cause.
      Caused by: java.sql.BatchUpdateException: Batch entry 0 delete from BUILD where BUILD_ID='512561036' was aborted.  Call getNextException to see the cause.
              at org.postgresql.jdbc2.AbstractJdbc2Statement$BatchResultHandler.handleError(AbstractJdbc2Statement.java:2569)
              at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1796)
              at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:407)
              at org.postgresql.jdbc2.AbstractJdbc2Statement.executeBatch(AbstractJdbc2Statement.java:2708)
              at com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.executeBatch(NewProxyPreparedStatement.java:1723)
              at net.sf.hibernate.impl.BatchingBatcher.doExecuteBatch(BatchingBatcher.java:54)
              at net.sf.hibernate.impl.BatcherImpl.executeBatch(BatcherImpl.java:128)
              at net.sf.hibernate.impl.BatcherImpl.prepareStatement(BatcherImpl.java:61)
              at net.sf.hibernate.impl.BatcherImpl.prepareStatement(BatcherImpl.java:58)
              at net.sf.hibernate.impl.BatcherImpl.prepareBatchStatement(BatcherImpl.java:111)
              at net.sf.hibernate.persister.EntityPersister.delete(EntityPersister.java:564)
              at net.sf.hibernate.impl.ScheduledDeletion.execute(ScheduledDeletion.java:29)
              at net.sf.hibernate.impl.SessionImpl.execute(SessionImpl.java:2451)
              at net.sf.hibernate.impl.SessionImpl.executeAll(SessionImpl.java:2437)
              at net.sf.hibernate.impl.SessionImpl.execute(SessionImpl.java:2399)
              at net.sf.hibernate.impl.SessionImpl.flush(SessionImpl.java:2263)
              at net.sf.hibernate.transaction.JDBCTransaction.commit(JDBCTransaction.java:61)
              at org.springframework.orm.hibernate.HibernateTransactionManager.doCommit(HibernateTransactionManager.java:510)
              at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:662)
              at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:632)
              at sun.reflect.GeneratedMethodAccessor99.invoke(Unknown Source)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
              at java.lang.reflect.Method.invoke(Method.java:606)
              at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:304)
              at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:198)
              at com.sun.proxy.$Proxy20.commit(Unknown Source)
              at org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:314)
              at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:116)
              at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
              at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
              at com.sun.proxy.$Proxy45.deletePlan(Unknown Source)
              at com.atlassian.bamboo.deletion.DeletionServiceImpl.doDeletePlan(DeletionServiceImpl.java:626)
              at com.atlassian.bamboo.deletion.DeletionServiceImpl.doDeletePlans(DeletionServiceImpl.java:612)
              at com.atlassian.bamboo.deletion.DeletionServiceImpl.executeDelayedDeletion(DeletionServiceImpl.java:454)
              at sun.reflect.GeneratedMethodAccessor1840.invoke(Unknown Source)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
              at java.lang.reflect.Method.invoke(Method.java:606)
              at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:304)
              at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:182)
              at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:149)
              at com.atlassian.bamboo.security.acegi.intercept.aopalliance.AuthorityOverrideMethodSecurityInterceptor.invoke(AuthorityOverrideMethodSecurityInterceptor.java:29)
              at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
              at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
              at com.sun.proxy.$Proxy109.executeDelayedDeletion(Unknown Source)
              at com.atlassian.bamboo.deletion.DeletionServiceJob.execute(DeletionServiceJob.java:26)
              at org.quartz.core.JobRunShell.run(JobRunShell.java:202)
              at com.atlassian.bamboo.quartz.SystemAuthorizedThreadPool$1.run(SystemAuthorizedThreadPool.java:38)
              at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:525)
      

      Atlassian internal note: Seen on CBAC (5.4-m1 build 4201 - 24 Dec 13)

      Workaround

      Before going through the workaround below, please note that you must perform a valid backup of your Bamboo instance. If you are unsure about how to do so, please refer to our documentation: Exporting data for backup

      After backing up your instance, there are 2 approaches to solving this issue:

      Approach 1
      You can undelete the conflicted Plans by running a SQL query. These plans will then show up in the UI. After that you can proceed to deleting them again using the Bamboo UI. To show these plans again in the UI, run the query below:

      update build set marked_for_deletion = 'false' where build_id in (select master.build_id from build master, build child where master.marked_for_deletion = 'true' and child.marked_for_deletion = 'false' and child.master_id = master.build_id)
      

      Approach 2
      Alternatively of course you can mark the conflicting plan branches as deletable with the following SQL query. We don't really prefer this option of marking things deletable through SQL statements:

      update build set marked_for_deletion = 'true' where build_id in (select child.build_id from build master, build child where master.marked_for_deletion = 'true' and child.marked_for_deletion = 'false' and child.master_id = master.build_id)
      

      Side note: different DB servers might require different queries
      Note that the above queries are for PostgreSQL and need to be adapted to MySQL. For example, this is the latest query that should be run in a MySQL DB:

      update build 
      set marked_for_deletion=true 
      where build_id in (select child.build_id 
      from (select * from build) as master, (select * from build) as child 
      where master.marked_for_deletion=true and child.marked_for_deletion=false and child.master_id=master.build_id);

      Attachments

        Issue Links

          Activity

            People

              mgardias Marcin Gardias
              mmeinhold Martin Meinhold
              Votes:
              4 Vote for this issue
              Watchers:
              17 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: