Uploaded image for project: 'Confluence Cloud'
  1. Confluence Cloud
  2. CONFCLOUD-40012

Automatic recovery of Quartz scheduler after commit error

    XMLWordPrintable

Details

    • Our product teams collect and evaluate feedback from a number of different sources. To learn more about how we use customer feedback in the planning process, check out our new feature policy.

    Description

      NOTE: This suggestion is for Confluence Cloud. Using Confluence Server? See the corresponding suggestion.

      Problem Definition

      Quartz is unable to recover from commit error (any - either transaction deadlock or WSREP cluster state failure) like this:

      2015-11-13 06:33:12,988 ERROR [scheduler_Worker-1] [org.quartz.core.ErrorLogger] schedulerError Unable to notify JobListener(s) of Job to be executed: (Job will NOT be executed!). trigger= DEFAULT.IndexQueu
      eFlusher job= DEFAULT.IndexQueueFlusher
      org.quartz.SchedulerException: JobListener 'ScheduledJobListener' threw exception: Could not commit Hibernate transaction; nested exception is net.sf.hibernate.TransactionException: Commit failed with SQL e
      xception: [See nested exception: org.springframework.transaction.TransactionSystemException: Could not commit Hibernate transaction; nested exception is net.sf.hibernate.TransactionException: Commit failed
      with SQL exception: ]
      at org.quartz.core.QuartzScheduler.notifyJobListenersToBeExecuted(QuartzScheduler.java:1951)
      at org.quartz.core.JobRunShell.notifyListenersBeginning(JobRunShell.java:364)
      at org.quartz.core.JobRunShell.run(JobRunShell.java:190)
      at com.atlassian.confluence.schedule.quartz.ConfluenceQuartzThreadPool.lambda$runInThread$46(ConfluenceQuartzThreadPool.java:19)
      at com.atlassian.confluence.schedule.quartz.ConfluenceQuartzThreadPool$$Lambda$98/1862956413.run(Unknown Source)
      at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:549)
      Caused by: org.springframework.transaction.TransactionSystemException: Could not commit Hibernate transaction; nested exception is net.sf.hibernate.TransactionException: Commit failed with SQL exception:
      at org.springframework.orm.hibernate.HibernateTransactionManager.doCommit(HibernateTransactionManager.java:514)
      at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:732)
      at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:701)
      at sun.reflect.GeneratedMethodAccessor55.invoke(Unknown Source)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      at java.lang.reflect.Method.invoke(Method.java:497)
      at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:307)
      at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:198)
      at com.sun.proxy.$Proxy13.commit(Unknown Source)
      at org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:321)
      at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:116)
      at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
      at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
      at com.sun.proxy.$Proxy114.jobToBeExecuted(Unknown Source)
      at org.quartz.core.QuartzScheduler.notifyJobListenersToBeExecuted(QuartzScheduler.java:1949)
      ... 5 more
      

      After being 'unable to notify job listeners' it stops scheduling DEFAULT.IndexQueueFlusher task completely.

      Suggested Solution

      It would be good if it will be possible to set up the scheduler to restart automatically or manually restart the job from Scheduled Jobs page.

      Workaround

      Flush the index queue manually.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              ashaleev Anton Shaleev
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: