Uploaded image for project: 'Confluence Data Center'
  1. Confluence Data Center
  2. CONFSERVER-40064

indexing stops working completely after intermittent connection issue

    • Icon: Bug Bug
    • Resolution: Duplicate
    • Icon: Medium Medium
    • None
    • 5.8.4, 5.9.1, 5.8.10
    • None

      Steps to reproduce :

      1. Install Confluence 5.8 with MySQL server
      2. Login to Confluence
      3. Shut down MySQL server
      4. Open the atlassian-confluence.log with a text editor which can be found in <confluence_home_directory>/logs and take a look, you will see the following error :
        2015-11-27 16:19:20,002 ERROR [scheduler_Worker-3] [sf.hibernate.util.JDBCExceptionReporter] logExceptions Communications link failure
        
        The last packet successfully received from the server was 19,981 milliseconds ago.  The last packet sent successfully to the server was 0 milliseconds ago.
        2015-11-27 16:19:24,539 ERROR [scheduler_Worker-3] [org.quartz.core.ErrorLogger] schedulerError Unable to notify JobListener(s) of Job to be executed: (Job will NOT be executed!). trigger= DEFAULT.ConfluenceDavSessionInvalidatorJobDetail job= DEFAULT.ConfluenceDavSessionInvalidatorJobDetail
        org.quartz.SchedulerException: JobListener 'ScheduledJobListener' threw exception: Could not open Hibernate Session for transaction; nested exception is net.sf.hibernate.exception.GenericJDBCException: Cannot open connection [See nested exception: org.springframework.transaction.CannotCreateTransactionException: Could not open Hibernate Session for transaction; nested exception is net.sf.hibernate.exception.GenericJDBCException: Cannot open connection]
        	at org.quartz.core.QuartzScheduler.notifyJobListenersToBeExecuted(QuartzScheduler.java:1908)
        	at org.quartz.core.JobRunShell.notifyListenersBeginning(JobRunShell.java:335)
        	at org.quartz.core.JobRunShell.run(JobRunShell.java:173)
        	at com.atlassian.confluence.schedule.quartz.ConfluenceQuartzThreadPool$1.run(ConfluenceQuartzThreadPool.java:20)
        	at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:549)
        Caused by: org.springframework.transaction.CannotCreateTransactionException: Could not open Hibernate Session for transaction; nested exception is net.sf.hibernate.exception.GenericJDBCException: Cannot open connection
        	at org.springframework.orm.hibernate.HibernateTransactionManager.doBegin(HibernateTransactionManager.java:473)
        	at org.springframework.transaction.support.AbstractPlatformTransactionManager.getTransaction(AbstractPlatformTransactionManager.java:374)
        	at org.springframework.transaction.interceptor.TransactionAspectSupport.createTransactionIfNecessary(TransactionAspectSupport.java:263)
        	at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:101)
        	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
        	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
        	at com.sun.proxy.$Proxy99.jobToBeExecuted(Unknown Source)
        	at org.quartz.core.QuartzScheduler.notifyJobListenersToBeExecuted(QuartzScheduler.java:1906)
        	... 4 more
        Caused by: net.sf.hibernate.exception.GenericJDBCException: Cannot open connection
        	at net.sf.hibernate.exception.ErrorCodeConverter.handledNonSpecificException(ErrorCodeConverter.java:90)
        	at net.sf.hibernate.exception.ErrorCodeConverter.convert(ErrorCodeConverter.java:79)
        	at net.sf.hibernate.exception.JDBCExceptionHelper.convert(JDBCExceptionHelper.java:29)
        	at net.sf.hibernate.impl.BatcherImpl.convert(BatcherImpl.java:328)
        	at net.sf.hibernate.impl.BatcherImpl.openConnection(BatcherImpl.java:295)
        	at net.sf.hibernate.impl.SessionImpl.connect(SessionImpl.java:3388)
        	at net.sf.hibernate.impl.SessionImpl.connection(SessionImpl.java:3348)
        	at org.springframework.orm.hibernate.HibernateTransactionManager.doBegin(HibernateTransactionManager.java:422)
        	... 11 more
        Caused by: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
        
        The last packet successfully received from the server was 19,981 milliseconds ago.  The last packet sent successfully to the server was 0 milliseconds ago.
        	at sun.reflect.GeneratedConstructorAccessor444.newInstance(Unknown Source)
        	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
        	at com.mysql.jdbc.Util.handleNewInstance(Util.java:400)
        	at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:1038)
        	at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:3434)
        	at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:3334)
        	at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3774)
        	at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2447)
        	at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2594)
        	at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2541)
        	at com.mysql.jdbc.ConnectionImpl.setTransactionIsolation(ConnectionImpl.java:5167)
        	at com.mchange.v2.c3p0.impl.NewProxyConnection.setTransactionIsolation(NewProxyConnection.java:701)
        	at net.sf.hibernate.connection.C3P0ConnectionProvider.getConnection(C3P0ConnectionProvider.java:36)
        	at net.sf.hibernate.impl.BatcherImpl.openConnection(BatcherImpl.java:292)
        	... 14 more
        Caused by: java.net.SocketException: Software caused connection abort: recv failed
        	at java.net.SocketInputStream.socketRead0(Native Method)
        	at java.net.SocketInputStream.read(SocketInputStream.java:152)
        	at java.net.SocketInputStream.read(SocketInputStream.java:122)
        	at com.mysql.jdbc.util.ReadAheadInputStream.fill(ReadAheadInputStream.java:100)
        	at com.mysql.jdbc.util.ReadAheadInputStream.readFromUnderlyingStreamIfNecessary(ReadAheadInputStream.java:143)
        	at com.mysql.jdbc.util.ReadAheadInputStream.read(ReadAheadInputStream.java:173)
        	at com.mysql.jdbc.MysqlIO.readFully(MysqlIO.java:2923)
        	at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:3344)
      5. Start MySQL database server again
      6. Try and create a page named "testing unique result"
      7. Type the string "testing unique result" in the search bar and you will see the following page with no result :

      Expected result : The search should work as normal for subsequent creation of new pages after encountering intermittent connection issue. The index scheduler should continue executing as usual once Confluence is able to communicate with the MySQL database server. This expected behavior can be observed in Confluence 5.4 and 5.5 (have not tested other version yet)

      Current result : the index scheduler completely stop working in Confluence unless it is being restarted and search is not workable for all newly created Space and Pages. This includes the "All Updates" tab not being updated in the Confluence dashboard.

      Note : This issue affects Oracle and MS SQL server as well. It will potentially affects other database types as well. You can replicate this issue on a completely new Confluence instance.

            [CONFSERVER-40064] indexing stops working completely after intermittent connection issue

            Minh Tran added a comment -

            Dear All,

            This ticket has been marked as duplicate. Please follow the progress here https://jira.atlassian.com/browse/CONF-39825

            Thanks,
            Minh Tran
            Confluence BugMaster
            Atlassian

            Minh Tran added a comment - Dear All, This ticket has been marked as duplicate. Please follow the progress here https://jira.atlassian.com/browse/CONF-39825 Thanks, Minh Tran Confluence BugMaster Atlassian

            Please note that this is not MySQL specific.

            We use Microsoft SQL Server and this is one of two show-stopper indexer bugs preventing us upgrading from 5.4 to 5.8 (this bug was raised from the very long running issue CSP-157392).

            Peter Binney added a comment - Please note that this is not MySQL specific. We use Microsoft SQL Server and this is one of two show-stopper indexer bugs preventing us upgrading from 5.4 to 5.8 (this bug was raised from the very long running issue CSP-157392).

              Unassigned Unassigned
              dooi Der Lun
              Affected customers:
              5 This affects my team
              Watchers:
              15 Start watching this issue

                Created:
                Updated:
                Resolved: