Uploaded image for project: 'Bitbucket Server'
  1. Bitbucket Server
  2. BSERV-11531

Statistics about Webhook events can fail to update on Oracle databases

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed (View Workflow)
    • Priority: Medium
    • Resolution: Fixed
    • Affects Version/s: 5.8.0
    • Fix Version/s: 6.3.0
    • Component/s: Webhooks
    • Labels:

      Description

      After Webhook events are sent there are some statistics recorded on the database to store the number of successful calls, as well as last success and last failure times.

      Due to this bug https://ecosystem.atlassian.net/browse/AO-3395 the statistics may fail to update on Oracle databases

      A stacktrace like the one below is logged in the application logs:

      WARN  [threadpool:thread-15]  c.a.b.i.w.d.AsyncInvocationHistoryDao Failed to write webhook invocation data for <event> to the database
      com.atlassian.activeobjects.internal.ActiveObjectsSqlException: There was a SQL exception thrown by the Active Objects library:
      Database:
              - name:Oracle
              - version:Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
      With the Partitioning, OLAP, Advanced Analytics and Real Application Testing options
              - minor version:1
              - major version:12
      Driver:
              - name:Oracle JDBC driver
              - version:12.1.0.1.0
      
      java.sql.SQLException: ORA-24816: Expanded non LONG bind data supplied after actual LONG or LOB column
      
              at com.atlassian.activeobjects.internal.EntityManagedActiveObjects.create(EntityManagedActiveObjects.java:102)
              at com.atlassian.activeobjects.osgi.TenantAwareActiveObjects.create(TenantAwareActiveObjects.java:271)
              at org.eclipse.gemini.blueprint.service.importer.support.internal.aop.ServiceInvoker.doInvoke(ServiceInvoker.java:56)
              at org.eclipse.gemini.blueprint.service.importer.support.internal.aop.ServiceInvoker.invoke(ServiceInvoker.java:60)
              at org.eclipse.gemini.blueprint.service.util.internal.aop.ServiceTCCLInterceptor.invokeUnprivileged(ServiceTCCLInterceptor.java:70)
              at org.eclipse.gemini.blueprint.service.util.internal.aop.ServiceTCCLInterceptor.invoke(ServiceTCCLInterceptor.java:53)
              at org.eclipse.gemini.blueprint.service.importer.support.LocalBundleContextAdvice.invoke(LocalBundleContextAdvice.java:57)
              at com.atlassian.bitbucket.internal.webhook.dao.AoInvocationHistoryDao.saveInvocation(AoInvocationHistoryDao.java:285)
              at com.atlassian.bitbucket.internal.webhook.dao.AsyncInvocationHistoryDao.maybeFlush(AsyncInvocationHistoryDao.java:233)
              at com.atlassian.bitbucket.internal.webhook.dao.AsyncInvocationHistoryDao.flushPending(AsyncInvocationHistoryDao.java:218)
              at com.atlassian.bitbucket.internal.webhook.dao.AsyncInvocationHistoryDao.flush(AsyncInvocationHistoryDao.java:179)
              at com.atlassian.bitbucket.internal.webhook.dao.AsyncInvocationHistoryDao.flushAndReschedule(AsyncInvocationHistoryDao.java:208)
              at com.atlassian.stash.internal.concurrent.DefaultTransferableStateManager$StateTransferringRunnable.run(DefaultTransferableStateManager.java:166)
              at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
              at java.util.concurrent.FutureTask.run(FutureTask.java:266)
              at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
              at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
              at java.lang.Thread.run(Thread.java:748)
              ... 16 frames trimmed
      Caused by: java.sql.SQLException: ORA-24816: Expanded non LONG bind data supplied after actual LONG or LOB column
      
              at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:450)
              at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:399)
              at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:1017)
              at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:655)
              at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:249)
              at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:566)
              at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:215)
              at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:58)
              at oracle.jdbc.driver.T4CPreparedStatement.executeForRows(T4CPreparedStatement.java:943)
              at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1075)
              at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3820)
              at oracle.jdbc.driver.OraclePreparedStatement.executeUpdate(OraclePreparedStatement.java:3897)
              at oracle.jdbc.driver.OraclePreparedStatementWrapper.executeUpdate(OraclePreparedStatementWrapper.java:1361)
              at com.zaxxer.hikari.pool.ProxyPreparedStatement.executeUpdate(ProxyPreparedStatement.java:61)
              at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.executeUpdate(HikariProxyPreparedStatement.java)
              at net.java.ao.db.OracleDatabaseProvider.executeInsertReturningKey(OracleDatabaseProvider.java:297)
              at net.java.ao.DatabaseProvider.insertReturningKey(DatabaseProvider.java:1861)
              at net.java.ao.EntityManager.create(EntityManager.java:366)
              at net.java.ao.EntityManager.create(EntityManager.java:399)
              at com.atlassian.activeobjects.internal.EntityManagedActiveObjects.create(EntityManagedActiveObjects.java:100)
              ... 19 common frames omitted
      

      Webhook notifications should still be sent, even though the statistics are not updated correctly.

      This problem can occur when using version 12.1.0 of the Oracle jdbc driver.

      Workaround

      Upgrading to version 12.2.0 of the Oracle jdbc driver seems to resolve the problem.

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              fdoherty@atlassian.com Frank Doherty
              Reporter:
              fdoherty@atlassian.com Frank Doherty
              Votes:
              6 Vote for this issue
              Watchers:
              10 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: