Uploaded image for project: 'Bitbucket Data Center'
  1. Bitbucket Data Center
  2. BSERV-8324

Application fails to start after upgrade when using Oracle

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Highest Highest
    • 4.2.2, 4.3.1
    • 4.2.0, 4.2.1, 4.3.0
    • Enterprise

      Summary

      Upgrading from any older version of Stash or Bitbucket Server 4.0 to Bitbucket Server 4.2+ when running against Oracle causes the application to enter an unstable state.

      Environment

      • Any release Stash or Bitbucket Server 4.0
      • Running on Oracle database

      Steps to Reproduce

      This is caused if running the upgrade according to the upgrade guide:

      Expected Results

      Upgrade should finish successfully.

      Actual Results

      The below exception is thrown in the atlassian-bitbucket.log file immediately after the upgrade is completed:

      2016-01-18 11:50:13,499 INFO  [active-objects-init-compatibility-tenant-0]  c.a.a.i.ActiveObjectUpgradeManagerImpl Finished upgrading, model is up to date at version 0
2016-01-18 11:50:13,635 ERROR [active-objects-init-compatibility-tenant-0]  net.java.ao.sql Exception executing SQL update <CREATE INDEX "index_ao_8e6075_pro1711516542" ON "AO_8E6075_PROJECT_MAPPING"("EXTERNAL_ID")>
java.sql.SQLException: ORA-01408: such column list already indexed

	at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:440) ~[ojdbc6-11.2.0.2.0.jar:11.2.0.2.0]
	at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:396) ~[ojdbc6-11.2.0.2.0.jar:11.2.0.2.0]
	at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:837) ~[ojdbc6-11.2.0.2.0.jar:11.2.0.2.0]
	at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:445) ~[ojdbc6-11.2.0.2.0.jar:11.2.0.2.0]
	at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:191) ~[ojdbc6-11.2.0.2.0.jar:11.2.0.2.0]
	at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:523) ~[ojdbc6-11.2.0.2.0.jar:11.2.0.2.0]
	at oracle.jdbc.driver.T4CStatement.doOall8(T4CStatement.java:193) ~[ojdbc6-11.2.0.2.0.jar:11.2.0.2.0]
	at oracle.jdbc.driver.T4CStatement.executeForRows(T4CStatement.java:999) ~[ojdbc6-11.2.0.2.0.jar:11.2.0.2.0]
	at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1315) ~[ojdbc6-11.2.0.2.0.jar:11.2.0.2.0]
	at oracle.jdbc.driver.OracleStatement.executeUpdateInternal(OracleStatement.java:1822) ~[ojdbc6-11.2.0.2.0.jar:11.2.0.2.0]
	at oracle.jdbc.driver.OracleStatement.executeUpdate(OracleStatement.java:1787) ~[ojdbc6-11.2.0.2.0.jar:11.2.0.2.0]
	at oracle.jdbc.driver.OracleStatementWrapper.executeUpdate(OracleStatementWrapper.java:280) ~[ojdbc6-11.2.0.2.0.jar:11.2.0.2.0]
	at com.zaxxer.hikari.proxy.StatementProxy.executeUpdate(StatementProxy.java:119) ~[HikariCP-2.4.1.jar:na]
	at com.zaxxer.hikari.proxy.HikariStatementProxy.executeUpdate(HikariStatementProxy.java) ~[HikariCP-2.4.1.jar:na]
	at net.java.ao.DatabaseProvider.executeUpdate(DatabaseProvider.java:2246) [activeobjects-plugin-1.1.5.jar:na]
	at net.java.ao.DatabaseProvider.executeUpdateForAction(DatabaseProvider.java:2302) [activeobjects-plugin-1.1.5.jar:na]
	at net.java.ao.DatabaseProvider.executeUpdatesForActions(DatabaseProvider.java:2274) [activeobjects-plugin-1.1.5.jar:na]
	at net.java.ao.schema.SchemaGenerator.migrate(SchemaGenerator.java:91) [activeobjects-plugin-1.1.5.jar:na]
	at net.java.ao.EntityManager.migrate(EntityManager.java:128) [activeobjects-plugin-1.1.5.jar:na]
	at com.atlassian.activeobjects.internal.EntityManagedActiveObjects.migrate(EntityManagedActiveObjects.java:45) [activeobjects-plugin-1.1.5.jar:na]
	at com.atlassian.activeobjects.internal.AbstractActiveObjectsFactory$1.doInTransaction(AbstractActiveObjectsFactory.java:77) [activeobjects-plugin-1.1.5.jar:na]
	at com.atlassian.activeobjects.internal.AbstractActiveObjectsFactory$1.doInTransaction(AbstractActiveObjectsFactory.java:72) [activeobjects-plugin-1.1.5.jar:na]
	at com.atlassian.sal.core.transaction.HostContextTransactionTemplate$1.doInTransaction(HostContextTransactionTemplate.java:21) [sal-core-3.0.5.jar:na]
	at com.atlassian.stash.internal.sal.spi.HostContextAccessorImpl.doInTransaction(HostContextAccessorImpl.java:54) [bitbucket-platform-4.2.1.jar:na]
	at com.atlassian.plugin.util.ContextClassLoaderSettingInvocationHandler.invoke(ContextClassLoaderSettingInvocationHandler.java:30) [atlassian-plugins-core-4.0.4.jar:na]
	at org.eclipse.gemini.blueprint.service.importer.support.internal.aop.ServiceInvoker.doInvoke(ServiceInvoker.java:56) [gemini-blueprint-core-2.0.0.BUILD-atlassian-m002.jar:2.0.0.BUILD-atlassian-m002]
	at org.eclipse.gemini.blueprint.service.importer.support.internal.aop.ServiceInvoker.invoke(ServiceInvoker.java:60) [gemini-blueprint-core-2.0.0.BUILD-atlassian-m002.jar:2.0.0.BUILD-atlassian-m002]
	at org.eclipse.gemini.blueprint.service.util.internal.aop.ServiceTCCLInterceptor.invokeUnprivileged(ServiceTCCLInterceptor.java:70) [gemini-blueprint-core-2.0.0.BUILD-atlassian-m002.jar:2.0.0.BUILD-atlassian-m002]
	at org.eclipse.gemini.blueprint.service.util.internal.aop.ServiceTCCLInterceptor.invoke(ServiceTCCLInterceptor.java:53) [gemini-blueprint-core-2.0.0.BUILD-atlassian-m002.jar:2.0.0.BUILD-atlassian-m002]
	at org.eclipse.gemini.blueprint.service.importer.support.LocalBundleContextAdvice.invoke(LocalBundleContextAdvice.java:57) [gemini-blueprint-core-2.0.0.BUILD-atlassian-m002.jar:2.0.0.BUILD-atlassian-m002]
	at com.atlassian.sal.core.transaction.HostContextTransactionTemplate.execute(HostContextTransactionTemplate.java:18) [sal-core-3.0.5.jar:na]
	at org.eclipse.gemini.blueprint.service.importer.support.internal.aop.ServiceInvoker.doInvoke(ServiceInvoker.java:56) [gemini-blueprint-core-2.0.0.BUILD-atlassian-m002.jar:2.0.0.BUILD-atlassian-m002]
	at org.eclipse.gemini.blueprint.service.importer.support.internal.aop.ServiceInvoker.invoke(ServiceInvoker.java:60) [gemini-blueprint-core-2.0.0.BUILD-atlassian-m002.jar:2.0.0.BUILD-atlassian-m002]
	at org.eclipse.gemini.blueprint.service.util.internal.aop.ServiceTCCLInterceptor.invokeUnprivileged(ServiceTCCLInterceptor.java:70) [gemini-blueprint-core-2.0.0.BUILD-atlassian-m002.jar:2.0.0.BUILD-atlassian-m002]
	at org.eclipse.gemini.blueprint.service.util.internal.aop.ServiceTCCLInterceptor.invoke(ServiceTCCLInterceptor.java:53) [gemini-blueprint-core-2.0.0.BUILD-atlassian-m002.jar:2.0.0.BUILD-atlassian-m002]
	at org.eclipse.gemini.blueprint.service.importer.support.LocalBundleContextAdvice.invoke(LocalBundleContextAdvice.java:57) [gemini-blueprint-core-2.0.0.BUILD-atlassian-m002.jar:2.0.0.BUILD-atlassian-m002]
	at com.atlassian.activeobjects.internal.AbstractActiveObjectsFactory.create(AbstractActiveObjectsFactory.java:72) [activeobjects-plugin-1.1.5.jar:na]
	at com.atlassian.activeobjects.internal.DelegatingActiveObjectsFactory.create(DelegatingActiveObjectsFactory.java:32) [activeobjects-plugin-1.1.5.jar:na]
	at com.atlassian.activeobjects.osgi.TenantAwareActiveObjects$1$1$1.call(TenantAwareActiveObjects.java:91) [activeobjects-plugin-1.1.5.jar:na]
	at com.atlassian.activeobjects.osgi.TenantAwareActiveObjects$1$1$1.call(TenantAwareActiveObjects.java:86) [activeobjects-plugin-1.1.5.jar:na]
	at com.atlassian.sal.core.executor.ThreadLocalDelegateCallable.call(ThreadLocalDelegateCallable.java:38) [sal-core-3.0.5.jar:na]
	at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_45]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_45]
	at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
	... 47 frames trimmed

      Notes

      ActiveObjects has a bug where if entity column getters are annotated with @Indexed and either @PrimaryKey or @Unique under Oracle it will try to create the additional index (for @Indexed) even though Oracle will have already created indexes for the primary key and unique columns when the table is created. Oracle allows only one index per column. This causes the mirroring plugin to fail to start due to errors from Oracle during migration and unfortunately causes the application to enter an unstable state. The logs will report: "java.sql.SQLException: ORA-01408: such column list already indexed"

      The fix is to remove the redundant @Indexed annotations from the offending classes.

      Solution

      Upgrade to Bitbucket Server 4.2.2, Bitbucket Server 4.3.1, or higher.

            [BSERV-8324] Application fails to start after upgrade when using Oracle

            Shyju kumar added a comment - - edited

            1. Simple solution is to disable Hipchat plugin& Hipchat Integration Plugin from admin. Re-Index..
            2. if 1 doesn't work then >Stop Jira>go to DB , Index drop IDX_USER_ATTR_DIR_NAME_LVAL >Restart>Check in catalina.out for errors on boot

            Hipchat doesnt looks compatible to RDS Oracle 12c or Oracle 11g.

            Regards
            shyju

            Shyju kumar added a comment - - edited 1. Simple solution is to disable Hipchat plugin& Hipchat Integration Plugin from admin. Re-Index.. 2. if 1 doesn't work then >Stop Jira >go to DB , Index drop IDX_USER_ATTR_DIR_NAME_LVAL >Restart >Check in catalina.out for errors on boot Hipchat doesnt looks compatible to RDS Oracle 12c or Oracle 11g. Regards shyju

            Gabe Yoachum added a comment -

            What is the impact of this? Aside from noticing the error in the logs, we are not sure what this causes. I asked because we are trying to figure out if we can wait until our next patching cycle in 90 days.

            Gabe Yoachum added a comment - What is the impact of this? Aside from noticing the error in the logs, we are not sure what this causes. I asked because we are trying to figure out if we can wait until our next patching cycle in 90 days.

            Victor Guevara Masis added a comment -

            We could replicate the issue only after a backup using the Backup Tool 2.0.1. If using the version 1.9.1, the issue does not appear.

            Victor Guevara Masis added a comment - We could replicate the issue only after a backup using the Backup Tool 2.0.1. If using the version 1.9.1, the issue does not appear.

              mstudman Michael Studman (Inactive)
              mstudman Michael Studman (Inactive)
              Affected customers:
              2 This affects my team
              Watchers:
              12 Start watching this issue

                Created:
                Updated:
                Resolved: