Summary

• Upgrading Bamboo v5.15.5 to v6.0.3 brings MySQL server down

Environment

• Bamboo v5.15.5 (source)
• Bamboo v6.0.3 (aiming upgrade version)
• MySQL v5.6.29

  mysql> show variables like "%max_connection%";
  +-----------------+-------+
  | Variable_name   | Value |
  +-----------------+-------+
  | max_connections | 500   |
  +-----------------+-------+
  1 row in set (0.00 sec)
  

Steps to Reproduce

1. stop Bamboo v5.15.5
2. copy <bamboo-home-v5.15.5> onto a different location (e.g. <bamboo-home-6.0.3>)
3. create a database dump of v5.15.5

   mysqldump -u root bamboo5155 > bambooupgrade.sql
   

1. create a new database and and restore dump

   mysql> create database `bamboo603` CHARACTER SET utf8 COLLATE utf8_bin;
   

   mysql -u root `bamboo603` < bambooupgrade.sql
   

1. update <bamboo-home-6.0.3>/bamboo.cfg.xml to point to the database from above

   <property name="hibernate.connection.url">jdbc:mysql://localhost/bamboo603?autoReconnect=true</property>
   

1. start Bamboo v6.0.3

Expected Results

• Upgrade should run smoothly and Bamboo should be up and running

Actual Results

• MySQL server stops communicating and Bamboo fails with the following in the <bamboo-install>/logs/catalina.out

  2017-06-12 17:58:38,924 INFO [localhost-startStop-1] [BambooContainer] Bamboo version 6.0.3 initialised OK
  2017-06-12 17:58:38,925 INFO [AtlassianEvent::0-BAM::EVENTS:pool-1-thread-7] [lifecycle] Starting FeatureManagerStartupListener
  2017-06-12 17:58:38,927 INFO [AtlassianEvent::0-BAM::EVENTS:pool-1-thread-9] [LogLicenseDetailsServerStartedListener] Bamboo license SEN: SEN-L5082204
  2017-06-12 17:58:38,930 INFO [AtlassianEvent::0-BAM::EVENTS:pool-1-thread-7] [lifecycle] *******************************
  2017-06-12 17:58:38,930 INFO [AtlassianEvent::0-BAM::EVENTS:pool-1-thread-7] [lifecycle] * Application links configured*
  2017-06-12 17:58:38,936 INFO [AtlassianEvent::0-BAM::EVENTS:pool-1-thread-7] [lifecycle] * Application Link. Id:'e01fbf8d-8053-3b15-9a76-3343c9147ac5', Name: '[Bitbucket 4.14.1]', Url:'http://git.webserver, Type: 'applinks.bitbucket'
  2017-06-12 17:58:38,936 INFO [AtlassianEvent::0-BAM::EVENTS:pool-1-thread-7] [lifecycle] ********************************
  2017-06-12 17:58:38,941 WARN [scheduler_Worker-8] [OrphanedBuildResultRemovalJob] Deletion disabled by a command line switch.
  2017-06-12 17:58:38,944 INFO [scheduler_Worker-9] [PlanVcsRevisionHistoryCleanupScheduler] Starting Plan VCS Revision History Cleanup
  2017-06-12 17:58:38,954 INFO [localhost-startStop-1] [CompressingFilter] CompressingFilter has initialized
  2017-06-12 17:58:39,845 INFO [localhost-startStop-1] [SpringObjectFactory] Setting autowire strategy to name
  2017-06-12 17:58:39,847 INFO [localhost-startStop-1] [StrutsSpringObjectFactory] Struts-Spring integration initialised successfully
  2017-06-12 17:58:40,507 INFO [localhost-startStop-1] [Dispatchers] Dispatcher org.apache.struts2.dispatcher.Dispatcher initialised
  2017-06-12 17:58:40,549 INFO [localhost-startStop-1] [SpringObjectFactory] Setting autowire strategy to name
  2017-06-12 17:58:40,550 INFO [localhost-startStop-1] [StrutsSpringObjectFactory] Struts-Spring integration initialised successfully
  2017-06-12 17:58:41,447 INFO [localhost-startStop-1] [AccessLogFilter] AccessLogFilter initialized. Format is: <user> <url> <starting memory free (kb)> +- <difference in free mem (kb)> <query time (ms)>
  2017-06-12 17:58:41,489 INFO [localhost-startStop-1] [SpringObjectFactory] Setting autowire strategy to name
  2017-06-12 17:58:41,489 INFO [localhost-startStop-1] [StrutsSpringObjectFactory] Struts-Spring integration initialised successfully
  2017-06-12 17:58:41,627 INFO [localhost-startStop-1] [Dispatchers] Dispatcher org.apache.struts2.dispatcher.Dispatcher initialised
  2017-06-12 17:58:41,663 INFO [localhost-startStop-1] [SpringObjectFactory] Setting autowire strategy to name
  2017-06-12 17:58:41,663 INFO [localhost-startStop-1] [StrutsSpringObjectFactory] Struts-Spring integration initialised successfully
  12-Jun-2017 17:58:41.809 INFO [main] org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler ["http-nio-8085"]
  12-Jun-2017 17:58:41.816 INFO [main] org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler ["http-bio-8086"]
  12-Jun-2017 17:58:41.817 INFO [main] org.apache.catalina.startup.Catalina.start Server startup in 219541 ms
  2017-06-12 17:58:42,000 INFO [http-nio-8085-exec-1] [AccessLogFilter] 127.0.0.1 GET http://bamboo.webserver/ 410955kb
  2017-06-12 17:59:00,944 INFO [NavLink RestCapabilitiesClient:thread-1] [RestCapabilitiesClient] Failed to request capabilities from 'http://git.webserver/rest/capabilities': Service Unavailable
  12-Jun-2017 17:59:51.844 WARNING [ContainerBackgroundProcessor[StandardEngine[Catalina]]] org.apache.catalina.valves.StuckThreadDetectionValve.notifyStuckThreadDetected Thread "http-nio-8085-exec-1" (id=183) has been active for 69,979 milliseconds (since 6/12/17 5:58 PM) to serve the same request for http://bamboo.webserver/ and may be stuck (configured threshold for this StuckThreadDetectionValve is 60 seconds). There is/are 1 thread(s) in total that are monitored by this Valve and may be stuck.
   java.lang.Throwable
  	at java.lang.Object.wait(Native Method)
  	at com.mchange.v2.resourcepool.BasicResourcePool.awaitAvailable(BasicResourcePool.java:1465)
  	at com.mchange.v2.resourcepool.BasicResourcePool.prelimCheckoutResource(BasicResourcePool.java:644)
  	at com.mchange.v2.resourcepool.BasicResourcePool.checkoutResource(BasicResourcePool.java:554)
  	at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool.checkoutAndMarkConnectionInUse(C3P0PooledConnectionPool.java:758)
  	at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool.checkoutPooledConnection(C3P0PooledConnectionPool.java:685)
  	at com.mchange.v2.c3p0.impl.AbstractPoolBackedDataSource.getConnection(AbstractPoolBackedDataSource.java:140)
  	at org.hibernate.c3p0.internal.C3P0ConnectionProvider.getConnection(C3P0ConnectionProvider.java:73)
  	at org.hibernate.internal.AbstractSessionImpl$NonContextualJdbcConnectionAccess.obtainConnection(AbstractSessionImpl.java:386)
  	at org.hibernate.resource.jdbc.internal.LogicalConnectionManagedImpl.acquireConnectionIfNeeded(LogicalConnectionManagedImpl.java:87)
  	at org.hibernate.resource.jdbc.internal.LogicalConnectionManagedImpl.getPhysicalConnection(LogicalConnectionManagedImpl.java:112)
  	at org.hibernate.internal.SessionImpl.connection(SessionImpl.java:489)
  	at org.springframework.orm.hibernate5.HibernateTransactionManager.doBegin(HibernateTransactionManager.java:446)
  	at org.springframework.transaction.support.AbstractPlatformTransactionManager.getTransaction(AbstractPlatformTransactionManager.java:373)
  	at sun.reflect.GeneratedMethodAccessor161.invoke(Unknown Source)
  	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  	at java.lang.reflect.Method.invoke(Method.java:497)
  	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:302)
  	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
  	at com.sun.proxy.$Proxy64.getTransaction(Unknown Source)
  	at org.springframework.transaction.interceptor.TransactionAspectSupport.createTransactionIfNecessary(TransactionAspectSupport.java:427)
  	at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:276)
  	at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96)
  	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
  	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:208)
  	at com.sun.proxy.$Proxy66.readAclsById(Unknown Source)
  	at com.atlassian.bamboo.security.BambooCachingPermissionManagerFacadeImpl$BambooCachingPermissionManager.primeAclCache(BambooCachingPermissionManagerFacadeImpl.java:61)
  	at com.atlassian.bamboo.security.BambooCachingPermissionManagerFacadeImpl.primeAclCache(BambooCachingPermissionManagerFacadeImpl.java:89)
  	at com.atlassian.bamboo.webwork.StarterAction.getPlans(StarterAction.java:229)
  	at com.atlassian.bamboo.webwork.StarterAction.start(StarterAction.java:143)
  	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  ...