Uploaded image for project: 'Bamboo Data Center'
  1. Bamboo Data Center
  2. BAM-16291

Elastic agent stuck in Pending state with Java6

    XMLWordPrintable

Details

    Description

      Summary

      After the weekly release of 30-Aug-2015 some Custom AMIs aren't able to start properly. They will remain pending and will eventually be shut down.

      Environment

      • Bamboo Cloud
      • Custom AMI
      • Using JDK6

      Steps to Reproduce

      1. Use a custom AMI
      2. Make sure the JAVA_HOME is pointed to JDK6
      3. Save the custom AMI and try to start up the image from Bamboo

      Expected Results

      EC2 instance and agent start up normally.

      Actual Results

      The Agent fails to start up and throws the below exception in the agent.out file:

      2015-08-31 08:15:34,843 FATAL [Thread-1] [AgentBootstrap] Exiting due to fatal exception.
      java.lang.reflect.InvocationTargetException
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
      	at java.lang.reflect.Method.invoke(Method.java:597)
      	at com.atlassian.bamboo.agent.bootstrap.AgentRunner.run(AgentRunner.java:28)
      	at java.lang.Thread.run(Thread.java:662)
      Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'buildAgentController' defined in class path resource [applicationContextRemoteAgent.xml]: Cannot resolve reference to bean 'agentQueueAccessor' while setting constructor argument; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'agentQueueAccessor' defined in class path resource [applicationContextRemoteAgentJMS.xml]: Cannot resolve reference to bean 'connectionFactory' while setting bean property 'connectionFactory'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'connectionFactory' defined in class path resource [applicationContextRemoteAgentJMS.xml]: Cannot create inner bean 'com.atlassian.bamboo.v2.build.agent.BambooActiveMQConnectionFactory#2cf5d030' of type [com.atlassian.bamboo.v2.build.agent.BambooActiveMQConnectionFactory] while setting constructor argument; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'com.atlassian.bamboo.v2.build.agent.BambooActiveMQConnectionFactory#2cf5d030' defined in class path resource [applicationContextRemoteAgentJMS.xml]: Instantiation of bean failed; nested exception is org.springframework.beans.BeanInstantiationException: Could not instantiate bean class [com.atlassian.bamboo.v2.build.agent.BambooActiveMQConnectionFactory]: Constructor threw exception; nested exception is java.lang.IncompatibleClassChangeError: Class java.net.Socket does not implement the requested interface java.io.Closeable
      	at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:336)
      	at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveValueIfNecessary(BeanDefinitionValueResolver.java:108)
      	at org.springframework.beans.factory.support.ConstructorResolver.resolveConstructorArguments(ConstructorResolver.java:646)
      	at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:140)
      	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:1114)
      	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1017)
      	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:504)
      	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:475)
      	at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:302)
      	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:228)
      	at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:298)
      	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:193)
      	at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:703)
      	at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:760)
      	at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:482)
      	at org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:139)
      	at org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:105)
      	at com.atlassian.bamboo.v2.build.agent.remote.RemoteAgent.initApplicationContext(RemoteAgent.java:227)
      	at com.atlassian.bamboo.v2.build.agent.remote.RemoteAgent.start(RemoteAgent.java:70)
      	at com.atlassian.bamboo.agent.elastic.client.ElasticAgent.start(ElasticAgent.java:31)
      	... 6 more
      Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'agentQueueAccessor' defined in class path resource [applicationContextRemoteAgentJMS.xml]: Cannot resolve reference to bean 'connectionFactory' while setting bean property 'connectionFactory'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'connectionFactory' defined in class path resource [applicationContextRemoteAgentJMS.xml]: Cannot create inner bean 'com.atlassian.bamboo.v2.build.agent.BambooActiveMQConnectionFactory#2cf5d030' of type [com.atlassian.bamboo.v2.build.agent.BambooActiveMQConnectionFactory] while setting constructor argument; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'com.atlassian.bamboo.v2.build.agent.BambooActiveMQConnectionFactory#2cf5d030' defined in class path resource [applicationContextRemoteAgentJMS.xml]: Instantiation of bean failed; nested exception is org.springframework.beans.BeanInstantiationException: Could not instantiate bean class [com.atlassian.bamboo.v2.build.agent.BambooActiveMQConnectionFactory]: Constructor threw exception; nested exception is java.lang.IncompatibleClassChangeError: Class java.net.Socket does not implement the requested interface java.io.Closeable
      	at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:336)
      	at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveValueIfNecessary(BeanDefinitionValueResolver.java:108)
      	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyPropertyValues(AbstractAutowireCapableBeanFactory.java:1456)
      	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1197)
      	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:537)
      	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:475)
      	at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:302)
      	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:228)
      	at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:298)
      	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:193)
      	at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:328)
      	... 25 more
      Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'connectionFactory' defined in class path resource [applicationContextRemoteAgentJMS.xml]: Cannot create inner bean 'com.atlassian.bamboo.v2.build.agent.BambooActiveMQConnectionFactory#2cf5d030' of type [com.atlassian.bamboo.v2.build.agent.BambooActiveMQConnectionFactory] while setting constructor argument; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'com.atlassian.bamboo.v2.build.agent.BambooActiveMQConnectionFactory#2cf5d030' defined in class path resource [applicationContextRemoteAgentJMS.xml]: Instantiation of bean failed; nested exception is org.springframework.beans.BeanInstantiationException: Could not instantiate bean class [com.atlassian.bamboo.v2.build.agent.BambooActiveMQConnectionFactory]: Constructor threw exception; nested exception is java.lang.IncompatibleClassChangeError: Class java.net.Socket does not implement the requested interface java.io.Closeable
      	at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveInnerBean(BeanDefinitionValueResolver.java:290)
      	at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveValueIfNecessary(BeanDefinitionValueResolver.java:122)
      	at org.springframework.beans.factory.support.ConstructorResolver.resolveConstructorArguments(ConstructorResolver.java:646)
      	at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:140)
      	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:1114)
      	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1017)
      	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:504)
      	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:475)
      	at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:302)
      	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:228)
      	at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:298)
      	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:193)
      	at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:328)
      	... 35 more
      Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'com.atlassian.bamboo.v2.build.agent.BambooActiveMQConnectionFactory#2cf5d030' defined in class path resource [applicationContextRemoteAgentJMS.xml]: Instantiation of bean failed; nested exception is org.springframework.beans.BeanInstantiationException: Could not instantiate bean class [com.atlassian.bamboo.v2.build.agent.BambooActiveMQConnectionFactory]: Constructor threw exception; nested exception is java.lang.IncompatibleClassChangeError: Class java.net.Socket does not implement the requested interface java.io.Closeable
      	at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:278)
      	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:1114)
      	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1017)
      	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:504)
      	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:475)
      	at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveInnerBean(BeanDefinitionValueResolver.java:276)
      	... 47 more
      Caused by: org.springframework.beans.BeanInstantiationException: Could not instantiate bean class [com.atlassian.bamboo.v2.build.agent.BambooActiveMQConnectionFactory]: Constructor threw exception; nested exception is java.lang.IncompatibleClassChangeError: Class java.net.Socket does not implement the requested interface java.io.Closeable
      	at org.springframework.beans.BeanUtils.instantiateClass(BeanUtils.java:164)
      	at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:125)
      	at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:270)
      	... 52 more
      Caused by: java.lang.IncompatibleClassChangeError: Class java.net.Socket does not implement the requested interface java.io.Closeable
      	at com.google.common.io.Closeables.close(Closeables.java:80)
      	at com.google.common.io.Closeables.closeQuietly(Closeables.java:99)
      	at com.atlassian.bamboo.v2.build.agent.BambooActiveMQConnectionFactory.ping(BambooActiveMQConnectionFactory.java:191)
      	at com.atlassian.bamboo.v2.build.agent.BambooActiveMQConnectionFactory.validateUri(BambooActiveMQConnectionFactory.java:145)
      	at com.atlassian.bamboo.v2.build.agent.BambooActiveMQConnectionFactory.setBrokerURL(BambooActiveMQConnectionFactory.java:63)
      	at org.apache.activemq.ActiveMQConnectionFactory.<init>(ActiveMQConnectionFactory.java:142)
      	at org.apache.activemq.ActiveMQConnectionFactory.<init>(ActiveMQConnectionFactory.java:138)
      	at org.apache.activemq.ActiveMQSslConnectionFactory.<init>(ActiveMQSslConnectionFactory.java:75)
      	at com.atlassian.bamboo.v2.build.agent.BambooActiveMQConnectionFactory.<init>(BambooActiveMQConnectionFactory.java:53)
      	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
      	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
      	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
      	at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
      	at org.springframework.beans.BeanUtils.instantiateClass(BeanUtils.java:148)
      	... 54 more
      

      Notes

      This will affect older custom images which were created when Java 6 was still the default JDK installed in Atlassian instances.

      Workaround

      Install JDK8u45 and make sure that JAVA_HOME is properly set system-wide to point at JDK8u45. This will allow the Bamboo Agent to start up again normally.

      Downloads of previous versions of Java can be found at the Oracle Java Archive page.

      Attachments

        Issue Links

          Activity

            People

              pbruski Przemek Bruski
              imaduro Ivan Maduro (Inactive)
              Votes:
              16 Vote for this issue
              Watchers:
              31 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: