Details
-
Bug
-
Resolution: Done
-
High
-
5.13.0.1
-
None
-
Severity 1 - Critical
-
Description
Summary
DefaultListableBeanFactory Ignoring bean creation exception. The nested exception is logged only when the log4j properties is tweaked to log DEBUG for org.springframework.beans.factory.support
The nested exception at the root of the problem often causes Bamboo to 'hang' on the following with nothing meaningful logged. E.g:
2016-08-01 23:21:00,440 INFO [localhost-startStop-1] [StdSchedulerFactory] Quartz scheduler 'scheduler' initialized from an externally provided properties instance.
2016-08-01 23:21:00,440 INFO [localhost-startStop-1] [StdSchedulerFactory] Quartz scheduler version: 1.8.6
2016-08-01 23:21:00,441 INFO [localhost-startStop-1] [QuartzScheduler] JobFactory set to: com.atlassian.bamboo.quartz.AutowiringJobFactory@293ad6c3
ActiveMQ will then repeatedly restart on roughly a 15 minute interval while the application is left up.
Steps to Reproduce
- Add the following to <bamboo-install>/atlassian-bamboo/WEB-INF/classes/log4j.properties:
log4j.logger.org.springframework.beans.factory.support=DEBUG
- Change ownership / revoke permissions on <bamboo-home>/caches/plugins/felix for the Bamboo user
- Start Bamboo
Expected Results
Something like the following is logged at a higher level that is captured by default. Ideally we would send this up to the UI.
org.springframework.beans.BeanInstantiationException: Could not instantiate bean class [com.atlassian.plugin.osgi.container.impl.DefaultOsgiPersistentCache]: Constructor threw exception; nested exception is com.atlassian.plugin.osgi.container.OsgiContainerException: Unable to clean the cache directory: /var/bamboo-home/caches/plugins/felix
Actual Results
The below exception is logged in the catalina.out file only when DEBUG for is enabled
2016-08-30 14:24:43,825 DEBUG [localhost-startStop-1] [DefaultListableBeanFactory] Ignoring bean creation exception on FactoryBean type check: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'planStatePersister' defined in class path resource [applicationContextManagers.xml]: Cannot create inner bean 'com.atlassian.bamboo.plan.PlanStatePersisterImpl#8888e86' of type [com.atlassian.bamboo.plan.PlanStatePersisterImpl] while setting bean property 'target'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'com.atlassian.bamboo.plan.PlanStatePersisterImpl#8888e86' defined in class path resource [applicationContextManagers.xml]: Cannot resolve reference to bean 'planManager' while setting constructor argument; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'planManager' defined in class path resource [applicationContextManagers.xml]: Cannot create inner bean 'com.atlassian.bamboo.plan.PlanManagerImpl#17374ac2' of type [com.atlassian.bamboo.plan.PlanManagerImpl] while setting bean property 'target'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'com.atlassian.bamboo.plan.PlanManagerImpl#17374ac2' defined in class path resource [applicationContextManagers.xml]: Cannot resolve reference to bean 'planScheduler' while setting constructor argument; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'planScheduler' defined in class path resource [applicationContextManagers.xml]: Cannot resolve reference to bean 'triggerTypeManager' while setting constructor argument; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'buildStrategyManager' defined in class path resource [applicationContextManagers.xml]: Cannot resolve reference to bean 'pluginAccessor' while setting constructor argument; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'pluginManager' defined in class path resource [applicationContextPlugin.xml]: Cannot create inner bean 'util:list#5ebc7cda' of type [org.springframework.beans.factory.config.ListFactoryBean] while setting constructor argument; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'util:list#5ebc7cda': Cannot resolve reference to bean 'bundledPluginLoader' while setting bean property 'sourceList' with key [0]; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'bundledPluginLoader' defined in class path resource [applicationContextPlugin.xml]: Cannot create inner bean 'util:list#477d8f5b' of type [org.springframework.beans.factory.config.ListFactoryBean] while setting constructor argument; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'util:list#477d8f5b': Cannot resolve reference to bean 'osgiPluginFactory' while setting bean property 'sourceList' with key [0]; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'osgiPluginFactory' defined in class path resource [applicationContextPluginCommon.xml]: Cannot resolve reference to bean 'osgiPersistentCache' while setting constructor argument; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'osgiPersistentCache' defined in class path resource [applicationContextPluginCommon.xml]: Instantiation of bean failed; nested exception is org.springframework.beans.BeanInstantiationException: Could not instantiate bean class [com.atlassian.plugin.osgi.container.impl.DefaultOsgiPersistentCache]: Constructor threw exception; nested exception is com.atlassian.plugin.osgi.container.OsgiContainerException: Unable to clean the cache directory: /var/bamboo-home/caches/plugins/felix
Notes
There are additional causes and exceptions that can be uncovered in the same area. See the following knowledge-base article for a few other known causes:
The remote agent logs the same exception as WARN:
INFO | jvm 1 | 2016/08/30 16:46:23 | 2016-08-30 16:46:23,081 WARN [AgentRunnerThread] [ClassPathXmlApplicationContext] Exception encountered during context initialization - cancelling refresh attempt INFO | jvm 1 | 2016/08/30 16:46:23 | org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'storageTagService': Injection of autowired dependencies failed; nested exception is org.springframework.beans.factory.BeanCreationException: Could not autowire field: private com.atlassian.bamboo.v2.build.agent.capability.AgentContext com.atlassian.bamboo.v2.build.agent.remote.RemoteAgentStorageTagService.agentContext; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'directoryScanningPluginLoader' defined in class path resource [applicationContextPluginCommon.xml]: Cannot create inner bean 'util:list#1ff8f273' of type [org.springframework.beans.factory.config.ListFactoryBean] while setting constructor argument; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'util:list#1ff8f273': Cannot resolve reference to bean 'osgiPluginFactory' while setting bean property 'sourceList' with key [0]; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'osgiPluginFactory' defined in class path resource [applicationContextPluginCommon.xml]: Cannot resolve reference to bean 'osgiPersistentCache' while setting constructor argument; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'osgiPersistentCache' defined in class path resource [applicationContextPluginCommon.xml]: Instantiation of bean failed; nested exception is org.springframework.beans.BeanInstantiationException: Could not instantiate bean class [com.atlassian.plugin.osgi.container.impl.DefaultOsgiPersistentCache]: Constructor threw exception; nested exception is com.atlassian.plugin.osgi.container.OsgiContainerException: Unable to clean the cache directory: /var/bamboo-home/caches/plugins/felix INFO | jvm 1 | 2016/08/30 16:46:23 | at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessPropertyValues(AutowiredAnnotationBeanPostProcessor.java:301) INFO | jvm 1 | 2016/08/30 16:46:23 | at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1186) INFO | jvm 1 | 2016/08/30 16:46:23 | at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:53