-
Type:
Bug
-
Resolution: Low Engagement
-
Priority:
Medium
-
None
-
Affects Version/s: 5.0.1, 5.3.1, 4.14.9
-
Component/s: None
-
1
-
Severity 3 - Minor
-
0
Summary
When using the AWS quickstart/CloudFormation template, the nodes do not join the cluster after the Data Center instance has been created.
Environment
- AWS
Steps to Reproduce
- Use quickstart to launch a new DC instance in AWS
Expected Results
- All nodes should join the cluster
Actual Results
The below exception is thrown in the atlassian-bitbucket.log file of the node that did not join the cluster:
2017-09-08 17:59:26,511 INFO [main] o.s.boot.SpringApplication . ____ _ __ _ _ /\\ / ___'_ __ _ _(_)_ __ __ _ \ \ \ \ ( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \ \\/ ___)| |_)| | | | | || (_| | ) ) ) ) ' |____| .__|_| |_|_| |_\__, | / / / / =========|_|==============|___/=/_/_/_/ :: Spring Boot :: (v1.5.3.RELEASE) 2017-09-08 17:59:26,787 INFO [main] c.a.b.i.b.BitbucketServerApplication Starting BitbucketServerApplication on ip-10-0-21-69 with PID 14942 (/opt/atlassian/bitbucket/app/WEB-INF/classes started by atlb itbucket in /) 2017-09-08 17:59:26,788 INFO [main] c.a.b.i.b.BitbucketServerApplication No active profile set, falling back to default profiles: default 2017-09-08 17:59:28,198 INFO [main] c.a.b.i.boot.log.BuildInfoLogger Starting Bitbucket 5.0.1 (a53df94 built on Wed May 10 03:40:47 UTC 2017) 2017-09-08 17:59:28,198 INFO [main] c.a.b.i.boot.log.BuildInfoLogger JVM: Oracle Corporation OpenJDK 64-Bit Server VM 1.8.0_141-b16 2017-09-08 17:59:30,185 INFO [main] c.a.b.i.b.BitbucketServerApplication Started BitbucketServerApplication in 4.963 seconds (JVM running for 7.11) 2017-09-08 17:59:34,663 INFO [spring-startup] c.a.s.internal.home.HomeLockAcquirer Successfully acquired lock on home directory /var/atlassian/application-data/bitbucket 2017-09-08 17:59:36,174 INFO [spring-startup] c.a.s.internal.ApplicationSettings Capturing benchmark home version 5.0.1 2017-09-08 17:59:40,342 ERROR [spring-startup] c.a.s.internal.home.HomeLockAcquirer Home directory /media/atl/bitbucket/shared has already been locked by another instance 2017-09-08 17:59:40,343 WARN [spring-startup] o.s.w.c.s.XmlWebApplicationContext Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.Unsat isfiedDependencyException: Error creating bean with name 'crowdAliasDao': Unsatisfied dependency expressed through method 'setSessionFactory' parameter 0; nested exception is org.springframework.beans.fac tory.BeanCreationException: Error creating bean with name 'sharedHomeLockAcquirer' defined in class path resource [stash-context.xml]: Invocation of init method failed; nested exception is com.atlassian.s tash.internal.home.HomeLockDeniedException: Home directory '/media/atl/bitbucket/shared' is already locked by another running instance of Bitbucket. Please stop the other instance of Bitbucket and restart this instance. If you are absolutely certain that no other instance of Bitbucket is running, please remove the lock file '/media/atl/bitbucket/shared/.lock' and restart this instance. 2017-09-08 17:59:40,369 INFO [spring-startup] c.a.s.internal.home.HomeLockAcquirer Releasing lock on /var/atlassian/application-data/bitbucket 2017-09-08 17:59:40,374 ERROR [spring-startup] c.a.j.s.w.s.JohnsonDispatcherServlet SpringMVC dispatcher [springMvc] could not be started org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'crowdAliasDao': Unsatisfied dependency expressed through method 'setSessionFactory' parameter 0; nested exc eption is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'sharedHomeLockAcquirer' defined in class path resource [stash-context.xml]: Invocation of init method fail ed; nested exception is com.atlassian.stash.internal.home.HomeLockDeniedException: Home directory '/media/atl/bitbucket/shared' is already locked by another running instance of Bitbucket. Please stop the other instance of Bitbucket and restart this instance. If you are absolutely certain that no other instance of Bitbucket is running, please remove the lock file '/media/atl/bitbucket/shared/.lock' and restart this instance. at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredMethodElement.inject(AutowiredAnnotationBeanPostProcessor.java:667) at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:866) at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:542) at javax.servlet.GenericServlet.init(GenericServlet.java:158) at java.lang.Thread.run(Thread.java:748) ... 18 frames trimmed Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'sharedHomeLockAcquirer' defined in class path resource [stash-context.xml]: Invocation of init method fai led; nested exception is com.atlassian.stash.internal.home.HomeLockDeniedException: Home directory '/media/atl/bitbucket/shared' is already locked by another running instance of Bitbucket. Please stop the other instance of Bitbucket and restart this instance. If you are absolutely certain that no other instance of Bitbucket is running, please remove the lock file '/media/atl/bitbucket/shared/.lock' and restart this instance. at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1628) ... 5 common frames omitted Caused by: com.atlassian.stash.internal.home.HomeLockDeniedException: Home directory '/media/atl/bitbucket/shared' is already locked by another running instance of Bitbucket. Please stop the other instance of Bitbucket and restart this instance. If you are absolutely certain that no other instance of Bitbucket is running, please remove the lock file '/media/atl/bitbucket/shared/.lock' and restart this instance. at com.atlassian.stash.internal.home.HomeLockAcquirer.onLockDenied(HomeLockAcquirer.java:148) at com.atlassian.stash.internal.home.HomeLockAcquirer.lock(HomeLockAcquirer.java:63) ... 5 common frames omitted
Notes
- I was able to replicate this with 4.14.9, 5.0.1 and 5.3.1
Workaround
- As the error mentions, if the .lock file in the $BITBUCKET_HOME/shared/ directory is removed and the service on the affected node is restarted, it will join the cluster after startup.