-
Suggestion
-
Resolution: Unresolved
-
None
-
2
-
4
-
Problem Definition
When Amazon adds a new IP range to the involved services (EC2 and S3), elastic agents are not able to communicate to Atlassian buckets to download the needed files.
An error like this one is thrown in the Elastic Agent log file:
Syncing Elastic Bamboo Agent files... 2017-09-13 17:17:24,597 INFO [main] [S3Sync] Syncing from: bamboo-agent-release-us-w1/5.14.4.1/7c492fa079f7969a702e3781fb83511749a24cd307b48c6fcdfb7409e6f31ac7/ to /opt/bamboo-elastic-agent 2017-09-13 17:17:25,225 INFO [main] [S3Synchroniser] Syncing s3://bamboo-agent-release-us-w1/5.14.4.1/7c492fa079f7969a702e3781fb83511749a24cd307b48c6fcdfb7409e6f31ac7/ to /opt/bamboo-elastic-agent 2017-09-13 17:17:25,226 INFO [main] [AmazonClients] Detecting bucket location for [bamboo-agent-release-us-w1] 2017-09-13 17:17:25,576 WARN [main] [AmazonClients] Unable to get bucket location for [bamboo-agent-release-us-w1], using default. Error: com.amazonaws.services.s3.model.AmazonS3Exception: Access Denied (Service: Amazon S3; Status Code: 403; Error Code: AccessDenied; Request ID: 975109E1DDA85904), S3 Extended Request ID: oute8zxYA3gMECo86qNYDrtvSh8EcCubbkrJPIOae+/TadliLLHmoQ87h0hWeutW4AStTrus29Q= 2017-09-13 17:17:25,576 INFO [main] [S3Synchroniser] Fetching the list of remote objects... 2017-09-13 17:17:25,583 ERROR [main] [ElasticAgentInstaller] Unable to access S3 bucket, dumping the route table: 2017-09-13 17:17:25,583 INFO [main] [ElasticAgentInstaller] Starting [route, -n] Exception in thread "main" java.io.IOException: Cannot run program "route": error=2, No such file or directory at java.lang.ProcessBuilder.start(ProcessBuilder.java:1048) at com.atlassian.bamboo.agent.elastic.installer.ElasticAgentInstaller.runCommand(ElasticAgentInstaller.java:135) at com.atlassian.bamboo.agent.elastic.installer.ElasticAgentInstaller.dumpRouteTable(ElasticAgentInstaller.java:125) at com.atlassian.bamboo.agent.elastic.installer.ElasticAgentInstaller.install(ElasticAgentInstaller.java:84) at com.atlassian.bamboo.agent.elastic.installer.ElasticAgentInstaller.main(ElasticAgentInstaller.java:199) Caused by: java.io.IOException: error=2, No such file or directory at java.lang.UNIXProcess.forkAndExec(Native Method) at java.lang.UNIXProcess.<init>(UNIXProcess.java:248) at java.lang.ProcessImpl.start(ProcessImpl.java:134) at java.lang.ProcessBuilder.start(ProcessBuilder.java:1029) ... 4 more
Suggested Solution
Nowadays, when this situation happens, the new IP range needs to be whitelisted manually, in the perfect scenario, this change will be handled automatically.
Workaround
Please ensure the EC2 instance is able to connect to the AWS IP Ranges. If this is not working, please reach the support team.
- is related to
-
BAM-18763 Elastic Bamboo failing to run "route" command
-
- Closed
-
![[Extranet] Page](/images/icons/generic_link_16.png "[Extranet] Page"){kind=icon}
- relates to
-
BDEV-15969 Loading...
We encountered this for the 2nd time as well today, resulting in a notable outage of our build processes.
In terms of the suggested solution, AWS meanwhile provides resp. AWS IP Address Ranges Notifications - while the introductory blog points out that a consumer is still responsible for fetching the file, parsing the JSON, and extracting the desired information, the notifications seem to be near real-time (or even upfront) so that the chance of a user receiving a not yet whitelisted IP address could be nicely minimized (or even eliminated: somebody somewhere once mentioned a grace period between advertisement and actual usage of new IP address ranges in EC2, but I cannot seem to find it again).
Many thanks,