Issue Summary

Elastic Agents which start in IMDSv2 are unable to check their own instance metadata.

Images created in Amazon Linux 2023, by default are started flagged as IMDSv2 required.

An AMI image which has been flagged as having IMDSv2 required cannot have that entry disabled on the AMI. However images can be started using IMDSv1 with start options.

This is reproducible on Data Center: (yes)

Steps to Reproduce

1. Step 1 Create an Elastic Agent instance using (AL2023)
2. Step 2 start the instance

Expected Results

Image starts and Agent connects

Actual Results

Image starts
Agent is unable to connect because it cannot verify that it was "Started by Bamboo", as it's unable to check it's metadata

Logs will show an error similar to:

Exception in thread "main" java.io.IOException: Server returned HTTP response code: 401 for URL: http://169.254.169.254/2016-09-02/user-data

Workarounds

1. Use the available Stock Images
   • Bamboo Stock images
2. Configure the EC2 instance to modify its metadata upon startup to set IMDSv2 as optional
   1. In the Bamboo Elastic Agent configuration menu, select "Upload AWS account identifiers to elastic instances" and save
   2. In the Instance Startup script add:

      #!/bin/bash
      # Set AWS credentials
      export AWS_ACCESS_KEY_ID=$(cat /root/awsAccessKey)
      export AWS_SECRET_ACCESS_KEY=$(cat /root/awsSecretKey)
      
      # Get a token for IMDSv2
      TOKEN=$(curl -X PUT -H "X-aws-ec2-metadata-token-ttl-seconds: 21600" -s http://169.254.169.254/latest/api/token)
      
      # Use the token to get the region
      REGION=$(curl -H "X-aws-ec2-metadata-token: $TOKEN" -s http://169.254.169.254/latest/meta-data/placement/region)
      
      # Use the token to get the instance ID
      INSTANCE_ID=$(curl -H "X-aws-ec2-metadata-token: $TOKEN" -s http://169.254.169.254/latest/meta-data/instance-id)
      
      # Run AWS CLI command
      aws ec2 modify-instance-metadata-options --region $REGION --instance-id $INSTANCE_ID --http-endpoint=enabled --http-tokens=optional
      

   3. After following the steps above, all instances will be able to communicate over IMDSv1 and you will likely have your environment restored.
   4. All the steps above were tested in a Stock image. In case you use a custom image, your experience may vary, and you may require further adjustments that may be beyond the support scope.