• 5
    • 81
    • Our product teams collect and evaluate feedback from a number of different sources. To learn more about how we use customer feedback in the planning process, check out our new feature policy.

      Currently Elastic Bamboo requires you give it your account level credentials to run properly. This isn't really acceptable in many environments and we should be able to use IAM to create a bamboo user with restricted permission (i.e. bamboo doesn't need permissions to create a new VPC, or delete non-bamboo instances).

      To me this is a major security problem. The response so far from support was this:

      "After confirming with the Bamboo seniors, I can say that the Bamboo user does need full access to the account. We also recommend that a dedicated account be created for Bamboo to ensure that, should Bamboo "go sideways," any damage is restricted only to the Bamboo user's account."

      Running a dedicated account just for bamboo isn't horribly practical if you have other resources the builds depend on, especially in a VPC environment.

          Form Name

            [BAM-11932] Bamboo requires full permissions to EC2

            r0b0ticus added a comment -

            https://jira.atlassian.com/browse/BAM-11932?focusedCommentId=2310432&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-2310432

            This is absolutely correct. If the Bamboo server is running in AWS you should be able to use instance profiles which is not an option now.

            r0b0ticus added a comment - https://jira.atlassian.com/browse/BAM-11932?focusedCommentId=2310432&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-2310432 This is absolutely correct. If the Bamboo server is running in AWS you should be able to use instance profiles which is not an option now.

            Hi Krystian,

            Any update on this?

            Regards

            Michal

             

            Michal Gawlik added a comment - Hi Krystian, Any update on this? Regards Michal  

            GP added a comment -

            I strongly support for the IAM Role policy to be used instead of Access keys/Secret Keys

            GP added a comment - I strongly support for the IAM Role policy to be used instead of Access keys/Secret Keys

            It's been over 7 years and we still don't have a set of permissions that Bamboo needs? How, just how.

            bchester404 added a comment - It's been over 7 years and we still don't have a set of permissions that Bamboo needs? How, just how.

            Atlassian Update

            Hi everyone,

            Thanks for your interest in this issue.
            This request is considered a potential addition to our longer-term roadmap.

            We'll typically review this request in about 6 months time, at which point we’ll consider whether we need to alter its status.

            For the nearest future we've decided to prioritise other areas of the Bamboo roadmap, including:

            1. Robustness of Plan Branches
            2. Performance and stability improvements
            3. Providing building blocks for High Availability and Disaster Recovery solutions
            4. Improving permission system
            5. Allowing per-project allocation of resources
            6. Improving Bitbucket Server and Jira integrations

            You can learn more about our approach to highly voted server suggestions here.

            To learn more on how your suggestions are reviewed, see our updated workflow for server feature suggestions.

            Kind regards,
            Bamboo Team

            Krystian Brazulewicz added a comment - Atlassian Update Hi everyone, Thanks for your interest in this issue. This request is considered a potential addition to our longer-term roadmap. We'll typically review this request in about 6 months time, at which point we’ll consider whether we need to alter its status. For the nearest future we've decided to prioritise other areas of the Bamboo roadmap, including: Robustness of Plan Branches Performance and stability improvements Providing building blocks for High Availability and Disaster Recovery solutions Improving permission system Allowing per-project allocation of resources Improving Bitbucket Server and Jira integrations You can learn more about our approach to highly voted server suggestions here . To learn more on how your suggestions are reviewed, see our updated workflow for server feature suggestions . Kind regards, Bamboo Team

            I believe it probably is.  I don't think there's much interest in resolving this issue since it's almost 7 years old...

            You would think they would at least have the guts to say "we won't do this".

            Michael Nielsen added a comment - I believe it probably is.  I don't think there's much interest in resolving this issue since it's almost 7 years old... You would think they would at least have the guts to say "we won't do this".

            24/Jul/2012? This can't be accurate can it?

            jeremiahsmall added a comment - 24/Jul/2012? This can't be accurate can it?

            +1
            Please add support for EC2 IAM Role / Instance Profile based permissions for Bamboo.
            AWS do not recommend to use IAM Users with Key/Secret in production environments - everything should be based on IAM Roles and federated access.

            Lack of this feature is a showstopper for adoption of Bamboo.

            Maciej Malek added a comment - +1 Please add support for EC2 IAM Role / Instance Profile based permissions for Bamboo. AWS do not recommend to use IAM Users with Key/Secret in production environments - everything should be based on IAM Roles and federated access. Lack of this feature is a showstopper for adoption of Bamboo.

            Yes, this is also a blocker for us. We need to stay with Jenkins...

            Michal Gawlik added a comment - Yes, this is also a blocker for us. We need to stay with Jenkins...

            The lack of proper support for EC2 is almost a show-stopper for us.

             

            I'm thinking we may need to junk Bamboo in favor of Jenkins.

             

            Maybe that's really what Atlassian wants anyway.

            Michael Nielsen added a comment - The lack of proper support for EC2 is almost a show-stopper for us.   I'm thinking we may need to junk Bamboo in favor of Jenkins.   Maybe that's really what Atlassian wants anyway.

            nrivett added a comment -

            Atlassian this really screws with our integration here. You are basically making it impossible to migrate to elastic bamboo. I don't think this feature is production ready and should probably be taken off the features list

            nrivett added a comment - Atlassian this really screws with our integration here. You are basically making it impossible to migrate to elastic bamboo. I don't think this feature is production ready and should probably be taken off the features list

            Still no official solution?

            Terho Siikanen added a comment - Still no official solution?

            Filtering based on IAM role is currently the best approach.

            Przemek Bruski added a comment - Filtering based on IAM role is currently the best approach.

            The problem with the restrictions based on the tag name is that the bamboo server also has the createTag privilege, which means a malicious user or a nasty bug can basically retag all the instances and terminate them

            Another approach we are considering to use is to limit the dangerous operation to EC2 instances which have a predefined IAM role. Same security problem applies, but at least it's harder to exploit.

            dejan_golja added a comment - The problem with the restrictions based on the tag name is that the bamboo server also has the createTag privilege, which means a malicious user or a nasty bug can basically retag all the instances and terminate them Another approach we are considering to use is to limit the dangerous operation to EC2 instances which have a predefined IAM role. Same security problem applies, but at least it's harder to exploit.

            This should be an out of the box thing. Any ETA on when this will be limited?

            David Markowski (DO NOT USE) added a comment - This should be an out of the box thing. Any ETA on when this will be limited?

            @Oliver I was able to compose the policy below from policies shared here and from Atlassian Answers.

            {
                "Version": "2012-10-17",
                "Statement": [
                    {
                        "Sid": "AllowBambooToStartNewBuildAgents",
                        "Effect": "Allow",
                        "Action": [
                            "ec2:Describe*",
                            "ec2:RequestSpot*",
                            "ec2:CancelSpot*",
                            "ec2:AllocateAddress",
                            "ec2:AssociateAddress",
                            "ec2:AttachVolume",
                            "ec2:AuthorizeSecurityGroupIngress",
                            "ec2:CancelSpotInstanceRequests",
                            "ec2:CreateKeyPair",
                            "ec2:CreateSecurityGroup",
                            "ec2:CreateTags",
                            "ec2:CreateVolume",
                            "ec2:DeleteVolume",
                            "ec2:DescribeAddresses",
                            "ec2:DescribeAvailabilityZones",
                            "ec2:DescribeImages",
                            "ec2:DescribeInstances",
                            "ec2:DescribeKeyPairs",
                            "ec2:DescribeRegions",
                            "ec2:DescribeSecurityGroups",
                            "ec2:DescribeSpotInstanceRequests",
                            "ec2:DescribeSpotPriceHistory",
                            "ec2:DescribeSubnets",
                            "ec2:DescribeVolumes",
                            "ec2:DescribeVpcs",
                            "ec2:GetConsoleOutput",
                            "ec2:ModifyInstanceAttribute",
                            "ec2:ReleaseAddress",
                            "ec2:RequestSpotInstances",
                            "ec2:RunInstances"
                        ],
                        "Resource": "*"
                    },
                    {
                        "Sid": "OnlyAllowBambooToTerminateBuildAgents",
                        "Effect": "Allow",
                        "Action": [
                            "ec2:TerminateInstances",
                            "ec2:StopInstances",
                            "ec2:StartInstances"
                        ],
                        "Resource": "arn:aws:ec2:[aws-region]:[aws-account-id]:instance/*",
                        "Condition": {
                            "StringEquals": {
                                "ec2:ResourceTag/Name": "bam::[bamboo.hosted.domain]::js-bamboo"
                            }
                        }
                    },
                    {
                        "Sid": "AllowBambooAccessToTheBuildArtifactsS3Bucket",
                        "Effect": "Allow",
                        "Action": "s3:*",
                        "Resource": [
                            "arn:aws:s3:::[bamboo-bucket-name]",
                            "arn:aws:s3:::[bamboo-bucket-name]/*"
                        ]
                    }
                ]
            }
            

            There may even be room to improve this even more, but its working for me as is.
            If you are running Bamboo as part of the Atlassian Cloud offering then replace [bamboo.hosted.domain] with the [your.site.name].atlassian.net.

            Mark Rekveld - Marvelution added a comment - @Oliver I was able to compose the policy below from policies shared here and from Atlassian Answers. { "Version" : "2012-10-17" , "Statement" : [ { "Sid" : "AllowBambooToStartNewBuildAgents" , "Effect" : "Allow" , "Action" : [ "ec2:Describe*" , "ec2:RequestSpot*" , "ec2:CancelSpot*" , "ec2:AllocateAddress" , "ec2:AssociateAddress" , "ec2:AttachVolume" , "ec2:AuthorizeSecurityGroupIngress" , "ec2:CancelSpotInstanceRequests" , "ec2:CreateKeyPair" , "ec2:CreateSecurityGroup" , "ec2:CreateTags" , "ec2:CreateVolume" , "ec2:DeleteVolume" , "ec2:DescribeAddresses" , "ec2:DescribeAvailabilityZones" , "ec2:DescribeImages" , "ec2:DescribeInstances" , "ec2:DescribeKeyPairs" , "ec2:DescribeRegions" , "ec2:DescribeSecurityGroups" , "ec2:DescribeSpotInstanceRequests" , "ec2:DescribeSpotPriceHistory" , "ec2:DescribeSubnets" , "ec2:DescribeVolumes" , "ec2:DescribeVpcs" , "ec2:GetConsoleOutput" , "ec2:ModifyInstanceAttribute" , "ec2:ReleaseAddress" , "ec2:RequestSpotInstances" , "ec2:RunInstances" ], "Resource" : "*" }, { "Sid" : "OnlyAllowBambooToTerminateBuildAgents" , "Effect" : "Allow" , "Action" : [ "ec2:TerminateInstances" , "ec2:StopInstances" , "ec2:StartInstances" ], "Resource" : "arn:aws:ec2:[aws-region]:[aws-account-id]:instance/*" , "Condition" : { "StringEquals" : { "ec2:ResourceTag/Name" : "bam::[bamboo.hosted.domain]::js-bamboo" } } }, { "Sid" : "AllowBambooAccessToTheBuildArtifactsS3Bucket" , "Effect" : "Allow" , "Action" : "s3:*" , "Resource" : [ "arn:aws:s3:::[bamboo-bucket-name]" , "arn:aws:s3:::[bamboo-bucket-name]/*" ] } ] } There may even be room to improve this even more, but its working for me as is. If you are running Bamboo as part of the Atlassian Cloud offering then replace [bamboo.hosted.domain] with the [your.site.name].atlassian.net .

            Oliver Day added a comment -

            The official response from Atlassian is "We understand the security risk and we are working to make it work differently in the future, but at the moment this is a requirement."
            This issue was first reported in 2012 so when exactly will this work differently?

            Oliver Day added a comment - The official response from Atlassian is "We understand the security risk and we are working to make it work differently in the future, but at the moment this is a requirement." This issue was first reported in 2012 so when exactly will this work differently?

            Fixed it by adding "ec2:GetConsoleOutput" after "ec2:CancelSpot*".

            Deleted Account (Inactive) added a comment - Fixed it by adding "ec2:GetConsoleOutput" after "ec2:CancelSpot*".

            @Daniel Thank you for the example, it works but I don't get logs anymore of my instance on the atlassian site. Could it be related to the policy?

            I'm talking about this page: https://xxx.atlassian.net/builds/admin/elastic/viewElasticInstance.action?instanceId=xxx

            Deleted Account (Inactive) added a comment - @Daniel Thank you for the example, it works but I don't get logs anymore of my instance on the atlassian site. Could it be related to the policy? I'm talking about this page: https://xxx.atlassian.net/builds/admin/elastic/viewElasticInstance.action?instanceId=xxx

            I tried reaching Atlassian's support for a policy as well, but failed to get an official policy too, and all I could obtain were some links to the forum. After some fiddling, managed to get it working by myself with this policy:

            {
                "Version": "2012-10-17",
                "Statement": [
                    {
                        "Effect": "Allow",
                        "Action": "iam:PassRole",
                        "Resource": "arn:aws:iam::<account_number>:role/bamboo_role"
                    },
                    {
                        "Effect": "Allow",
                        "Action": [
                            "ec2:Describe*",
                            "ec2:CreateTags",
                            "ec2:RunInstances",
                            "ec2:RequestSpot*",
                            "ec2:CancelSpot*"
                        ],
                        "Resource": "*"
                    },
                    {
                        "Effect": "Allow",
                        "Action": "ec2:TerminateInstances",
                        "Resource": "arn:aws:ec2:*:*:instance/*",
                        "Condition": {
                            "StringEquals": {
                                "ec2:ResourceTag/Name": "bam::your.domain.com::js-bamboo"
                            }
                        }
                    }
                ]
            }

            It isn't perfect, but it works for now.

            Deleted Account (Inactive) added a comment - I tried reaching Atlassian's support for a policy as well, but failed to get an official policy too, and all I could obtain were some links to the forum. After some fiddling, managed to get it working by myself with this policy: { "Version" : "2012-10-17" , "Statement" : [ { "Effect" : "Allow" , "Action" : "iam:PassRole" , "Resource" : "arn:aws:iam::<account_number>:role/bamboo_role" }, { "Effect" : "Allow" , "Action" : [ "ec2:Describe*" , "ec2:CreateTags" , "ec2:RunInstances" , "ec2:RequestSpot*" , "ec2:CancelSpot*" ], "Resource" : "*" }, { "Effect" : "Allow" , "Action" : "ec2:TerminateInstances" , "Resource" : "arn:aws:ec2:*:*:instance/*" , "Condition" : { "StringEquals" : { "ec2:ResourceTag/Name" : "bam::your.domain.com::js-bamboo" } } } ] } It isn't perfect, but it works for now.

            Rob Olmos added a comment -

            Over 3 years later and, even worse than not having an official IAM policy, Bamboo docs are still requesting the master account credentials.

            Security is not an afterthought.

            Rob Olmos added a comment - Over 3 years later and, even worse than not having an official IAM policy, Bamboo docs are still requesting the master account credentials. Security is not an afterthought.

            Tom added a comment -

            Now wonder why I couldn't find the IAM policy in Confluence for Bamboo... Atlassian hasn't released one.

            I can see if EC2 was simply one integration out of many for Bamboo Cloud, but IT IS REQUIRED.

            So, Atlassian should follow Amazon best practices, limiting access, and provide an IAM policy (and what each component is needed for in terms of product functionality).

            What Atlassian has in terms of great products, they lack in Security (still can't believe 2FA is not in every product or cloud for that matter).

            Tom added a comment - Now wonder why I couldn't find the IAM policy in Confluence for Bamboo... Atlassian hasn't released one. I can see if EC2 was simply one integration out of many for Bamboo Cloud, but IT IS REQUIRED. So, Atlassian should follow Amazon best practices, limiting access, and provide an IAM policy (and what each component is needed for in terms of product functionality). What Atlassian has in terms of great products, they lack in Security (still can't believe 2FA is not in every product or cloud for that matter).

            I'm not sure why this is not documented any better. Like Alex's example Amazon has an in depth article about how to limit an IAM user: https://blogs.aws.amazon.com/security/post/Tx1ZU3LW4LLPQY2/How-to-Help-Lock-Down-a-User-s-Amazon-EC2-Capabilities-to-a-Single-VPC.

            I've tested it and you can get Bamboo to work with a limited role.

            You can also limit it with a super simple AWS policy like:

            {
              "Version": "2012-10-17",
              "Statement":[{
                "Effect":"Allow",
                "Action":["ec2:RunInstances",
                              "ec2:Describe*"]
                 "Resource":"*"
                 },
                 {
                 "Effect":"Deny",
                 "NotAction":["ec2:StopInstances",
                                     "ec2:StartInstances",
                                     "ec2:TerminateInstances"],
                 "Resource":"*"
                 }
              ]
            }
            

            This worked perfectly for me

            Jeff Calhoun added a comment - I'm not sure why this is not documented any better. Like Alex's example Amazon has an in depth article about how to limit an IAM user: https://blogs.aws.amazon.com/security/post/Tx1ZU3LW4LLPQY2/How-to-Help-Lock-Down-a-User-s-Amazon-EC2-Capabilities-to-a-Single-VPC . I've tested it and you can get Bamboo to work with a limited role. You can also limit it with a super simple AWS policy like: { "Version" : "2012-10-17" , "Statement" :[{ "Effect" : "Allow" , "Action" :[ "ec2:RunInstances" , "ec2:Describe*" ] "Resource" : "*" }, { "Effect" : "Deny" , "NotAction" :[ "ec2:StopInstances" , "ec2:StartInstances" , "ec2:TerminateInstances" ], "Resource" : "*" } ] } This worked perfectly for me

            +1

            This practice is against AWS own guidelines, and its just basic security 101. You never give full access to an AWS user, if access to that user is ever compromised, its possible to bring down a hole system

            Matthew de Detrich added a comment - +1 This practice is against AWS own guidelines, and its just basic security 101. You never give full access to an AWS user, if access to that user is ever compromised, its possible to bring down a hole system

            Atlassian,
            The first security recommendation from AWS documentation is to remove API access for users with full permissions including the admin.
            You requirements/guidelines that the bamboo user needs full admin permissions is clearly not in line and does not appear thought through.
            Please publish best practices guide how to isolate the secure a development cloud including users policies.

            Adalbert Wysocki added a comment - Atlassian, The first security recommendation from AWS documentation is to remove API access for users with full permissions including the admin. You requirements/guidelines that the bamboo user needs full admin permissions is clearly not in line and does not appear thought through. Please publish best practices guide how to isolate the secure a development cloud including users policies.

            This was the IAM Policy we used to limit access to our EC2 infrastructure. You could probably tweak it further and lock it down a bit more.

            {
                "Version": "2012-10-17",
                "Statement": [
                    {
                        "Sid": "BambooAllowedAll",
                        "Effect": "Allow",
                        "Action": [
                            "ec2:Describe*",
                            "ec2:RequestSpot*",
                            "ec2:CancelSpot*",
                            "ec2:AllocateAddress",
                            "ec2:AssociateAddress",
                            "ec2:AttachVolume",
                            "ec2:AuthorizeSecurityGroupIngress",
                            "ec2:CancelSpotInstanceRequests",
                            "ec2:CreateKeyPair",
                            "ec2:CreateSecurityGroup",
                            "ec2:CreateTags",
                            "ec2:CreateVolume",
                            "ec2:DeleteVolume",
                            "ec2:DescribeAddresses",
                            "ec2:DescribeAvailabilityZones",
                            "ec2:DescribeImages",
                            "ec2:DescribeInstances",
                            "ec2:DescribeKeyPairs",
                            "ec2:DescribeRegions",
                            "ec2:DescribeSecurityGroups",
                            "ec2:DescribeSpotInstanceRequests",
                            "ec2:DescribeSpotPriceHistory",
                            "ec2:DescribeSubnets",
                            "ec2:DescribeVolumes",
                            "ec2:DescribeVpcs",
                            "ec2:GetConsoleOutput",
                            "ec2:ModifyInstanceAttribute",
                            "ec2:ReleaseAddress",
                            "ec2:RequestSpotInstances",
                            "ec2:RunInstances",
                            "s3:GetObject",
                            "s3:ListBucket"
                        ],
                        "Resource": "*"
                    },
                    {
                        "Sid": "BambooAllowedOnlyOnCreated",
                        "Effect": "Allow",
                        "Action": [
                            "ec2:TerminateInstances",
                            "ec2:StopInstances",
                            "ec2:StartInstances"
                        ],
                        "Resource": "arn:aws:ec2:eu-west-1:<YOUR_AWS_ACCOUNT_NUMBER>:instance/*",
                        "Condition": {
                            "StringEquals": {
                                "ec2:ResourceTag/Name": "bam::<YOUR_JIRA_CLOUD_URL>.atlassian.net::js-bamboo"
                            }
                        }
                    }
                ]
            }
            

            ajtrichards added a comment - This was the IAM Policy we used to limit access to our EC2 infrastructure. You could probably tweak it further and lock it down a bit more. { "Version" : "2012-10-17" , "Statement" : [ { "Sid" : "BambooAllowedAll" , "Effect" : "Allow" , "Action" : [ "ec2:Describe*" , "ec2:RequestSpot*" , "ec2:CancelSpot*" , "ec2:AllocateAddress" , "ec2:AssociateAddress" , "ec2:AttachVolume" , "ec2:AuthorizeSecurityGroupIngress" , "ec2:CancelSpotInstanceRequests" , "ec2:CreateKeyPair" , "ec2:CreateSecurityGroup" , "ec2:CreateTags" , "ec2:CreateVolume" , "ec2:DeleteVolume" , "ec2:DescribeAddresses" , "ec2:DescribeAvailabilityZones" , "ec2:DescribeImages" , "ec2:DescribeInstances" , "ec2:DescribeKeyPairs" , "ec2:DescribeRegions" , "ec2:DescribeSecurityGroups" , "ec2:DescribeSpotInstanceRequests" , "ec2:DescribeSpotPriceHistory" , "ec2:DescribeSubnets" , "ec2:DescribeVolumes" , "ec2:DescribeVpcs" , "ec2:GetConsoleOutput" , "ec2:ModifyInstanceAttribute" , "ec2:ReleaseAddress" , "ec2:RequestSpotInstances" , "ec2:RunInstances" , "s3:GetObject" , "s3:ListBucket" ], "Resource" : "*" }, { "Sid" : "BambooAllowedOnlyOnCreated" , "Effect" : "Allow" , "Action" : [ "ec2:TerminateInstances" , "ec2:StopInstances" , "ec2:StartInstances" ], "Resource" : "arn:aws:ec2:eu-west-1:<YOUR_AWS_ACCOUNT_NUMBER>:instance/*" , "Condition" : { "StringEquals" : { "ec2:ResourceTag/Name" : "bam::<YOUR_JIRA_CLOUD_URL>.atlassian.net::js-bamboo" } } } ] }

            +1. Excessive permissions (and completely missing official documentation on what permissions are required) completely rules out Bamboo for my organization.

            Jared Warren added a comment - +1. Excessive permissions (and completely missing official documentation on what permissions are required) completely rules out Bamboo for my organization.

            Kim Toms added a comment -

            This came to my attention when Atlasssian demanded that we move our artifacts to S3, and I worked back through the IAM account assigned. Being told to "Pound Sand" is not very helpful.

            Kim Toms added a comment - This came to my attention when Atlasssian demanded that we move our artifacts to S3, and I worked back through the IAM account assigned. Being told to "Pound Sand" is not very helpful.

            Jada White added a comment -

            +1 Please resolve this. Presents a major security & compliance violation and really should not be so difficult.

            Jada White added a comment - +1 Please resolve this. Presents a major security & compliance violation and really should not be so difficult.

            Alan Cox added a comment -

            Come on Atlassian. Seeing you all cut corners like this is a real big disappointment. This fundamentally rules out Bamboo Cloud for us.

            Alan Cox added a comment - Come on Atlassian. Seeing you all cut corners like this is a real big disappointment. This fundamentally rules out Bamboo Cloud for us.

            From my question on this very issue: https://answers.atlassian.com/questions/16879320/iam-policy-permissions-for-bamboo-aws-requirements

            One quick REAL WORLD EXAMPLE situation for you, as I know "hackers" seem impossible until it's too late for some people. So how about this situation:

            An ATLASSIAN developer accidentally pushes a commit to production that effects the Bamboo automatic instance scaling system. Due to the difference of comma out of place in an array, the conditional evaluator for what instances should be TERMINATED when a bamboo agent stops running breaks. The system then believes all EC2 instances have had their agents shut down and it's way over it's max limit. So is terminates all EC2 instances.

            How many customers would lose data do you think in this situation?

            Improbable?... look here, this is on my screen right now:

            Elastic Bamboo Error : EC2 Instance i-8xxxxxxx is in an unrecognised state: UNKNOWN: Failure in EC2 instance i-8xxxxxxx. Attempting to terminate.

            David Cahill added a comment - From my question on this very issue: https://answers.atlassian.com/questions/16879320/iam-policy-permissions-for-bamboo-aws-requirements One quick REAL WORLD EXAMPLE situation for you, as I know "hackers" seem impossible until it's too late for some people. So how about this situation: An ATLASSIAN developer accidentally pushes a commit to production that effects the Bamboo automatic instance scaling system. Due to the difference of comma out of place in an array, the conditional evaluator for what instances should be TERMINATED when a bamboo agent stops running breaks. The system then believes all EC2 instances have had their agents shut down and it's way over it's max limit. So is terminates all EC2 instances. How many customers would lose data do you think in this situation? Improbable?... look here, this is on my screen right now: Elastic Bamboo Error : EC2 Instance i-8xxxxxxx is in an unrecognised state: UNKNOWN: Failure in EC2 instance i-8xxxxxxx. Attempting to terminate.

            Unfortunately I see this theme repeated throughout Atlassian products - very reasonable requests are ignored. Clearly an organisation with a culture of arrogance.

            Cameron Beattie added a comment - Unfortunately I see this theme repeated throughout Atlassian products - very reasonable requests are ignored. Clearly an organisation with a culture of arrogance.

            +1 and strongly agree that not publishing a recommended, least-privilege IAM policy is irresponsible and lazy.

            Cameron Stokes added a comment - +1 and strongly agree that not publishing a recommended, least-privilege IAM policy is irresponsible and lazy.

            Tim O'Guin added a comment -

            This is a pretty serious limitation. Just to evaluate Bamboo I'm going to need to create a separate AWS account with its own VPC, then setup a peering connection with my current VPC just to be able to use private resources during builds.

            Or I have to put Bamboo on a server and manage the agents myself.

            It shouldn't take 3 years to document how Elastic Bamboo uses my AWS account and what permissions it needs.

            Tim O'Guin added a comment - This is a pretty serious limitation. Just to evaluate Bamboo I'm going to need to create a separate AWS account with its own VPC, then setup a peering connection with my current VPC just to be able to use private resources during builds. Or I have to put Bamboo on a server and manage the agents myself. It shouldn't take 3 years to document how Elastic Bamboo uses my AWS account and what permissions it needs.

            Also, for our environment, it is unacceptable to grant full permissions. Even full EC2 permissions would be unacceptable.

            I will have to use a different product until a minimal set of IAM policies is defined and documented.

            Michael Sparks added a comment - Also, for our environment, it is unacceptable to grant full permissions. Even full EC2 permissions would be unacceptable. I will have to use a different product until a minimal set of IAM policies is defined and documented.

            Dmitri K added a comment -

            This helps somewhat - at least there are possible permissions listed.
            Still, I would recommend

            • making Bamboo more IAM-aware - otherwise it just raises eyebrows - and setting up a whole new (sub)account could be an obstacle or at least an inconvenience - it should only require minimal and specific permissions in limited scope
            • having clear documentation on which permissions are needed for what features - e.g. I was able to get it basically working with ReadOnly + select EC2 operations
            • improving error messages when operations fail, to indicate what was attempted, and what permission is needed - e.g. in the absence of a terminate permission, it will silently accept a terminate request and nothing happens
              Show that Atlassian is security-conscious!

            Dmitri K added a comment - This helps somewhat - at least there are possible permissions listed. Still, I would recommend making Bamboo more IAM-aware - otherwise it just raises eyebrows - and setting up a whole new (sub)account could be an obstacle or at least an inconvenience - it should only require minimal and specific permissions in limited scope having clear documentation on which permissions are needed for what features - e.g. I was able to get it basically working with ReadOnly + select EC2 operations improving error messages when operations fail, to indicate what was attempted, and what permission is needed - e.g. in the absence of a terminate permission, it will silently accept a terminate request and nothing happens Show that Atlassian is security-conscious!

            Sharing my workaround for this, I have set up a second AWS account with consolidated billing.
            This allows Bamboo to have as many rights as it wants within its own account without letting it touch other AWS resources.

            http://docs.aws.amazon.com/awsaccountbilling/latest/aboutv2/consolidated-billing.html

            Hope this helps people in a similar situation until Atlassian can provide IAM Policy guidance.

            Will Dowling added a comment - Sharing my workaround for this, I have set up a second AWS account with consolidated billing. This allows Bamboo to have as many rights as it wants within its own account without letting it touch other AWS resources. http://docs.aws.amazon.com/awsaccountbilling/latest/aboutv2/consolidated-billing.html Hope this helps people in a similar situation until Atlassian can provide IAM Policy guidance.

            I have voted for this, please give it higher priorty security is defiantly a concern. Why can't you publish a recommended policy?
            In the meantime I will take snippets from above and try and build my own as I suspect many others have had to do

            Rob Langley added a comment - I have voted for this, please give it higher priorty security is defiantly a concern. Why can't you publish a recommended policy? In the meantime I will take snippets from above and try and build my own as I suspect many others have had to do

            I don't understand how this issue could be rated "Minor" and get so few focus from Atlassian.
            @Atlassian: I started evaluating Bamboo today and I can tell I'm already moving to other products. There is no way I provide full access to Bamboo and no way I start injecting obscure/undocumented/unsupported rules/workaround into my infra.

            Vincent Bourdaraud added a comment - I don't understand how this issue could be rated "Minor" and get so few focus from Atlassian. @Atlassian: I started evaluating Bamboo today and I can tell I'm already moving to other products. There is no way I provide full access to Bamboo and no way I start injecting obscure/undocumented/unsupported rules/workaround into my infra.

            Hubert added a comment -

            @steve8 Ah, negative matches. Creative! Thanks

            Hubert added a comment - @steve8 Ah, negative matches. Creative! Thanks

            @hubez having the elastic agent capable of setting tags would circumvent the restrictions. Our IAM policy has a condition that asserts the ServiceLevel tag we use is not Production, QA, or Development. These tags are manually placed on each non bamboo instance which gives bamboo the permission to only terminate instances without that tag.

            Steve Buzonas added a comment - @hubez having the elastic agent capable of setting tags would circumvent the restrictions. Our IAM policy has a condition that asserts the ServiceLevel tag we use is not Production, QA, or Development. These tags are manually placed on each non bamboo instance which gives bamboo the permission to only terminate instances without that tag.

            Hubert added a comment -

            @steve8 Do you manually tag when an instance launches? Or do you have the Elastic Agent auto tag itself?

            I'm trying to figure out how to only let bamboo tag bamboo things, and not give permissions for it to tag ANY instance of mine. I still can't find a tight solution that restricts bamboo stuff to only bamboo without exposing the rest of my AWS infrastructure, in case my Bamboo API key ever gets leaked/stolen during a Bamboo breach.

            Hubert added a comment - @steve8 Do you manually tag when an instance launches? Or do you have the Elastic Agent auto tag itself? I'm trying to figure out how to only let bamboo tag bamboo things, and not give permissions for it to tag ANY instance of mine. I still can't find a tight solution that restricts bamboo stuff to only bamboo without exposing the rest of my AWS infrastructure, in case my Bamboo API key ever gets leaked/stolen during a Bamboo breach.

            Right now our current workaround is to utilize an existing tag that we put on all of our instances. We give an explicit deny to "*" when the resource has one of those tags.

            Steve Buzonas added a comment - Right now our current workaround is to utilize an existing tag that we put on all of our instances. We give an explicit deny to "*" when the resource has one of those tags.

            When is this going to be fixed. Atlassian wtf?!

            Jesse Sanford added a comment - When is this going to be fixed. Atlassian wtf?!

            I thought it would be the ID of the account you supply in Bamboo? I haven't tried it yet.

            Andrew Khoury added a comment - I thought it would be the ID of the account you supply in Bamboo? I haven't tried it yet.

            The owner of an instance is not the user who created it but the id of the account.
            So unfortunately it's not possible to use this solution.

            Deleted Account (Inactive) added a comment - The owner of an instance is not the user who created it but the id of the account. So unfortunately it's not possible to use this solution.

            I'm thinking a nicer solution would be to filter on the user rather than the server role, though I guess it comes down to personal preference.

            I think something like this might work:

            "Condition": {
            "StringEquals":

            { "ec2:Owner": "arn:aws:iam::xxxx:user/bamboo-elastic-user" }

            }

            Is there any official documentation on this?

            Andrew Khoury added a comment - I'm thinking a nicer solution would be to filter on the user rather than the server role, though I guess it comes down to personal preference. I think something like this might work: "Condition": { "StringEquals": { "ec2:Owner": "arn:aws:iam::xxxx:user/bamboo-elastic-user" } } Is there any official documentation on this?

            I think it's definitively a security issue :

            • I don't want bamboo to access all my instances.
            • I don't want someone who click on "Terminate" on a production instance.

            Adding an option "Instance Profile" (see doc) can be a good solution ?

            This option, can solve this security problem as it's possible to :

            • allow bamboo to create new instance with a specific Instance Profile (see tutorial)
            • Allow * to bamboo on all instances with the given Instance Profile.

            Deleted Account (Inactive) added a comment - I think it's definitively a security issue : I don't want bamboo to access all my instances. I don't want someone who click on "Terminate" on a production instance. Adding an option "Instance Profile" ( see doc ) can be a good solution ? This option, can solve this security problem as it's possible to : allow bamboo to create new instance with a specific Instance Profile ( see tutorial ) Allow * to bamboo on all instances with the given Instance Profile.

            We had this working. Broke a couple of days ago. Turns out a new permission is required:
            ec2:DescribeAccountAttributes

            FYI for those it might help.

            Lucas Nelson added a comment - We had this working. Broke a couple of days ago. Turns out a new permission is required: ec2:DescribeAccountAttributes FYI for those it might help.

            This is a little crazy, unless I am missing something. Right now, my Bamboo instance refuses to boot any instances because the other ~100 or so are already running my production environment. I'd like to set the permission so Bamboo can't even SEE my non-Bamboo boxes.

            Is there a way to do this that is sane?

            The permissions on AWS don't work that way, unfortunately - they won't filter the instance list. The only way to handle your situation is to bump up the limit of uncontrolled instances in Bamboo. I guess if you're a heavy EC2 user, you know that you need to monitor your account anyway. I've opened an improvement request BAM-14022 to handle your case more gracefully.

            Przemek Bruski added a comment - This is a little crazy, unless I am missing something. Right now, my Bamboo instance refuses to boot any instances because the other ~100 or so are already running my production environment. I'd like to set the permission so Bamboo can't even SEE my non-Bamboo boxes. Is there a way to do this that is sane? The permissions on AWS don't work that way, unfortunately - they won't filter the instance list. The only way to handle your situation is to bump up the limit of uncontrolled instances in Bamboo. I guess if you're a heavy EC2 user, you know that you need to monitor your account anyway. I've opened an improvement request BAM-14022 to handle your case more gracefully.

            This is a little crazy, unless I am missing something. Right now, my Bamboo instance refuses to boot any instances because the other ~100 or so are already running my production environment. I'd like to set the permission so Bamboo can't even SEE my non-Bamboo boxes.

            Is there a way to do this that is sane?

            Leads Account added a comment - This is a little crazy, unless I am missing something. Right now, my Bamboo instance refuses to boot any instances because the other ~100 or so are already running my production environment. I'd like to set the permission so Bamboo can't even SEE my non-Bamboo boxes. Is there a way to do this that is sane?

            Przemek Bruski added a comment - - edited

            This is the full list of AWS actions used by Bamboo:

            {
                "Version": "2012-10-17",
                "Statement": [
                    {
                        "Sid": "Stmt1379467306333",
                        "Action": [
                            "ec2:AttachVolume",
                            "ec2:AuthorizeSecurityGroupIngress",
                            "ec2:CancelSpotInstanceRequests",
                            "ec2:CreateKeyPair",
                            "ec2:CreateSecurityGroup",
                            "ec2:CreateTags",
                            "ec2:CreateVolume",
                            "ec2:DeleteVolume",
                            "ec2:DescribeAccountAttributes",
                            "ec2:DescribeAddresses",
                            "ec2:DescribeAvailabilityZones",
                            "ec2:DescribeImages",
                            "ec2:DescribeInstances",
                            "ec2:DescribeKeyPairs",
                            "ec2:DescribeRegions",
                            "ec2:DescribeSecurityGroups",
                            "ec2:DescribeSpotInstanceRequests",
                            "ec2:DescribeSpotPriceHistory",
                            "ec2:DescribeSubnets",
                            "ec2:DescribeVolumes",
                            "ec2:DescribeVpcs",
                            "ec2:GetConsoleOutput",
                            "ec2:ModifyInstanceAttribute",
                            "ec2:RequestSpotInstances",
                            "ec2:RunInstances",
                            "ec2:TerminateInstances",
                            "s3:GetBucketLocation",
                            "s3:GetObject",
                            "s3:ListBucket"
                        ],
                        "Effect": "Allow",
                        "Resource": "*"
                    }
                ]
            }
            

            Przemek Bruski added a comment - - edited This is the full list of AWS actions used by Bamboo: { "Version": "2012-10-17", "Statement": [ { "Sid": "Stmt1379467306333", "Action": [ "ec2:AttachVolume", "ec2:AuthorizeSecurityGroupIngress", "ec2:CancelSpotInstanceRequests", "ec2:CreateKeyPair", "ec2:CreateSecurityGroup", "ec2:CreateTags", "ec2:CreateVolume", "ec2:DeleteVolume", "ec2:DescribeAccountAttributes", "ec2:DescribeAddresses", "ec2:DescribeAvailabilityZones", "ec2:DescribeImages", "ec2:DescribeInstances", "ec2:DescribeKeyPairs", "ec2:DescribeRegions", "ec2:DescribeSecurityGroups", "ec2:DescribeSpotInstanceRequests", "ec2:DescribeSpotPriceHistory", "ec2:DescribeSubnets", "ec2:DescribeVolumes", "ec2:DescribeVpcs", "ec2:GetConsoleOutput", "ec2:ModifyInstanceAttribute", "ec2:RequestSpotInstances", "ec2:RunInstances", "ec2:TerminateInstances", "s3:GetBucketLocation", "s3:GetObject", "s3:ListBucket" ], "Effect": "Allow", "Resource": "*" } ] }

            Found the correct IAM policy that works for our use case (1 instance, using EBS volume, not using Spot for now):

            {
              "Statement": [
                {
                  "Sid": "Stmt1379467306333",
                  "Action": [
                    "ec2:AllocateAddress",
                    "ec2:AssociateAddress",
                    "ec2:AssociateDhcpOptions",
                    "ec2:AttachNetworkInterface",
                    "ec2:AttachVolume",
                    "ec2:CancelSpotInstanceRequests",
                    "ec2:ConfirmProductInstance",
                    "ec2:CreateNetworkInterface",
                    "ec2:CreateTags",
                    "ec2:DescribeAddresses",
                    "ec2:DescribeAvailabilityZones",
                    "ec2:DescribeBundleTasks",
                    "ec2:DescribeConversionTasks",
                    "ec2:DescribeExportTasks",
                    "ec2:DescribeImageAttribute",
                    "ec2:DescribeImages",
                    "ec2:DescribeInstanceAttribute",
                    "ec2:DescribeInstanceStatus",
                    "ec2:DescribeInstances",
                    "ec2:DescribeKeyPairs",
                    "ec2:DescribePlacementGroups",
                    "ec2:DescribeRegions",
                    "ec2:DescribeReservedInstances",
                    "ec2:DescribeReservedInstancesOfferings",
                    "ec2:DescribeSecurityGroups",
                    "ec2:DescribeSnapshotAttribute",
                    "ec2:DescribeSnapshots",
                    "ec2:DescribeSpotDatafeedSubscription",
                    "ec2:DescribeSpotInstanceRequests",
                    "ec2:DescribeSpotPriceHistory",
                    "ec2:DescribeSubnets",
                    "ec2:DescribeTags",
                    "ec2:DescribeVolumeAttribute",
                    "ec2:DescribeVolumeStatus",
                    "ec2:DescribeVolumes",
                    "ec2:GetConsoleOutput",
                    "ec2:ImportInstance",
                    "ec2:ImportKeyPair",
                    "ec2:ImportVolume",
                    "ec2:MonitorInstances",
                    "ec2:RegisterImage",
                    "ec2:ReportInstanceStatus",
                    "ec2:RequestSpotInstances",
                    "ec2:RunInstances",
                    "ec2:StartInstances"
                  ],
                  "Effect": "Allow",
                  "Resource": "*"
                },
                 {
                  "Effect":"Allow",
                  "Action":"ec2:*",
                  "Resource":"*",
                  "Condition": { 
                    "StringEquals": { 
                      "ec2:ResourceTag/ServerRole": "ElasticBamboo"
                    }
                  }
                }
              ]
            }

            The instance created need to have the tag "ServerRole=ElasticBamboo" set, otherwise you won't be able to terminate it.

            You can use the following command to get the instance to tag itself:

            ec2-create-tags $(GET http://169.254.169.254/latest/meta-data/instance-id) -O $AWS_ACCESS_KEY -W $AWS_SECRET_KEY --tag "ServerRole=ElasticBamboo"

            You'll need to replace the AWS keys with your key (can't query the meta-data for that unfortunately because we have to go via a IAM User instead of a Role...).

            Hope it helps whoever has the problem.
            Atlassian dudes might want to update their documentation too...

            Mathieu Debieuvre added a comment - Found the correct IAM policy that works for our use case (1 instance, using EBS volume, not using Spot for now): { "Statement": [ { "Sid": "Stmt1379467306333", "Action": [ "ec2:AllocateAddress", "ec2:AssociateAddress", "ec2:AssociateDhcpOptions", "ec2:AttachNetworkInterface", "ec2:AttachVolume", "ec2:CancelSpotInstanceRequests", "ec2:ConfirmProductInstance", "ec2:CreateNetworkInterface", "ec2:CreateTags", "ec2:DescribeAddresses", "ec2:DescribeAvailabilityZones", "ec2:DescribeBundleTasks", "ec2:DescribeConversionTasks", "ec2:DescribeExportTasks", "ec2:DescribeImageAttribute", "ec2:DescribeImages", "ec2:DescribeInstanceAttribute", "ec2:DescribeInstanceStatus", "ec2:DescribeInstances", "ec2:DescribeKeyPairs", "ec2:DescribePlacementGroups", "ec2:DescribeRegions", "ec2:DescribeReservedInstances", "ec2:DescribeReservedInstancesOfferings", "ec2:DescribeSecurityGroups", "ec2:DescribeSnapshotAttribute", "ec2:DescribeSnapshots", "ec2:DescribeSpotDatafeedSubscription", "ec2:DescribeSpotInstanceRequests", "ec2:DescribeSpotPriceHistory", "ec2:DescribeSubnets", "ec2:DescribeTags", "ec2:DescribeVolumeAttribute", "ec2:DescribeVolumeStatus", "ec2:DescribeVolumes", "ec2:GetConsoleOutput", "ec2:ImportInstance", "ec2:ImportKeyPair", "ec2:ImportVolume", "ec2:MonitorInstances", "ec2:RegisterImage", "ec2:ReportInstanceStatus", "ec2:RequestSpotInstances", "ec2:RunInstances", "ec2:StartInstances" ], "Effect": "Allow", "Resource": "*" }, { "Effect":"Allow", "Action":"ec2:*", "Resource":"*", "Condition": { "StringEquals": { "ec2:ResourceTag/ServerRole": "ElasticBamboo" } } } ] } The instance created need to have the tag "ServerRole=ElasticBamboo" set, otherwise you won't be able to terminate it. You can use the following command to get the instance to tag itself: ec2-create-tags $(GET http://169.254.169.254/latest/meta-data/instance-id) -O $AWS_ACCESS_KEY -W $AWS_SECRET_KEY --tag "ServerRole=ElasticBamboo" You'll need to replace the AWS keys with your key (can't query the meta-data for that unfortunately because we have to go via a IAM User instead of a Role...). Hope it helps whoever has the problem. Atlassian dudes might want to update their documentation too...

              Unassigned Unassigned
              4266f0f1452d Chris Spradlin
              Votes:
              154 Vote for this issue
              Watchers:
              116 Start watching this issue

                Created:
                Updated: