-
Suggestion
-
Resolution: Unresolved
-
None
-
OnDemand & Server
-
5
-
81
-
Our product teams collect and evaluate feedback from a number of different sources. To learn more about how we use customer feedback in the planning process, check out our new feature policy.
Currently Elastic Bamboo requires you give it your account level credentials to run properly. This isn't really acceptable in many environments and we should be able to use IAM to create a bamboo user with restricted permission (i.e. bamboo doesn't need permissions to create a new VPC, or delete non-bamboo instances).
To me this is a major security problem. The response so far from support was this:
"After confirming with the Bamboo seniors, I can say that the Bamboo user does need full access to the account. We also recommend that a dedicated account be created for Bamboo to ensure that, should Bamboo "go sideways," any damage is restricted only to the Bamboo user's account."
Running a dedicated account just for bamboo isn't horribly practical if you have other resources the builds depend on, especially in a VPC environment.
- mentioned in
-
Page Failed to load
-
Page Failed to load
-
Page Failed to load
-
Page Failed to load
-
Page Failed to load
- relates to
-
BDEV-10295 Loading...
-
BDEV-16385 Loading...
- was cloned as
-
BDEV-9600 Loading...
Form Name |
---|
[BAM-11932] Bamboo requires full permissions to EC2
I strongly support for the IAM Role policy to be used instead of Access keys/Secret Keys
It's been over 7 years and we still don't have a set of permissions that Bamboo needs? How, just how.
Hi everyone,
Thanks for your interest in this issue.
This request is considered a potential addition to our longer-term roadmap.
We'll typically review this request in about 6 months time, at which point we’ll consider whether we need to alter its status.
For the nearest future we've decided to prioritise other areas of the Bamboo roadmap, including:
- Robustness of Plan Branches
- Performance and stability improvements
- Providing building blocks for High Availability and Disaster Recovery solutions
- Improving permission system
- Allowing per-project allocation of resources
- Improving Bitbucket Server and Jira integrations
You can learn more about our approach to highly voted server suggestions here.
To learn more on how your suggestions are reviewed, see our updated workflow for server feature suggestions.
Kind regards,
Bamboo Team
I believe it probably is. I don't think there's much interest in resolving this issue since it's almost 7 years old...
You would think they would at least have the guts to say "we won't do this".
+1
Please add support for EC2 IAM Role / Instance Profile based permissions for Bamboo.
AWS do not recommend to use IAM Users with Key/Secret in production environments - everything should be based on IAM Roles and federated access.
Lack of this feature is a showstopper for adoption of Bamboo.
The lack of proper support for EC2 is almost a show-stopper for us.
I'm thinking we may need to junk Bamboo in favor of Jenkins.
Maybe that's really what Atlassian wants anyway.
Atlassian this really screws with our integration here. You are basically making it impossible to migrate to elastic bamboo. I don't think this feature is production ready and should probably be taken off the features list
The problem with the restrictions based on the tag name is that the bamboo server also has the createTag privilege, which means a malicious user or a nasty bug can basically retag all the instances and terminate them
Another approach we are considering to use is to limit the dangerous operation to EC2 instances which have a predefined IAM role. Same security problem applies, but at least it's harder to exploit.
This should be an out of the box thing. Any ETA on when this will be limited?
@Oliver I was able to compose the policy below from policies shared here and from Atlassian Answers.
{ "Version": "2012-10-17", "Statement": [ { "Sid": "AllowBambooToStartNewBuildAgents", "Effect": "Allow", "Action": [ "ec2:Describe*", "ec2:RequestSpot*", "ec2:CancelSpot*", "ec2:AllocateAddress", "ec2:AssociateAddress", "ec2:AttachVolume", "ec2:AuthorizeSecurityGroupIngress", "ec2:CancelSpotInstanceRequests", "ec2:CreateKeyPair", "ec2:CreateSecurityGroup", "ec2:CreateTags", "ec2:CreateVolume", "ec2:DeleteVolume", "ec2:DescribeAddresses", "ec2:DescribeAvailabilityZones", "ec2:DescribeImages", "ec2:DescribeInstances", "ec2:DescribeKeyPairs", "ec2:DescribeRegions", "ec2:DescribeSecurityGroups", "ec2:DescribeSpotInstanceRequests", "ec2:DescribeSpotPriceHistory", "ec2:DescribeSubnets", "ec2:DescribeVolumes", "ec2:DescribeVpcs", "ec2:GetConsoleOutput", "ec2:ModifyInstanceAttribute", "ec2:ReleaseAddress", "ec2:RequestSpotInstances", "ec2:RunInstances" ], "Resource": "*" }, { "Sid": "OnlyAllowBambooToTerminateBuildAgents", "Effect": "Allow", "Action": [ "ec2:TerminateInstances", "ec2:StopInstances", "ec2:StartInstances" ], "Resource": "arn:aws:ec2:[aws-region]:[aws-account-id]:instance/*", "Condition": { "StringEquals": { "ec2:ResourceTag/Name": "bam::[bamboo.hosted.domain]::js-bamboo" } } }, { "Sid": "AllowBambooAccessToTheBuildArtifactsS3Bucket", "Effect": "Allow", "Action": "s3:*", "Resource": [ "arn:aws:s3:::[bamboo-bucket-name]", "arn:aws:s3:::[bamboo-bucket-name]/*" ] } ] }
There may even be room to improve this even more, but its working for me as is.
If you are running Bamboo as part of the Atlassian Cloud offering then replace [bamboo.hosted.domain] with the [your.site.name].atlassian.net.
The official response from Atlassian is "We understand the security risk and we are working to make it work differently in the future, but at the moment this is a requirement."
This issue was first reported in 2012 so when exactly will this work differently?
Fixed it by adding "ec2:GetConsoleOutput" after "ec2:CancelSpot*".
@Daniel Thank you for the example, it works but I don't get logs anymore of my instance on the atlassian site. Could it be related to the policy?
I'm talking about this page: https://xxx.atlassian.net/builds/admin/elastic/viewElasticInstance.action?instanceId=xxx
I tried reaching Atlassian's support for a policy as well, but failed to get an official policy too, and all I could obtain were some links to the forum. After some fiddling, managed to get it working by myself with this policy:
{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": "iam:PassRole", "Resource": "arn:aws:iam::<account_number>:role/bamboo_role" }, { "Effect": "Allow", "Action": [ "ec2:Describe*", "ec2:CreateTags", "ec2:RunInstances", "ec2:RequestSpot*", "ec2:CancelSpot*" ], "Resource": "*" }, { "Effect": "Allow", "Action": "ec2:TerminateInstances", "Resource": "arn:aws:ec2:*:*:instance/*", "Condition": { "StringEquals": { "ec2:ResourceTag/Name": "bam::your.domain.com::js-bamboo" } } } ] }
It isn't perfect, but it works for now.
Over 3 years later and, even worse than not having an official IAM policy, Bamboo docs are still requesting the master account credentials.
Security is not an afterthought.
Now wonder why I couldn't find the IAM policy in Confluence for Bamboo... Atlassian hasn't released one.
I can see if EC2 was simply one integration out of many for Bamboo Cloud, but IT IS REQUIRED.
So, Atlassian should follow Amazon best practices, limiting access, and provide an IAM policy (and what each component is needed for in terms of product functionality).
What Atlassian has in terms of great products, they lack in Security (still can't believe 2FA is not in every product or cloud for that matter).
I'm not sure why this is not documented any better. Like Alex's example Amazon has an in depth article about how to limit an IAM user: https://blogs.aws.amazon.com/security/post/Tx1ZU3LW4LLPQY2/How-to-Help-Lock-Down-a-User-s-Amazon-EC2-Capabilities-to-a-Single-VPC.
I've tested it and you can get Bamboo to work with a limited role.
You can also limit it with a super simple AWS policy like:
{ "Version": "2012-10-17", "Statement":[{ "Effect":"Allow", "Action":["ec2:RunInstances", "ec2:Describe*"] "Resource":"*" }, { "Effect":"Deny", "NotAction":["ec2:StopInstances", "ec2:StartInstances", "ec2:TerminateInstances"], "Resource":"*" } ] }
This worked perfectly for me
+1
This practice is against AWS own guidelines, and its just basic security 101. You never give full access to an AWS user, if access to that user is ever compromised, its possible to bring down a hole system
Atlassian,
The first security recommendation from AWS documentation is to remove API access for users with full permissions including the admin.
You requirements/guidelines that the bamboo user needs full admin permissions is clearly not in line and does not appear thought through.
Please publish best practices guide how to isolate the secure a development cloud including users policies.
This was the IAM Policy we used to limit access to our EC2 infrastructure. You could probably tweak it further and lock it down a bit more.
{ "Version": "2012-10-17", "Statement": [ { "Sid": "BambooAllowedAll", "Effect": "Allow", "Action": [ "ec2:Describe*", "ec2:RequestSpot*", "ec2:CancelSpot*", "ec2:AllocateAddress", "ec2:AssociateAddress", "ec2:AttachVolume", "ec2:AuthorizeSecurityGroupIngress", "ec2:CancelSpotInstanceRequests", "ec2:CreateKeyPair", "ec2:CreateSecurityGroup", "ec2:CreateTags", "ec2:CreateVolume", "ec2:DeleteVolume", "ec2:DescribeAddresses", "ec2:DescribeAvailabilityZones", "ec2:DescribeImages", "ec2:DescribeInstances", "ec2:DescribeKeyPairs", "ec2:DescribeRegions", "ec2:DescribeSecurityGroups", "ec2:DescribeSpotInstanceRequests", "ec2:DescribeSpotPriceHistory", "ec2:DescribeSubnets", "ec2:DescribeVolumes", "ec2:DescribeVpcs", "ec2:GetConsoleOutput", "ec2:ModifyInstanceAttribute", "ec2:ReleaseAddress", "ec2:RequestSpotInstances", "ec2:RunInstances", "s3:GetObject", "s3:ListBucket" ], "Resource": "*" }, { "Sid": "BambooAllowedOnlyOnCreated", "Effect": "Allow", "Action": [ "ec2:TerminateInstances", "ec2:StopInstances", "ec2:StartInstances" ], "Resource": "arn:aws:ec2:eu-west-1:<YOUR_AWS_ACCOUNT_NUMBER>:instance/*", "Condition": { "StringEquals": { "ec2:ResourceTag/Name": "bam::<YOUR_JIRA_CLOUD_URL>.atlassian.net::js-bamboo" } } } ] }
+1. Excessive permissions (and completely missing official documentation on what permissions are required) completely rules out Bamboo for my organization.
This came to my attention when Atlasssian demanded that we move our artifacts to S3, and I worked back through the IAM account assigned. Being told to "Pound Sand" is not very helpful.
+1 Please resolve this. Presents a major security & compliance violation and really should not be so difficult.
Come on Atlassian. Seeing you all cut corners like this is a real big disappointment. This fundamentally rules out Bamboo Cloud for us.
From my question on this very issue: https://answers.atlassian.com/questions/16879320/iam-policy-permissions-for-bamboo-aws-requirements
One quick REAL WORLD EXAMPLE situation for you, as I know "hackers" seem impossible until it's too late for some people. So how about this situation:
An ATLASSIAN developer accidentally pushes a commit to production that effects the Bamboo automatic instance scaling system. Due to the difference of comma out of place in an array, the conditional evaluator for what instances should be TERMINATED when a bamboo agent stops running breaks. The system then believes all EC2 instances have had their agents shut down and it's way over it's max limit. So is terminates all EC2 instances.
How many customers would lose data do you think in this situation?
Improbable?... look here, this is on my screen right now:
Elastic Bamboo Error : EC2 Instance i-8xxxxxxx is in an unrecognised state: UNKNOWN: Failure in EC2 instance i-8xxxxxxx. Attempting to terminate.
Unfortunately I see this theme repeated throughout Atlassian products - very reasonable requests are ignored. Clearly an organisation with a culture of arrogance.
+1 and strongly agree that not publishing a recommended, least-privilege IAM policy is irresponsible and lazy.
This is a pretty serious limitation. Just to evaluate Bamboo I'm going to need to create a separate AWS account with its own VPC, then setup a peering connection with my current VPC just to be able to use private resources during builds.
Or I have to put Bamboo on a server and manage the agents myself.
It shouldn't take 3 years to document how Elastic Bamboo uses my AWS account and what permissions it needs.
Also, for our environment, it is unacceptable to grant full permissions. Even full EC2 permissions would be unacceptable.
I will have to use a different product until a minimal set of IAM policies is defined and documented.
This helps somewhat - at least there are possible permissions listed.
Still, I would recommend
- making Bamboo more IAM-aware - otherwise it just raises eyebrows - and setting up a whole new (sub)account could be an obstacle or at least an inconvenience - it should only require minimal and specific permissions in limited scope
- having clear documentation on which permissions are needed for what features - e.g. I was able to get it basically working with ReadOnly + select EC2 operations
- improving error messages when operations fail, to indicate what was attempted, and what permission is needed - e.g. in the absence of a terminate permission, it will silently accept a terminate request and nothing happens
Show that Atlassian is security-conscious!
Sharing my workaround for this, I have set up a second AWS account with consolidated billing.
This allows Bamboo to have as many rights as it wants within its own account without letting it touch other AWS resources.
http://docs.aws.amazon.com/awsaccountbilling/latest/aboutv2/consolidated-billing.html
Hope this helps people in a similar situation until Atlassian can provide IAM Policy guidance.
I have voted for this, please give it higher priorty security is defiantly a concern. Why can't you publish a recommended policy?
In the meantime I will take snippets from above and try and build my own as I suspect many others have had to do
I don't understand how this issue could be rated "Minor" and get so few focus from Atlassian.
@Atlassian: I started evaluating Bamboo today and I can tell I'm already moving to other products. There is no way I provide full access to Bamboo and no way I start injecting obscure/undocumented/unsupported rules/workaround into my infra.
@hubez having the elastic agent capable of setting tags would circumvent the restrictions. Our IAM policy has a condition that asserts the ServiceLevel tag we use is not Production, QA, or Development. These tags are manually placed on each non bamboo instance which gives bamboo the permission to only terminate instances without that tag.
@steve8 Do you manually tag when an instance launches? Or do you have the Elastic Agent auto tag itself?
I'm trying to figure out how to only let bamboo tag bamboo things, and not give permissions for it to tag ANY instance of mine. I still can't find a tight solution that restricts bamboo stuff to only bamboo without exposing the rest of my AWS infrastructure, in case my Bamboo API key ever gets leaked/stolen during a Bamboo breach.
Right now our current workaround is to utilize an existing tag that we put on all of our instances. We give an explicit deny to "*" when the resource has one of those tags.
I thought it would be the ID of the account you supply in Bamboo? I haven't tried it yet.
The owner of an instance is not the user who created it but the id of the account.
So unfortunately it's not possible to use this solution.
I'm thinking a nicer solution would be to filter on the user rather than the server role, though I guess it comes down to personal preference.
I think something like this might work:
"Condition": {
"StringEquals":
}
Is there any official documentation on this?
I think it's definitively a security issue :
- I don't want bamboo to access all my instances.
- I don't want someone who click on "Terminate" on a production instance.
Adding an option "Instance Profile" (see doc) can be a good solution ?
This option, can solve this security problem as it's possible to :
- allow bamboo to create new instance with a specific Instance Profile (see tutorial)
- Allow * to bamboo on all instances with the given Instance Profile.
We had this working. Broke a couple of days ago. Turns out a new permission is required:
ec2:DescribeAccountAttributes
FYI for those it might help.
This is a little crazy, unless I am missing something. Right now, my Bamboo instance refuses to boot any instances because the other ~100 or so are already running my production environment. I'd like to set the permission so Bamboo can't even SEE my non-Bamboo boxes.
Is there a way to do this that is sane?
The permissions on AWS don't work that way, unfortunately - they won't filter the instance list. The only way to handle your situation is to bump up the limit of uncontrolled instances in Bamboo. I guess if you're a heavy EC2 user, you know that you need to monitor your account anyway. I've opened an improvement request BAM-14022 to handle your case more gracefully.
This is a little crazy, unless I am missing something. Right now, my Bamboo instance refuses to boot any instances because the other ~100 or so are already running my production environment. I'd like to set the permission so Bamboo can't even SEE my non-Bamboo boxes.
Is there a way to do this that is sane?
This is the full list of AWS actions used by Bamboo:
{ "Version": "2012-10-17", "Statement": [ { "Sid": "Stmt1379467306333", "Action": [ "ec2:AttachVolume", "ec2:AuthorizeSecurityGroupIngress", "ec2:CancelSpotInstanceRequests", "ec2:CreateKeyPair", "ec2:CreateSecurityGroup", "ec2:CreateTags", "ec2:CreateVolume", "ec2:DeleteVolume", "ec2:DescribeAccountAttributes", "ec2:DescribeAddresses", "ec2:DescribeAvailabilityZones", "ec2:DescribeImages", "ec2:DescribeInstances", "ec2:DescribeKeyPairs", "ec2:DescribeRegions", "ec2:DescribeSecurityGroups", "ec2:DescribeSpotInstanceRequests", "ec2:DescribeSpotPriceHistory", "ec2:DescribeSubnets", "ec2:DescribeVolumes", "ec2:DescribeVpcs", "ec2:GetConsoleOutput", "ec2:ModifyInstanceAttribute", "ec2:RequestSpotInstances", "ec2:RunInstances", "ec2:TerminateInstances", "s3:GetBucketLocation", "s3:GetObject", "s3:ListBucket" ], "Effect": "Allow", "Resource": "*" } ] }
Found the correct IAM policy that works for our use case (1 instance, using EBS volume, not using Spot for now):
{ "Statement": [ { "Sid": "Stmt1379467306333", "Action": [ "ec2:AllocateAddress", "ec2:AssociateAddress", "ec2:AssociateDhcpOptions", "ec2:AttachNetworkInterface", "ec2:AttachVolume", "ec2:CancelSpotInstanceRequests", "ec2:ConfirmProductInstance", "ec2:CreateNetworkInterface", "ec2:CreateTags", "ec2:DescribeAddresses", "ec2:DescribeAvailabilityZones", "ec2:DescribeBundleTasks", "ec2:DescribeConversionTasks", "ec2:DescribeExportTasks", "ec2:DescribeImageAttribute", "ec2:DescribeImages", "ec2:DescribeInstanceAttribute", "ec2:DescribeInstanceStatus", "ec2:DescribeInstances", "ec2:DescribeKeyPairs", "ec2:DescribePlacementGroups", "ec2:DescribeRegions", "ec2:DescribeReservedInstances", "ec2:DescribeReservedInstancesOfferings", "ec2:DescribeSecurityGroups", "ec2:DescribeSnapshotAttribute", "ec2:DescribeSnapshots", "ec2:DescribeSpotDatafeedSubscription", "ec2:DescribeSpotInstanceRequests", "ec2:DescribeSpotPriceHistory", "ec2:DescribeSubnets", "ec2:DescribeTags", "ec2:DescribeVolumeAttribute", "ec2:DescribeVolumeStatus", "ec2:DescribeVolumes", "ec2:GetConsoleOutput", "ec2:ImportInstance", "ec2:ImportKeyPair", "ec2:ImportVolume", "ec2:MonitorInstances", "ec2:RegisterImage", "ec2:ReportInstanceStatus", "ec2:RequestSpotInstances", "ec2:RunInstances", "ec2:StartInstances" ], "Effect": "Allow", "Resource": "*" }, { "Effect":"Allow", "Action":"ec2:*", "Resource":"*", "Condition": { "StringEquals": { "ec2:ResourceTag/ServerRole": "ElasticBamboo" } } } ] }
The instance created need to have the tag "ServerRole=ElasticBamboo" set, otherwise you won't be able to terminate it.
You can use the following command to get the instance to tag itself:
ec2-create-tags $(GET http://169.254.169.254/latest/meta-data/instance-id) -O $AWS_ACCESS_KEY -W $AWS_SECRET_KEY --tag "ServerRole=ElasticBamboo"
You'll need to replace the AWS keys with your key (can't query the meta-data for that unfortunately because we have to go via a IAM User instead of a Role...).
Hope it helps whoever has the problem.
Atlassian dudes might want to update their documentation too...
https://jira.atlassian.com/browse/BAM-11932?focusedCommentId=2310432&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-2310432
This is absolutely correct. If the Bamboo server is running in AWS you should be able to use instance profiles which is not an option now.