Bamboo throws "Failed to create EC2 security group" when it tries to start EC2 instance

@4000000052510137297b29dc Caused by: com.atlassian.aws.AWSException: Failed to create EC2 security group.
@4000000052510137297b2dc4       at com.atlassian.aws.AWSAccountImpl.newSecurityGroup(AWSAccountImpl.java:402)
@4000000052510137297b2dc4       at com.atlassian.bamboo.agent.elastic.server.ElasticFunctionalityFacadeImpl.ensureSecurityGroupsExist(ElasticFunctionalityFacadeImpl.java:258)
@4000000052510137297b31ac       at com.atlassian.bamboo.agent.elastic.server.ElasticFunctionalityFacadeImpl.startupAgents(ElasticFunctionalityFacadeImpl.java:153)
@4000000052510137297b973c       ... 263 more
@4000000052510137297b973c Caused by: AmazonServiceException: Status Code: 400, AWS Service: AmazonEC2, AWS Request ID: a97d9332-5e71-4964-9f0f-5ebe9aa35d1e, AWS Error Code: InvalidGroup.Duplicate, AWS Error Message: The security group 'elasticbamboo' already exists for VPC 'vpc-4693992d'
@4000000052510137297ba6dc       at com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:644)
@4000000052510137297ba6dc       at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:338)