Uploaded image for project: 'Bamboo Data Center'
  1. Bamboo Data Center
  2. BAM-4430

manual and scheduled shutdown of elastic instance fails to delete attached ebs volumes

    XMLWordPrintable

Details

    • Bug
    • Resolution: Obsolete
    • Medium
    • 2.7
    • 2.3.1
    • Elastic Bamboo
    • None

    Description

      EBS volumes cannot be detached/deleted when instances are shut down by the cron scheduler.

      INFO | jvm 1 | 2009/08/24 07:40:00 | 2009-08-24 07:40:00,006 INFO [QuartzScheduler_Worker-6] [ElasticFunctionalityFacadeImpl] Adjusting elastic agents with schedule: com.atlassian.bamboo.agent.elastic.schedule.ElasticInstanceScheduleImpl@26e492[25886722,0 40 7 ? * MON-FRI,<null>,EQUALS,0,true]
      INFO | jvm 1 | 2009/08/24 07:40:00 | 2009-08-24 07:40:00,006 INFO [QuartzScheduler_Worker-6] [ElasticFunctionalityFacadeImpl] Adjusting all elastic instance...
      INFO | jvm 1 | 2009/08/24 07:40:00 | 2009-08-24 07:40:00,023 INFO [QuartzScheduler_Worker-6] [ElasticFunctionalityFacadeImpl] Attempting to shutdown 1 of 'Default' elastic instances
      INFO | jvm 1 | 2009/08/24 07:40:01 | 2009-08-24 07:40:01,264 INFO [BAM::Events:pool-1-thread-13] [ElasticInstanceManagerImpl] Elastic Agent "Elastic Agent on i-6ad13f02" stopped on instance i-6ad13f02
      INFO | jvm 1 | 2009/08/24 07:40:01 | 2009-08-24 07:40:01,277 INFO [pool-3-thread-3] [ElasticInstanceManagerImpl] Elastic instance i-6ad13f02 transitioned from RUNNING to SHUTTING_DOWN.
      INFO | jvm 1 | 2009/08/24 07:40:01 | 2009-08-24 07:40:01,287 INFO [BAM::Events:pool-1-thread-7] [ElasticInstanceManagerImpl] Elastic Agent "Elastic Agent on i-6ad13f02" stopped on instance i-6ad13f02
      INFO | jvm 1 | 2009/08/24 07:40:01 | 2009-08-24 07:40:01,632 INFO [pool-3-thread-3] [RemoteEC2InstanceImpl] EC2 instance i-6ad13f02 transitioned from running (16) to shutting-down (32)
      INFO | jvm 1 | 2009/08/24 07:40:01 | 2009-08-24 07:40:01,633 INFO [BAM::Events:pool-1-thread-7] [ElasticInstanceManagerImpl] Requested termination of elastic instance: i-6ad13f02
      INFO | jvm 1 | 2009/08/24 07:40:01 | 2009-08-24 07:40:01,633 INFO [BAM::Events:pool-1-thread-13] [ElasticInstanceManagerImpl] Requested termination of elastic instance: i-6ad13f02
      INFO | jvm 1 | 2009/08/24 07:40:25 | 2009-08-24 07:40:25,833 INFO [pool-3-thread-3] [RemoteEC2InstanceImpl] EC2 instance i-6ad13f02 has terminated.
      INFO | jvm 1 | 2009/08/24 07:40:25 | 2009-08-24 07:40:25,833 INFO [pool-3-thread-3] [ElasticInstanceManagerImpl] Elastic instance i-6ad13f02 transitioned from SHUTTING_DOWN to TERMINATED.
      INFO | jvm 1 | 2009/08/24 07:40:25 | 2009-08-24 07:40:25,833 INFO [pool-3-thread-3] [ElasticInstanceManagerImpl] Detected that the elastic instance i-6ad13f02 has stopped.
      INFO | jvm 1 | 2009/08/24 07:40:25 | 2009-08-24 07:40:25,836 INFO [pool-3-thread-3] [EBSVolumeSupervisorImpl] Deleting EBS volume vol-c90ffaa0
      INFO | jvm 1 | 2009/08/24 07:40:26 | 2009-08-24 07:40:26,036 WARN [pool-3-thread-3] [EBSVolumeImpl] Attempt to detach EBS volume vol-c90ffaa0 from EC2 instance i-6ad13f02 failed. Proceeding with deletion.
      INFO | jvm 1 | 2009/08/24 07:40:26 | com.xerox.amazonws.ec2.EC2Exception: Client error : The volume 'vol-c90ffaa0' is not 'attached'.
      INFO | jvm 1 | 2009/08/24 07:40:26 | at com.xerox.amazonws.ec2.Jec2.makeRequestInt(Jec2.java:1680)
      INFO | jvm 1 | 2009/08/24 07:40:26 | at com.xerox.amazonws.ec2.Jec2.detachVolume(Jec2.java:1569)
      INFO | jvm 1 | 2009/08/24 07:40:26 | at com.atlassian.aws.ec2.EBSVolumeImpl.delete(EBSVolumeImpl.java:38)
      INFO | jvm 1 | 2009/08/24 07:40:26 | at com.atlassian.bamboo.agent.elastic.server.EBSVolumeSupervisorImpl.purge(EBSVolumeSupervisorImpl.java:119)
      INFO | jvm 1 | 2009/08/24 07:40:26 | at com.atlassian.bamboo.agent.elastic.server.RemoteElasticInstanceImpl$1.ec2InstanceStateChanged(RemoteElasticInstanceImpl.java:307)
      INFO | jvm 1 | 2009/08/24 07:40:26 | at com.atlassian.aws.ec2.RemoteEC2InstanceImpl$4.run(RemoteEC2InstanceImpl.java:518)
      INFO | jvm 1 | 2009/08/24 07:40:26 | at com.atlassian.aws.ec2.RemoteEC2InstanceImpl$CatchingRunnableDecorator.run(RemoteEC2InstanceImpl.java:98)
      INFO | jvm 1 | 2009/08/24 07:40:26 | at com.atlassian.aws.ec2.RemoteEC2InstanceImpl.setState(RemoteEC2InstanceImpl.java:513)
      INFO | jvm 1 | 2009/08/24 07:40:26 | at com.atlassian.aws.ec2.RemoteEC2InstanceImpl.terminated(RemoteEC2InstanceImpl.java:346)
      INFO | jvm 1 | 2009/08/24 07:40:26 | at com.atlassian.aws.ec2.EC2InstanceState$3.supervise(EC2InstanceState.java:125)
      INFO | jvm 1 | 2009/08/24 07:40:26 | at com.atlassian.aws.ec2.RemoteEC2InstanceImpl.backgroundSupervise(RemoteEC2InstanceImpl.java:437)
      INFO | jvm 1 | 2009/08/24 07:40:26 | at com.atlassian.aws.ec2.RemoteEC2InstanceImpl.access$300(RemoteEC2InstanceImpl.java:25)
      INFO | jvm 1 | 2009/08/24 07:40:26 | at com.atlassian.aws.ec2.RemoteEC2InstanceImpl$2.run(RemoteEC2InstanceImpl.java:127)
      INFO | jvm 1 | 2009/08/24 07:40:26 | at com.atlassian.aws.ec2.RemoteEC2InstanceImpl$CatchingRunnableDecorator.run(RemoteEC2InstanceImpl.java:98)
      INFO | jvm 1 | 2009/08/24 07:40:26 | at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
      INFO | jvm 1 | 2009/08/24 07:40:26 | at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
      INFO | jvm 1 | 2009/08/24 07:40:26 | at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
      INFO | jvm 1 | 2009/08/24 07:40:26 | at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
      INFO | jvm 1 | 2009/08/24 07:40:26 | at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
      INFO | jvm 1 | 2009/08/24 07:40:26 | at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
      INFO | jvm 1 | 2009/08/24 07:40:26 | at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:885)
      INFO | jvm 1 | 2009/08/24 07:40:26 | at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
      INFO | jvm 1 | 2009/08/24 07:40:26 | at java.lang.Thread.run(Thread.java:619)

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              ukuhnhardt Ulrich Kuhnhardt [Atlassian]
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: