Uploaded image for project: 'Bamboo Data Center'
  1. Bamboo Data Center
  2. BAM-20276

Stopping build job on windows agent does not kill children of active docker task.

      Issue Summary

      Bamboo has the facility to stop a Build plan whilst it is executing. Stopping the active build should cause the agents running the build to stop any active task and terminate any child processes.
      If the Build plan contains a Docker task then the remote agent should terminate the docker run and any child processes that is spawned.
      On a Windows remote agent this does not appear to happen. Stopping the build does terminate the job and task but the child processes of the docker run are not stopped.

      Environment

      Bamboo 6.7.2

      • Linux server
      • Windows server 2016 (with docker installed) remote agent

      Steps to Reproduce

      1. Install bamboo 6.7.2 on Linux server
      2. Install windows server 2016 with docker installed
        • [I used Windows_Server-2016-English-Full-Containers-2019.01.10 (ami-0c123eec390f84734) on AWS]
      3. Create a build plan containing a Docker task.
        • Docker image: mcr.microsoft.com/windows/servercore
        • Container command: powershell -command Start-Sleep -s 10000
      4. Run job
      5. Stop job whilst it is executing the docker run task

      Expected Results

      When the job is stopped the remote agent should kill the child processes of the running tasks.

      Actual Results

      The below exception is thrown in the Build log but the child processes are not killed.
      The powershell -command Start-Sleep -s 10000 command continues to run.

      29-Jan-2019 03:49:19	Starting task 'Docker' of type 'com.atlassian.bamboo.plugins.bamboo-docker-plugin:task.docker.cli'
      29-Jan-2019 03:49:19	Running image name (mcr.microsoft.com/windows/servercore)
      29-Jan-2019 03:49:19	
      Beginning to execute external process for build 'tst - dock - Default Job #30 (TST-DOCK-JOB1-30)'
       ... running command line: 
      C:\Program Files\Docker\docker.exe run --rm mcr.microsoft.com/windows/servercore powershell -command Start-Sleep -s 10000
       ... in: C:\Users\Administrator\bamboo-agent-home\xml-data\build-dir\TST-DOCK-JOB1
       ... using extra environment variables: 
      ...
      29-Jan-2019 03:49:35	
      Request to stop 'TST-DOCK-JOB1-30' received from admin
      29-Jan-2019 03:49:35	Exit code: -1, output: STDOUT:  Beginning to execute external process for build 'tst - dock - Default Job #30 (TST-DOCK-JOB1-30)'\n ... running command line: \nC:\Program Files\Docker\docker.exe run --rm mcr.microsoft.com/windows/servercore powershell -command Start-Sleep -s 10000\n ... in: C:\Users\Administrator\bamboo-agent-home\xml-data\build-dir\TST-DOCK-JOB1\n ... ...
      29-Jan-2019 03:49:35	Force Stop build feature is enabled for current plan. Either Bamboo has detected the build has hung or it has been manually stopped.
      29-Jan-2019 03:49:35	Attempting to generate stack trace and terminate spawned sub-processes of process id: 5080
      29-Jan-2019 03:49:35	Error occurred while running Task '(2)' of type com.atlassian.bamboo.plugins.bamboo-docker-plugin:task.docker.cli.
      29-Jan-2019 03:49:35	com.atlassian.bamboo.task.TaskException: Failed to execute task
      29-Jan-2019 03:49:35	        at com.atlassian.bamboo.plugins.docker.service.RunService.execute(RunService.java:75)
      29-Jan-2019 03:49:35	        at com.atlassian.bamboo.plugins.docker.tasks.cli.DockerCliTask.execute(DockerCliTask.java:70)
      29-Jan-2019 03:49:35	        at com.atlassian.bamboo.task.TaskExecutorImpl.lambda$executeTasks$3(TaskExecutorImpl.java:319)
      29-Jan-2019 03:49:35	        at com.atlassian.bamboo.task.TaskExecutorImpl.executeTaskWithPrePostActions(TaskExecutorImpl.java:252)
      29-Jan-2019 03:49:35	        at com.atlassian.bamboo.task.TaskExecutorImpl.executeTasks(TaskExecutorImpl.java:319)
      29-Jan-2019 03:49:35	        at com.atlassian.bamboo.task.TaskExecutorImpl.execute(TaskExecutorImpl.java:112)
      ...
      29-Jan-2019 03:49:35	        at com.atlassian.bamboo.plugins.docker.client.DockerCmd.run(DockerCmd.java:51)
      29-Jan-2019 03:49:35	        at com.atlassian.bamboo.plugins.docker.service.RunService.execute(RunService.java:64)
      29-Jan-2019 03:49:35	        ... 17 more
      29-Jan-2019 03:49:35	Caused by: com.atlassian.utils.process.ProcessException: Error executing C:\Program Files\Docker\docker.exe run --rm mcr.microsoft.com/windows/servercore powershell -command Start-Sleep -s 10000
      29-Jan-2019 03:49:35	        at com.atlassian.bamboo.docker.DockerContainerServiceImpl.execute(DockerContainerServiceImpl.java:302)
      29-Jan-2019 03:49:35	        at com.atlassian.bamboo.docker.DockerContainerServiceImpl.execute(DockerContainerServiceImpl.java:272)
      29-Jan-2019 03:49:35	        at com.atlassian.bamboo.docker.DockerContainerServiceImpl.run(DockerContainerServiceImpl.java:106)
      29-Jan-2019 03:49:35	        ... 46 more
      29-Jan-2019 03:49:35	Running post build plugin 'Docker Container Cleanup'
      29-Jan-2019 03:49:35	getStackTraceAndKillRelatedProcesses for 0 processes
      ..
      

      Workaround

      Manually kill any left over processes.

              Unassigned Unassigned
              cberry@atlassian.com Chris Berry (Inactive)
              Affected customers:
              1 This affects my team
              Watchers:
              7 Start watching this issue

                Created:
                Updated: