Uploaded image for project: 'Bitbucket Cloud'
  1. Bitbucket Cloud
  2. BCLOUD-21664

Graceful shutdown of runner container doesn't work

    XMLWordPrintable

Details

    Description

      Issue Summary

      When we execute docker stop -t 120 <runner-container-name>, it doesn't shut down gracefully.

      When stopping a container, Docker will send SIGTERM immediately and SIGKILL if the container is still running after a delay. The argument for -t specifies how many seconds to wait between the two signals.

      SIGTERM tells the runner to shut down cleanly. Ideally, the runner should finish what it's currently doing and then shut down without requesting more work.

      Steps to Reproduce

      execute the following command on the runner hosts machine terminal, when no pipeline build is getting executed on the self-hosted runner

      docker stop -t 60 <runner-container-name>
      

      Expected Results

      • If the runner is idle it stops immediately
      • If the runner is busy it completes the current test pipeline and then stops
      • If the runner has not become idle after 60 seconds it stops unconditionally

      Actual Results

      • The runner does not shut down immediately if it is idle.
      • The runner does not stop next time it becomes idle.
      • After exactly 60 seconds the runner shuts down causing whatever test it's running at that time to fail.

      Workaround

      • Perform a manual stop on the container verifying that there are no builds running on the pipelines UI
      • With the use of Pipelines APIs, verify if there is a pipeline running on a particular repository and verify the status of the pipelines build and script to stop the runner container using `docker stop` command

      Attachments

        Activity

          People

            Unassigned Unassigned
            bkandula@atlassian.com Bharath
            Votes:
            8 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

              Created:
              Updated: