Hi 257cb8fcc776 ,

As discussed in the comment, we have released a variable BITBUCKET_SSH_KEY_FILE to the provide the path to the ssh private key file.

Please refer our documentation for more details.

Regards,
Jayant

I am closing this ticket since BuildKit is available now for everyone to use.
Thank you everyone for the support.

Hi Egor Yurtaev.

Persisting mounted build caches across diferent CI agents is very interesting but is currently an open issue for docker buildkit: https://github.com/moby/buildkit/issues/1512

I tried to outsmart the pipelines and setup the buildx CLI plugin like

pipelines:
  default:
    - step:
        services: [docker]
        script:
          - wget --no-verbose https://github.com/docker/buildx/releases/latest/download/buildx-v0.8.2.linux-amd64 --output-document ~/.docker/cli-plugins/docker-buildx
          - chmod a+x ~/.docker/cli-plugins/docker-buildx
          - docker buildx create --use
          - docker buildx build . --pull --cache-from registry.my.org/foo/bar --build-arg BUILDKIT_INLINE_CACHE=1 --progress=plain 

but this fails with

WARNING: No output specified for docker-container driver. Build result will only remain in the build cache. To push result image into registry use --push or to load image into docker use --load
#1 [internal] booting buildkit
#1 pulling image moby/buildkit:buildx-stable-1
#1 pulling image moby/buildkit:buildx-stable-1 2.3s done
#1 creating container buildx_buildkit_busy_haibt0 done
#1 ERROR: Error response from daemon: authorization denied by plugin pipelines: --privileged=true is not allowed
------
 > [internal] booting buildkit:
------
error: Error response from daemon: authorization denied by plugin pipelines: --privileged=true is not allowed 

Fat chance.

I never requested a privileged container, though. Am I doing something wrong?

Hi Raul,

Tthank you for looking into this! Is there's a corresponding public Jira ticket I could watch? Or will this be included in work for this feature?

Cheers,

JJ

Hi 257cb8fcc776 ,

Thanks for your feedback. I agree with your point, makes total sense. We still need to think about it properly, but it'd probably make sense to provide an environment variable with the path to the configured SSH key, so that also the configuration experience between Cloud and self-hosted is similar. I'll create a ticket in our backlog to look into that.

Regards,

Raul

Hi Raul,

I have managed to successfully use BuildKit but I have found to my dismay that on Runners (at least the Linux Runner) the SSH key is not present in /opt/atlassian/pipelines/agent/ssh/id_rsa but instead it's in the following directory:

/tmp/{runner-uuid}/ssh/id_rsa

This is problematic as the pipeline could run on any runner so UUID will change, and in addition the runner UUID is not exposed as an environment variable during the build process. As highlighted by Patrick Mead in an earlier comment, the only way currently to discover where the key is is to run something like this:

$(head -1 ~/.ssh/config | cut -f2 -d " ") # This isn't ideal as it depends heavily on how bitbucket sets the ssh config, if it changes builds will stop working 

Which is a fairly brittle method.

Would it be possible to do either of the following:

• symlink /tmp/{runner-uuid}/ssh/ directory during the build setup to /opt/atlassian/pipelines/agent/ssh/ if it detects it's on a runner?

• provide SSH directory as an environment variable during the build?
• provide runner UUID, as an environment variable during the build?

Hi 126a025855f3 , 

Thanks for your feedback. Unfortunately, since we provide a fresh environment for every build, that particular cache isn't persisted between steps, and could also be subject to GC clean-up. In order to get the best experience speeding up your builds and caching Docker layers (with or without Buildkit) I'd recommend using --cache-from, which allows to specify one or more tagged images as a cache source, using your own registry as the cache source between steps. This might also help to improve the performance of your build and save build minutes, without having the limitation of 1GB that our own implementation of docker caching currently has.

For example:

docker build \
--cache-from $IMAGE:latest \
--tag $IMAGE:$BITBUCKET_BUILD_NUMBER \
--tag $IMAGE:latest \
--file ./Dockerfile \
--build-arg BUILDKIT_INLINE_CACHE=1 \
"."

You can read the official docker documentation for more details. We'll clarify this to our public docs so that other users could benefit.

Thanks again for your feedback,  

Raul

Hi Raul, this is good news. But cache doesn't work between pipeline jobs if I use something like this:

RUN --mount=type=cache,id=yarn,target=/app/root/.yarn/cache yarn install --immutable

 
 Pipelines only cached docker layouts without buildkit cache

Hi everyone, 

Good news! we have rolled out an early access version of BuildKit support. We'd love you to try and share your feedback.

To use Buildkit, set the DOCKER_BUILDKIT=1 environment variable in the pipeline configuration (bitbucket-pipelines.yml) or as a workspace / repository variable. For example:

DOCKER_BUILDKIT=1 docker image build -t my/image:tag . 

There are some considerations and limitations when using Buildkit in Bitbucket Pipelines:

• Due to security reasons, we don't currently support -network=host or -security=insecure. 
• In order to use Buildkit SSH forwarding in Bitbucket Pipelines, we recommend using the path to the SSH key available in the build filesystem. For Bitbucket Pipelines SSH configured keys, use the path /opt/atlassian/pipelines/agent/ssh/id_rsa as shown in the following example:

DOCKER_BUILDKIT=1 docker image build -t my/image:tag --ssh default=/opt/atlassian/pipelines/agent/ssh/id_rsa --no-cache . 

For more information on Docker BuildKit, visit: Docker Docs — Build images with BuildKit.

Thanks for all the interest and feedback on this issue so far.

Regards,

Raul