Uploaded image for project: 'Bitbucket Cloud'
  1. Bitbucket Cloud
  2. BCLOUD-23353

Bitbucket Pipeline failing authentication requests against private Google Artifact Registry

      Issue Summary

      Bitbucket Pipeline make an unauthenticated calls and then authenticated calls to pull the image from private Google Artifact Registry even though the correct credentials are provided.

      Steps to Reproduce

      1. Specify a build image from private Google Artifact Registry in YAML
      2. Specify the valid credentials for private Google Artifact Registry

      Expected Results

      Bitbucket Pipeline should make only one call with the provided credentials to pull the image.

      Actual Results

      1. Bitbucket Pipeline first make aa unauthenticated call to pull the image
        then make another call to pull the image with the provided credentials and successfully pull the image

      Workaround

      This is not a blocker issue.
      The same issue is observed with Google Cloud Build
      The same issue is not observed while pulling the same image locally

            [BCLOUD-23353] Bitbucket Pipeline failing authentication requests against private Google Artifact Registry

            Google confirmed the cloud workstation image pull tries unauthenticated first so it doesn't leak credentials when they aren't needed.

            Brant Gurganus added a comment - Google confirmed the cloud workstation image pull tries unauthenticated first so it doesn't leak credentials when they aren't needed.

            So a possible correction on the Google Cloud Build behavior, the similar unauthenticated access before authenticated access behavior is when Google Computer Engine is pulling a docker image for a Google Cloud Workstation, not a build. In that scenario, trying unauthenticated first might make sense since you don't know if credentials (service account) are needed for that pull and making an unauthenticated request in that scenario might leak at least some authentication information.

            The Google Cloud Build does not have the unauthenticated request first and works first time.
            That particular scenario is a bit different than the Bitbucket pipeline scenario where you have the authentication information right there with the image to pull. It's not an implied authentication like it is with the Google Cloud service account situation. It's explicitly set on that image pull.
            See attached screenshot of Docker-HeadManifest requests. First entry is the Google Cloud Build fetching base image for a nightly scheduled rebuild... no unauthenticated request. Second and third requests are from the Google Cloud Workstation launch, the first one fails because an unauthenticated request is attempted, similar to Bitbucket's current behavior, but I don't believe Bitbucket's behavior is justified while Google's workstation image pull behavior may be.
            I put a Google ticket in to confirm that rationale on the behavior difference.

            Brant Gurganus added a comment - So a possible correction on the Google Cloud Build behavior, the similar unauthenticated access before authenticated access behavior is when Google Computer Engine is pulling a docker image for a Google Cloud Workstation, not a build. In that scenario, trying unauthenticated first might make sense since you don't know if credentials (service account) are needed for that pull and making an unauthenticated request in that scenario might leak at least some authentication information. The Google Cloud Build does not have the unauthenticated request first and works first time. That particular scenario is a bit different than the Bitbucket pipeline scenario where you have the authentication information right there with the image to pull. It's not an implied authentication like it is with the Google Cloud service account situation. It's explicitly set on that image pull. See attached screenshot of Docker-HeadManifest requests. First entry is the Google Cloud Build fetching base image for a nightly scheduled rebuild... no unauthenticated request. Second and third requests are from the Google Cloud Workstation launch, the first one fails because an unauthenticated request is attempted, similar to Bitbucket's current behavior, but I don't believe Bitbucket's behavior is justified while Google's workstation image pull behavior may be. I put a Google ticket in to confirm that rationale on the behavior difference.

            When testing service account authentication with Docker 27.1.1 build 6212585 with the service account token method at https://cloud.google.com/artifact-registry/docs/docker/authentication there was no extra unauthenticated calls, so Docker is using the authentication on all calls as expected and not making noisy failing calls as the pulls from Bitbucket are doing. If docker is what Bitbucket Pipelines are using, it seems something is causing initial unauthenticated calls. So version of docker may be a factor or a different tool like containerd is being used to pull the image and behaves differently.

            Brant Gurganus added a comment - When testing service account authentication with Docker 27.1.1 build 6212585 with the service account token method at https://cloud.google.com/artifact-registry/docs/docker/authentication there was no extra unauthenticated calls, so Docker is using the authentication on all calls as expected and not making noisy failing calls as the pulls from Bitbucket are doing. If docker is what Bitbucket Pipelines are using, it seems something is causing initial unauthenticated calls. So version of docker may be a factor or a different tool like containerd is being used to pull the image and behaves differently.

              Unassigned Unassigned
              skhandelwal@atlassian.com Sandeep K
              Affected customers:
              1 This affects my team
              Watchers:
              2 Start watching this issue

                Created:
                Updated: