Uploaded image for project: 'Bitbucket Cloud'
  1. Bitbucket Cloud
  2. BCLOUD-14645

Git: Fatal Error running `git diff`

XMLWordPrintable

      I am building a CI solution for Salesforce code. In Salesforce, there are no atomic deploys, so you must manually delete files that have been renamed or removed. Yea.

      My solution to this was to track the commit the pipeline runs on so that I can generate a list of all deleted files from the previous pipeline run with git diff <previous sha>..<current sha>.

      The problem is that even with clone: depth: full enabled in the pipeline.yml, git throws an error because the previous commit doesn't seem to exist. I verified this is an issue by recreating the pipeline in a Docker container. When I use my normally cloned repo on my local machine, the git diff command works fine. When I clone the repo the way the pipeline does via git clone --branch="staging" <url>, then the command fails and returns a fatal: bad object <sha> error.

      My pipeline:

      image: node:latest

clone:
  depth: full

definitions:
  caches:
    sfdx: ~/.sfdx
    sfdx-libs: ~/.local/share/sfdx
    yarn: ~/.yarn
pipelines:

  branches:
    staging:
        - step:
            caches:
              - node
              - sfdx
              - sfdx-libs
            script:
              - git show $BITBUCKET_COMMIT
              - echo $BITBUCKET_COMMIT > ~/.sfdx/current-build
              - tasks/build $BITBUCKET_BRANCH
              - tasks/deploy $BITBUCKET_BRANCH
              - echo $BITBUCKET_COMMIT > ~/.sfdx/previous-build

            [BCLOUD-14645] Git: Fatal Error running `git diff`

            Aneita added a comment -

            Not a problem, glad you've managed to get it working! Feel free to reach out if you have any other questions.

            Aneita added a comment - Not a problem, glad you've managed to get it working! Feel free to reach out if you have any other questions.

            jondum-raprec added a comment -

            I followed your example and recreated my use case... and it worked.

            Pipelines has changed quite a bit since I last tried, maybe a recent release fixed it? Probably unlikely but thanks for your help regardless.

            jondum-raprec added a comment - I followed your example and recreated my use case... and it worked. Pipelines has changed quite a bit since I last tried, maybe a recent release fixed it? Probably unlikely but thanks for your help regardless.

            Aneita added a comment -

            Hey Jonathan,

            Currently, Pipelines gets a special admin-level token to clone repos in Bitbucket which can't be shared with the build process. We have an open feature request to allow builds to push back to Bitbucket, tracked here: BCLOUD-13213.

            I can see it is also unintuitive that depth: full doesn't clone all refs from Bitbucket, but rather just fetches the ancestry of the specified branch. I've raised an issue to request this as an enhancement to Pipelines. At the moment, we are busy with some other highly voted requests, but we'll keep this ticket open to gauge interest from our users.

            As for the primary issue of git diff not working and returning a fatal error, I cannot seem to reproduce this. Are you able to confirm that your two commits are indeed part of the same ancestry? You can use git log <current sha> --pretty=%h | grep <previous sha> to check this.

            If you've found an example where the previous commit is in the history of the current HEAD, and you still cannot get git diff to work properly, it would be best for you to raise a support ticket, so that we can look at your repository and investigate the problem in more detail.

            Thanks,
            Aneita

            Aneita added a comment - Hey Jonathan, Currently, Pipelines gets a special admin-level token to clone repos in Bitbucket which can't be shared with the build process. We have an open feature request to allow builds to push back to Bitbucket, tracked here: BCLOUD-13213 . I can see it is also unintuitive that depth: full doesn't clone all refs from Bitbucket, but rather just fetches the ancestry of the specified branch. I've raised an issue to request this as an enhancement to Pipelines. At the moment, we are busy with some other highly voted requests, but we'll keep this ticket open to gauge interest from our users. As for the primary issue of git diff not working and returning a fatal error, I cannot seem to reproduce this . Are you able to confirm that your two commits are indeed part of the same ancestry? You can use git log <current sha> --pretty=%h | grep <previous sha> to check this. If you've found an example where the previous commit is in the history of the current HEAD, and you still cannot get git diff to work properly, it would be best for you to raise a support ticket , so that we can look at your repository and investigate the problem in more detail. Thanks, Aneita

            jondum-raprec added a comment -

            Thanks for the response Aneita. If my branch is staging and I'm comparing to the previous HEAD of staging, how is that not part of the same ancestry? It just seems a little unintuitive to me that this wouldn't work out of the box... And it's even more wonky that I have to set up SSH access keys for the pipeline to access the repo it's assigned to. Why does the built-in, initial git clone command get special access to my repo while my pipeline script does not?

            Perhaps it's worth bringing up to your team adding the ability to maintain and cache an actual full clone of the repo and doing subsequent git checkout -f && git fetch to only pull down diffs on pipeline executions?

            jondum-raprec added a comment - Thanks for the response Aneita. If my branch is staging and I'm comparing to the previous HEAD of staging , how is that not part of the same ancestry? It just seems a little unintuitive to me that this wouldn't work out of the box... And it's even more wonky that I have to set up SSH access keys for the pipeline to access the repo it's assigned to. Why does the built-in, initial git clone command get special access to my repo while my pipeline script does not? Perhaps it's worth bringing up to your team adding the ability to maintain and cache an actual full clone of the repo and doing subsequent git checkout -f && git fetch to only pull down diffs on pipeline executions?

            Aneita added a comment -

            Hi Jonathan,

            Currently, when you specify depth: full, Pipelines will clone all of the commits in the ancestry of the branch (as you've discovered via the git clone --branch="staging" <url> command). If you are trying to compare two commits on different branches that aren't part of the same ancestry, the comparison will error. As a workaround, you can try doing a git fetch --all which will fetch all remote refs. In order to do this, you may need to generate a pair of SSH keys and add the public key as an access key for itself. That way you'll be able to perform git read actions.

            Please let me know if this helps you.

            Aneita added a comment - Hi Jonathan, Currently, when you specify depth: full , Pipelines will clone all of the commits in the ancestry of the branch (as you've discovered via the git clone --branch="staging" <url> command). If you are trying to compare two commits on different branches that aren't part of the same ancestry, the comparison will error. As a workaround, you can try doing a git fetch --all which will fetch all remote refs. In order to do this, you may need to generate a pair of SSH keys and add the public key as an access key for itself. That way you'll be able to perform git read actions. Please let me know if this helps you.

              ayang@atlassian.com Aneita
              ce081a7efb71 jondum-raprec
              Affected customers:
              0 This affects my team
              Watchers:
              3 Start watching this issue

                Created:
                Updated:
                Resolved: