Uploaded image for project: 'Sourcetree For Mac'
  1. Sourcetree For Mac
  2. SRCTREE-1997

Pushing a subtree creates incorrect commit history using embedded git version

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Medium Medium
    • None
    • 1.8.0.2
    • Git
    • None
    • Mac OS X Mavericks
      Git 1.8.5 compiled using Homebrew

      This bug makes the synthetic history generated by SourceTree incompatible with other subtree users. The root cause seems to be an extra "-n\n" in the commit messages generated by the embedded version of Git.

      Steps to reproduce:

      1. Clone this repository: https://bitbucket.org/jennings/sourcetree-subtree-commit-message-bug
      2. Set SourceTree to use the embedded Git (version 1.8.4.2)
      3. Use "Add/Link Subtree" to link the "sub" directory with a remote repository.
      4. Push the subtree.
      5. Set SourceTree to use the system version of Git (tested with version 1.8.5)
      6. Push the subtree.

      Results:

      The subtree created using the embedded Git has a head commit ID of 6e92039. The subtree created using the system Git has a head commit ID of 329f350. Resultant repos can be seen here:

      https://bitbucket.org/jennings/sourcetree-subtree-commit-message-bug-embedded
      https://bitbucket.org/jennings/sourcetree-subtree-commit-message-bug-system

      Expected result:

      Both branch head commit IDs should be 329f350.

          Form Name

            [SRCTREE-1997] Pushing a subtree creates incorrect commit history using embedded git version

            KieranA added a comment -

            Glad to hear, I'll go ahead and close. Thanks for all the input Stephen, I appreciate your time on this.

            KieranA added a comment - Glad to hear, I'll go ahead and close. Thanks for all the input Stephen, I appreciate your time on this.

            jennings added a comment -

            That is my understanding as well. I did find that 1.8.0.3 fixes this issue. Thanks again for the prompt attention.

            jennings added a comment - That is my understanding as well. I did find that 1.8.0.3 fixes this issue. Thanks again for the prompt attention.

            KieranA added a comment -

            Hey stephen.g.jennings,

            My colleague just had a thought I had the other day but failed to mention. The SHA is generated by a combination of ancestors, commit message, and files in the commit. Because of the '-n' bug (which has now been fixed) it meant that the commit message was different in your two examples. This is why it was wrong.

            If you did the same tests again you should find it works fine (crossing my fingers ). But seriously, that's why it wouldn't have been working. It figures the SHA out from the push, not what's in your repository containing subtrees.

            Cheers

            KieranA added a comment - Hey stephen.g.jennings , My colleague just had a thought I had the other day but failed to mention. The SHA is generated by a combination of ancestors, commit message, and files in the commit. Because of the '-n' bug (which has now been fixed) it meant that the commit message was different in your two examples. This is why it was wrong. If you did the same tests again you should find it works fine (crossing my fingers ). But seriously, that's why it wouldn't have been working. It figures the SHA out from the push, not what's in your repository containing subtrees. Cheers

            KieranA added a comment -

            stephen.g.jennings this is up on my list, but right now it's not a blocker so forgive me if it goes untouched for a few days. We're trying to get blockers sorted first before these high priority issues.

            KieranA added a comment - stephen.g.jennings this is up on my list, but right now it's not a blocker so forgive me if it goes untouched for a few days. We're trying to get blockers sorted first before these high priority issues.

            jennings added a comment -

            Thank you! The product is fantastic and I'm eager to start using the subtree support with our team, it's exactly what we've needed.

            jennings added a comment - Thank you! The product is fantastic and I'm eager to start using the subtree support with our team, it's exactly what we've needed.

            KieranA added a comment -

            With the already existing problem the metadata isn't the same - the commit messages are different because of the '-n' issue. Perhaps that's the clue, I'd need to confirm this.

            I need to do the hotfix release immediately to resolve the "-n" issue, but I want to make sure we address this issue for 1.8.1. You're completely right in what the documentation says, so we should roll with that ensuring the SHAs do match up as expected.

            Cheers

            KieranA added a comment - With the already existing problem the metadata isn't the same - the commit messages are different because of the '-n' issue. Perhaps that's the clue, I'd need to confirm this. I need to do the hotfix release immediately to resolve the "-n" issue, but I want to make sure we address this issue for 1.8.1. You're completely right in what the documentation says, so we should roll with that ensuring the SHAs do match up as expected. Cheers

            jennings added a comment - - edited

            git subtree push is implemented using split:

            localrev=$(git subtree split --prefix="$prefix") || die
            git push $repository $localrev:refs/heads/$refspec
            

            So, it first uses split to create a synthetic history and stores the resulting commit ID in $localrev. Then it pushes this to the given repository. So, if the commits being generated aren't identical to those created by other Git versions, then I would guess the problem is probably in the split command.

            Although calling split does extract a new history each time, it should be identical every time because the generated commits get their commit metadata from the commits they're splitting from. Identical metadata and identical tree = identical commit ID.

            After updating Git in your development version, do you not get commit ID 329f350 if you split the test repository?

            jennings added a comment - - edited git subtree push is implemented using split : localrev=$(git subtree split --prefix= "$prefix" ) || die git push $repository $localrev:refs/heads/$refspec So, it first uses split to create a synthetic history and stores the resulting commit ID in $localrev. Then it pushes this to the given repository. So, if the commits being generated aren't identical to those created by other Git versions, then I would guess the problem is probably in the split command. Although calling split does extract a new history each time, it should be identical every time because the generated commits get their commit metadata from the commits they're splitting from. Identical metadata and identical tree = identical commit ID. After updating Git in your development version, do you not get commit ID 329f350 if you split the test repository?

            KieranA added a comment -

            Hey stephen.g.jennings,

            Thanks for the detailed information. We don't use split though, we use 'push', but the very definition of it should be equal of that of what you're suggesting as documented:

            Does a 'split' (see below) using the <prefix> supplied
            and then does a 'git push' to push the result to the
            repository and refspec. This can be used to push your
            subtree to different branches of the remote repository.

            Which would mean that 'push' itself could be in error. I'd have to take a deeper look into this, but I'm assuming it's because multiple calls to 'push' always extracts a new project history. On the flipside it also says this: "Repeated splits of exactly the same history are guaranteed to be identical".

            I'd have to take a further look into this to see what's going on, and what's expected based on the issued commands.

            KieranA added a comment - Hey stephen.g.jennings , Thanks for the detailed information. We don't use split though, we use 'push', but the very definition of it should be equal of that of what you're suggesting as documented: Does a 'split' (see below) using the <prefix> supplied and then does a 'git push' to push the result to the repository and refspec. This can be used to push your subtree to different branches of the remote repository. Which would mean that 'push' itself could be in error. I'd have to take a deeper look into this, but I'm assuming it's because multiple calls to 'push' always extracts a new project history. On the flipside it also says this: "Repeated splits of exactly the same history are guaranteed to be identical". I'd have to take a further look into this to see what's going on, and what's expected based on the issued commands.

            jennings added a comment -

            As a test, I just cloned the repository above on a different computer (a Windows machine, Git 1.8.4.0, different user.email setting), ran git subtree split, and got back commit ID 329f350, which is the same commit ID I got from my Mac when I use the system version of Git.

            jennings added a comment - As a test, I just cloned the repository above on a different computer (a Windows machine, Git 1.8.4.0, different user.email setting), ran git subtree split , and got back commit ID 329f350, which is the same commit ID I got from my Mac when I use the system version of Git.

            jennings added a comment - - edited

            Hi Kieran,

            Thanks for taking a look at this so quickly.

            The documentation says that git-subtree split should be identical every time:

            https://github.com/git/git/blob/master/contrib/subtree/git-subtree.txt#L116

            If they're not identical, it causes chaos in the split-off repository. Imagine a scenario where there is a super project ("Super") which has a folder inside that you choose to split off into it's own repo ("Sub") using git-subtree.

            Using Git 1.8.4.2, you split off the history of Sub and end up with three commits that you push to the Sub public repository:

            A -> B -> C

            Now you add another commit to Super that affects Sub. You also upgrade to Git 1.8.5. You run git subtree split again (or push, which calls split under the hood) and end up with these commits:

            A' -> B' -> C' -> D'

            Notice that all the commits are different. A and A' contain the same tree, but the commit message doesn't have the extra "-n" in it so the commits IDs are different. Now when you try to push this history to the Sub public repository, you get a merge conflict with no common ancestor.

            jennings added a comment - - edited Hi Kieran, Thanks for taking a look at this so quickly. The documentation says that git-subtree split should be identical every time: https://github.com/git/git/blob/master/contrib/subtree/git-subtree.txt#L116 If they're not identical, it causes chaos in the split-off repository. Imagine a scenario where there is a super project ("Super") which has a folder inside that you choose to split off into it's own repo ("Sub") using git-subtree. Using Git 1.8.4.2, you split off the history of Sub and end up with three commits that you push to the Sub public repository: A -> B -> C Now you add another commit to Super that affects Sub. You also upgrade to Git 1.8.5. You run git subtree split again (or push , which calls split under the hood) and end up with these commits: A' -> B' -> C' -> D' Notice that all the commits are different. A and A' contain the same tree, but the commit message doesn't have the extra "-n" in it so the commits IDs are different. Now when you try to push this history to the Sub public repository, you get a merge conflict with no common ancestor.

              Unassigned Unassigned
              e74945cc5897 jennings
              Affected customers:
              0 This affects my team
              Watchers:
              2 Start watching this issue

                Created:
                Updated:
                Resolved: