• We collect Bitbucket feedback from various sources, and we evaluate what we've collected when planning our product roadmap. To understand how this piece of feedback will be reviewed, see our Implementation of New Features Policy.

      We're making a plugin that will keep Stash Git repositories in sync with Subversion repository. This plugin uses SubGit.

      It all works fine, except for merge of pull requests. The problem is that SubGit expects all new commits to come via client pushes and pre-receive hook to be called. Then SubGit has a chance to reject users commit in case there are incoming changes from Subversion.

      With merge operation, hooks are not called and Git repository goes out of sync with Subversion repository.

      What we need for SubGit plugin to function properly:

      1) call a plugin's hook before updating branch reference to the newly integrated commits from the pull request;
      2) let plugin's hook reject that merge commit (in case there are incoming changes from Subversion being translated to Git commits and pull request have to be merged again).

      Alternatively, as a fallback, plugin should be able to disable pull requests feature for repository it keeps in sync with Subversion.

            [BSERV-3122] API to hook into pull request merge process.

            AlexH added a comment -

            Wonderfully detailed and helpful exposition, thank you Bryan! I will rest easy writing and maintaining my pull-request hooks as implementations of RepositoryMergeRequestCheck.

            AlexH added a comment - Wonderfully detailed and helpful exposition, thank you Bryan! I will rest easy writing and maintaining my pull-request hooks as implementations of RepositoryMergeRequestCheck .

            Alex,

            A normal plugin would never use this hook, and it was not built with them in mind. It exists almost solely to support the SubGit plugin. When a pull request is merged, it needs the full commit details to decide whether or not the associated Subversion repository is in a state that allows the merge to complete. If it's not, the merge is rejected and the user gets a well-formed error message provided by the plugin indicating why the commit was rejected. It's messaged as clearly as the plugin developer chooses to message it; their internationalized message is added to a bulleted list displayed in the UI. The bulleted list allows multiple plugins to be canceling the event and each providing their own message. (This event mirrors, more or less, the pre-receive hook callback SubGit would receive if Stash pushed the pull request merge into the repository instead of fetching it.)

            In practice, the overwhelming majority of the objects from any given pull request merge are already in the repository because Stash's diff screen shows the merge, not a simple diff-to-common-ancestor. All of those automatic merges used to show the diff are protected by the reflogs, meaning they are not unreachable (and remain packed during garbage collection). That means while this approach does create a few new objects that would be unreachable, it's unlikely it creates more than a handful (potentially as few as 1, the commit itself).

            Just to clarify, Stash disables pruning and automatic garbage collection in repositories that have been forked and it manually controls garbage collection for them. Unreachable objects are packed by Stash, in repositories with pruning disabled, so it is unlikely they will impact the overall repository much. I'll also note that that's not a permanent behaviour; hierarchical garbage collection will be added in a future release. At that time, any unreachable objects that have built up in any repositories will be removed.

            Hope this helps,
            Bryan Turner
            Atlassian Stash

            Bryan Turner (Inactive) added a comment - Alex, A normal plugin would never use this hook, and it was not built with them in mind. It exists almost solely to support the SubGit plugin. When a pull request is merged, it needs the full commit details to decide whether or not the associated Subversion repository is in a state that allows the merge to complete. If it's not, the merge is rejected and the user gets a well-formed error message provided by the plugin indicating why the commit was rejected. It's messaged as clearly as the plugin developer chooses to message it; their internationalized message is added to a bulleted list displayed in the UI. The bulleted list allows multiple plugins to be canceling the event and each providing their own message. (This event mirrors, more or less, the pre-receive hook callback SubGit would receive if Stash pushed the pull request merge into the repository instead of fetching it.) In practice, the overwhelming majority of the objects from any given pull request merge are already in the repository because Stash's diff screen shows the merge , not a simple diff-to-common-ancestor. All of those automatic merges used to show the diff are protected by the reflogs, meaning they are not unreachable (and remain packed during garbage collection). That means while this approach does create a few new objects that would be unreachable, it's unlikely it creates more than a handful (potentially as few as 1, the commit itself). Just to clarify, Stash disables pruning and automatic garbage collection in repositories that have been forked and it manually controls garbage collection for them. Unreachable objects are packed by Stash, in repositories with pruning disabled, so it is unlikely they will impact the overall repository much. I'll also note that that's not a permanent behaviour; hierarchical garbage collection will be added in a future release. At that time, any unreachable objects that have built up in any repositories will be removed. Hope this helps, Bryan Turner Atlassian Stash

            AlexH added a comment -

            Please forgive my ignorance, I'm trying to understand when this new API module would actually be appropriate to use.

            Compared to what was available in earlier versions of Stash, why would a plugin developer ever want to use this instead of the pre-approval/denial as provided by the RepositoryMergeRequestCheck hook?

            It seems like this method has several disadvantages in where and why it can fail and how that error is communicated back to the user. What happens to the commit that was accepted into the repo but subsequently rejected by the hook? Stash disables repo garbage collection, so would this hook cause a build up of unused objects in the object store?

            I can only think of two reasons why this module would ever be preferred over RepositoryMergeRequestCheck:

            1. Another integrating tool can't make a decision without the commit data available within the git tool itself
            2. Your check is so expensive you don't want it to run every time someone points their browser at the pull-request page and you only want to execute the check when the merge is attempted.

            I'm not questioning the validity of this hooks existence, only looking to understand if there's some advantage to this method that I'm not grokking.

            AlexH added a comment - Please forgive my ignorance, I'm trying to understand when this new API module would actually be appropriate to use. Compared to what was available in earlier versions of Stash, why would a plugin developer ever want to use this instead of the pre-approval/denial as provided by the RepositoryMergeRequestCheck hook? It seems like this method has several disadvantages in where and why it can fail and how that error is communicated back to the user. What happens to the commit that was accepted into the repo but subsequently rejected by the hook? Stash disables repo garbage collection, so would this hook cause a build up of unused objects in the object store? I can only think of two reasons why this module would ever be preferred over RepositoryMergeRequestCheck : Another integrating tool can't make a decision without the commit data available within the git tool itself Your check is so expensive you don't want it to run every time someone points their browser at the pull-request page and you only want to execute the check when the merge is attempted. I'm not questioning the validity of this hooks existence, only looking to understand if there's some advantage to this method that I'm not grokking.

            This will be included in the 2.8 release of Stash.

            The merge process has been revised slightly so that the merge is fetched into the repository without updating any refs, rather than fetching it directly to the target ref. A new GitPullRequestMergeRequestedEvent, added to the `stash-scm-git-api` module, is then raised with the PullRequest being merged and the SHA1 of the merge commit. Event listeners can retrieve the commit's details normally using any of the existing Stash service methods that accept hashes, like HistoryService.getChangeset(Repository, String), where the Repository is PullRequest.getToRef().getRepository().

            If the merge proves "unacceptable", the event has a cancel(KeyedMessage) method on it. Listeners must provide a message explaining why the merge is being canceled, and the message they provide will be shown to the user who tried to merge the pull request. The message provided should be clear and descriptive so that it can be readily understandable to an end user. If the event is canceled by any listener, the target ref will not be updated and the merge will be discarded. Otherwise, the target ref will be updated. Note that updating the ref can still fail, if it has been updated since the merge was started (for example by merging a different pull request, or by a push to the repository).

            Bryan Turner (Inactive) added a comment - This will be included in the 2.8 release of Stash. The merge process has been revised slightly so that the merge is fetched into the repository without updating any refs, rather than fetching it directly to the target ref. A new GitPullRequestMergeRequestedEvent , added to the `stash-scm-git-api` module, is then raised with the PullRequest being merged and the SHA1 of the merge commit. Event listeners can retrieve the commit's details normally using any of the existing Stash service methods that accept hashes, like HistoryService.getChangeset(Repository, String) , where the Repository is PullRequest.getToRef().getRepository() . If the merge proves "unacceptable", the event has a cancel(KeyedMessage) method on it. Listeners must provide a message explaining why the merge is being canceled, and the message they provide will be shown to the user who tried to merge the pull request. The message provided should be clear and descriptive so that it can be readily understandable to an end user. If the event is canceled by any listener, the target ref will not be updated and the merge will be discarded. Otherwise, the target ref will be updated. Note that updating the ref can still fail, if it has been updated since the merge was started (for example by merging a different pull request, or by a push to the repository).

            Take my vote!
            This issue is only obstruction for commercial implementation of Stash in my company.
            Good luck!

            Uldis Anšmits added a comment - Take my vote! This issue is only obstruction for commercial implementation of Stash in my company. Good luck!

            jens added a comment -

            semen.vadishev, unfortunately there has been no progress on this issue. We will have to set some time aside to spike this within the next month.

            jens added a comment - semen.vadishev , unfortunately there has been no progress on this issue. We will have to set some time aside to spike this within the next month.

            Any updates on this?

            Current version of SVN Mirror Add-on does not allow one to merge pull request via Stash UI. We're looking forward to improving that part of functionality.

            Semyon Vadishev added a comment - Any updates on this? Current version of SVN Mirror Add-on does not allow one to merge pull request via Stash UI. We're looking forward to improving that part of functionality.

            bturner suggested the following potential solution:

            Change the flow to be two-staged (phase 1: fetch, phase 2: ref update)

            • Fetch the objects from the temporary repository. Update temporary refs.
            • Raise the new event that can be cancelled.
            • Update the actual refs (under refs/heads) unless the event was vetoed.

            Caveat: This would be an integration/plugin point with the git-scm plugin not with the host application.

            Questions:

            • Should that be an official or a dark feature/deep integration style API
            • Impact on other existing functionality

            Stefan Saasen (Inactive) added a comment - bturner suggested the following potential solution: Change the flow to be two-staged (phase 1: fetch, phase 2: ref update) Fetch the objects from the temporary repository. Update temporary refs. Raise the new event that can be cancelled. Update the actual refs (under refs/heads ) unless the event was vetoed. Caveat: This would be an integration/plugin point with the git-scm plugin not with the host application. Questions: Should that be an official or a dark feature/deep integration style API Impact on other existing functionality

            Hello Jason,

            I've been digging into Stash internals on the matter of using `git fetch` instead of `git push` and have found the following comment:

            An alternative to this approach is to use {@code git push} to push the merge commit back into the real
            repository. However, using {@code git push} may trigger garbage collection which results in radically
            discrepant outliers in merge timings. For large repositories, garbage collection may take anywhere from
            several seconds to entire minutes. {@code git fetch} does not trigger garbage collection and is immune
            to these outliers. Additionally, it appears to generally just run faster anyway, most likely because it
            is also not affected by the number of refs in the real repository (where {@code git push}, since it has
            a normal ref advertisement phase, can execute noticeably slower in repositories with a large number of
            refs).
            

            Switching to `git push` for pull requests is something we'd be glad to see. If performance issues are the only blockers here, would you consider possible workarounds for `git push`, like playing with gc.auto config option?

            As Alex has described above, SubGit plugin has to disable pull requests at the moment which is embarrassing. So, we'd like to discuss any possibility to use `git push` (resp. pre-receive & post-receive hooks) for pull requests.

            Thank you in advance.

            Semyon Vadishev added a comment - Hello Jason, I've been digging into Stash internals on the matter of using `git fetch` instead of `git push` and have found the following comment: An alternative to this approach is to use {@code git push} to push the merge commit back into the real repository. However, using {@code git push} may trigger garbage collection which results in radically discrepant outliers in merge timings. For large repositories, garbage collection may take anywhere from several seconds to entire minutes. {@code git fetch} does not trigger garbage collection and is immune to these outliers. Additionally, it appears to generally just run faster anyway, most likely because it is also not affected by the number of refs in the real repository (where {@code git push}, since it has a normal ref advertisement phase, can execute noticeably slower in repositories with a large number of refs). Switching to `git push` for pull requests is something we'd be glad to see. If performance issues are the only blockers here, would you consider possible workarounds for `git push`, like playing with gc.auto config option? As Alex has described above, SubGit plugin has to disable pull requests at the moment which is embarrassing. So, we'd like to discuss any possibility to use `git push` (resp. pre-receive & post-receive hooks) for pull requests. Thank you in advance.

            Hello Sasha!

            Solution based on hooks is not safe, cos you need to have failback support in case of 'network disconections', hook failures on otherside, etc. Stash will no perform repeates to such cases. GitHub also not support retries. You just lost webhook invocation.

            We used TeamCity for such mirroring about you explain here. Teamcity (or, oops i'm sorry Bamboo ) will monitor Git under Stash and then run miroring script if changes detected. It is more stability rather what hooks solution. And work with any supported VCS, not only for Stash. And Atlassian have a chance do not add code (and bugs/support) into Stash, also

            Alexey Efimov added a comment - Hello Sasha! Solution based on hooks is not safe, cos you need to have failback support in case of 'network disconections', hook failures on otherside, etc. Stash will no perform repeates to such cases. GitHub also not support retries. You just lost webhook invocation. We used TeamCity for such mirroring about you explain here. Teamcity (or, oops i'm sorry Bamboo ) will monitor Git under Stash and then run miroring script if changes detected. It is more stability rather what hooks solution. And work with any supported VCS, not only for Stash. And Atlassian have a chance do not add code (and bugs/support) into Stash, also

              Unassigned Unassigned
              6a57e6c370d0 Alexander Kitaev
              Votes:
              4 Vote for this issue
              Watchers:
              17 Start watching this issue

                Created:
                Updated:
                Resolved: