Bamboo
  1. Bamboo
  2. BAM-8866

Bamboo not updating Mercurial sub-repositories when getting source on build server

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 3.0, 3.1
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Themes:
      DVCS
    • Last commented by user?:
      true
    • Story Points:
      2
    • Comments:
      9

      Description

      From SAC:
      "We use Mercurial and our project has a sub-repository (in a folder UI\Elegant) which is specified in the .hgsub file. The project is setup correctly, we can get it to any PC, commit, push, pull changes etc.. However, when changes to the sub-repository are made the Bamboo build fails. The hg.exe command appears to work in getting the project to the cache folder, though the sub-repository (UI\Elegant) is missing, but when it then tries to update the working folder (Bamboo\xml-data\build-dir\AIE-DEV-JOB1) it fails saying no working folder is specified for UI\Elegant.

      How can we resolve this?
      "
      https://support.atlassian.com/browse/BSP-4674

        Activity

        Hide
        Anton Mazkovoi [Atlassian] added a comment -

        Could we port the fix into a bug fix release of 3.1?

        We should definitely try it.

        Show
        Anton Mazkovoi [Atlassian] added a comment - Could we port the fix into a bug fix release of 3.1? We should definitely try it.
        Hide
        Simon Stebbing added a comment -

        I think this has crippled me too. I just did a major restructure of a project to use subrepositories.

        Here's my log btw, where 'common' is the first subrepository encountered.

        01-Jul-2011 01:50:47 	Build GA-T2-JOB1-5 started building on agent Default Agent
        01-Jul-2011 01:50:47 	Build always requires a clean checkout
        01-Jul-2011 01:50:47 	Cleaning build directory '/home/simon/Data/bamboo/xml-data/build-dir/GA-T2-JOB1'
        01-Jul-2011 01:50:47 	Updating source code to revision: ff5dd451c2c9c8662733d007ad840bdf748ef12b
        01-Jul-2011 01:50:47 	Obtaining remote repository to agent's cache directory '/home/simon/Data/bamboo/xml-data/build-dir/_hg-repositories-cache/f2d24b5d4d93796b69ff92e4e8b2eb8d0a467fb7'...
        01-Jul-2011 01:50:47 	Removing '/home/simon/Data/bamboo/xml-data/build-dir/_hg-repositories-cache/f2d24b5d4d93796b69ff92e4e8b2eb8d0a467fb7' as it does not contain a valid hg repository
        01-Jul-2011 01:50:47 	Creating empty repository in directory '/home/simon/Data/bamboo/xml-data/build-dir/_hg-repositories-cache/f2d24b5d4d93796b69ff92e4e8b2eb8d0a467fb7'
        01-Jul-2011 01:50:48 	Pulling from repository `/home/simon/Development/dolfants/app/clients/generic' to directory '/home/simon/Data/bamboo/xml-data/build-dir/_hg-repositories-cache/f2d24b5d4d93796b69ff92e4e8b2eb8d0a467fb7'
        01-Jul-2011 01:50:48 	pulling from /home/simon/Development/dolfants/app/clients/generic
        01-Jul-2011 01:50:48 	adding changesets
        01-Jul-2011 01:50:48 	adding manifests
        01-Jul-2011 01:50:48 	adding file changes
        01-Jul-2011 01:50:48 	added 4 changesets with 6 changes to 4 files
        01-Jul-2011 01:50:48 	(run 'hg update' to get a working copy)
        01-Jul-2011 01:50:48 	Copying repository from cache directory to source directory...
        01-Jul-2011 01:50:48 	Cloning repository `/home/simon/Data/bamboo/xml-data/build-dir/_hg-repositories-cache/f2d24b5d4d93796b69ff92e4e8b2eb8d0a467fb7' to directory '/home/simon/Data/bamboo/xml-data/build-dir/GA-T2-JOB1'
        01-Jul-2011 01:50:49 	pulling subrepo app/dolfants/common from /home/simon/Data/bamboo/xml-data/build-dir/_hg-repositories-cache/common
        01-Jul-2011 01:50:49 	Warning: failed to copy repository from cache directory (com.atlassian.bamboo.plugins.hg.HgCommandException: command [hg, update, --clean, --rev, ff5dd451c2c9c8662733d007ad840bdf748ef12b] failed. Working directory was `/home/simon/Data/bamboo/xml-data/build-dir/GA-T2-JOB1'. stderr: abort: repository /home/simon/Data/bamboo/xml-data/build-dir/_hg-repositories-cache/common not found! stdout: pulling subrepo app/dolfants/common from /home/simon/Data/bamboo/xml-data/build-dir/_hg-repositories-cache/common), trying to recover...
        01-Jul-2011 01:50:49 	Cleaned `/home/simon/Data/bamboo/xml-data/build-dir/GA-T2-JOB1' directory.
        01-Jul-2011 01:50:49 	Cloning repository `/home/simon/Data/bamboo/xml-data/build-dir/_hg-repositories-cache/f2d24b5d4d93796b69ff92e4e8b2eb8d0a467fb7' to directory '/home/simon/Data/bamboo/xml-data/build-dir/GA-T2-JOB1'
        01-Jul-2011 01:50:50 	Running update command again...
        01-Jul-2011 01:50:50 	pulling subrepo app/dolfants/common from /home/simon/Data/bamboo/xml-data/build-dir/_hg-repositories-cache/common
        01-Jul-2011 01:50:50 	Error occurred while executing the build for GA-T2-JOB1-5 : command [hg, update, --clean, --rev, ff5dd451c2c9c8662733d007ad840bdf748ef12b] failed. Working directory was `/home/simon/Data/bamboo/xml-data/build-dir/GA-T2-JOB1'. stderr: abort: repository /home/simon/Data/bamboo/xml-data/build-dir/_hg-repositories-cache/common not found! stdout: pulling subrepo app/dolfants/common from /home/simon/Data/bamboo/xml-data/build-dir/_hg-repositories-cache/common
        01-Jul-2011 01:50:50 	Running on server: post build plugin 'NCover Results Collector'
        01-Jul-2011 01:50:50A 	Running on server: post build plugin 'Clover Delta Calculator'
        01-Jul-2011 01:50:50 	Running on server: post build plugin 'Build Hanging Detection Configuration'
        01-Jul-2011 01:50:50 	All post build plugins have finished
        01-Jul-2011 01:50:50 	Generating build results summary...
        01-Jul-2011 01:50:50 	Saving build results to disk...
        01-Jul-2011 01:50:50 	Indexing build results...
        01-Jul-2011 01:50:50 	Finished building GA-T2-JOB1-5.
        
        Show
        Simon Stebbing added a comment - I think this has crippled me too. I just did a major restructure of a project to use subrepositories. Here's my log btw, where 'common' is the first subrepository encountered. 01-Jul-2011 01:50:47 Build GA-T2-JOB1-5 started building on agent Default Agent 01-Jul-2011 01:50:47 Build always requires a clean checkout 01-Jul-2011 01:50:47 Cleaning build directory '/home/simon/Data/bamboo/xml-data/build-dir/GA-T2-JOB1' 01-Jul-2011 01:50:47 Updating source code to revision: ff5dd451c2c9c8662733d007ad840bdf748ef12b 01-Jul-2011 01:50:47 Obtaining remote repository to agent's cache directory '/home/simon/Data/bamboo/xml-data/build-dir/_hg-repositories-cache/f2d24b5d4d93796b69ff92e4e8b2eb8d0a467fb7'... 01-Jul-2011 01:50:47 Removing '/home/simon/Data/bamboo/xml-data/build-dir/_hg-repositories-cache/f2d24b5d4d93796b69ff92e4e8b2eb8d0a467fb7' as it does not contain a valid hg repository 01-Jul-2011 01:50:47 Creating empty repository in directory '/home/simon/Data/bamboo/xml-data/build-dir/_hg-repositories-cache/f2d24b5d4d93796b69ff92e4e8b2eb8d0a467fb7' 01-Jul-2011 01:50:48 Pulling from repository `/home/simon/Development/dolfants/app/clients/ generic ' to directory '/home/simon/Data/bamboo/xml-data/build-dir/_hg-repositories-cache/f2d24b5d4d93796b69ff92e4e8b2eb8d0a467fb7' 01-Jul-2011 01:50:48 pulling from /home/simon/Development/dolfants/app/clients/ generic 01-Jul-2011 01:50:48 adding changesets 01-Jul-2011 01:50:48 adding manifests 01-Jul-2011 01:50:48 adding file changes 01-Jul-2011 01:50:48 added 4 changesets with 6 changes to 4 files 01-Jul-2011 01:50:48 (run 'hg update' to get a working copy) 01-Jul-2011 01:50:48 Copying repository from cache directory to source directory... 01-Jul-2011 01:50:48 Cloning repository `/home/simon/Data/bamboo/xml-data/build-dir/_hg-repositories-cache/f2d24b5d4d93796b69ff92e4e8b2eb8d0a467fb7' to directory '/home/simon/Data/bamboo/xml-data/build-dir/GA-T2-JOB1' 01-Jul-2011 01:50:49 pulling subrepo app/dolfants/common from /home/simon/Data/bamboo/xml-data/build-dir/_hg-repositories-cache/common 01-Jul-2011 01:50:49 Warning: failed to copy repository from cache directory (com.atlassian.bamboo.plugins.hg.HgCommandException: command [hg, update, --clean, --rev, ff5dd451c2c9c8662733d007ad840bdf748ef12b] failed. Working directory was `/home/simon/Data/bamboo/xml-data/build-dir/GA-T2-JOB1'. stderr: abort: repository /home/simon/Data/bamboo/xml-data/build-dir/_hg-repositories-cache/common not found! stdout: pulling subrepo app/dolfants/common from /home/simon/Data/bamboo/xml-data/build-dir/_hg-repositories-cache/common), trying to recover... 01-Jul-2011 01:50:49 Cleaned `/home/simon/Data/bamboo/xml-data/build-dir/GA-T2-JOB1' directory. 01-Jul-2011 01:50:49 Cloning repository `/home/simon/Data/bamboo/xml-data/build-dir/_hg-repositories-cache/f2d24b5d4d93796b69ff92e4e8b2eb8d0a467fb7' to directory '/home/simon/Data/bamboo/xml-data/build-dir/GA-T2-JOB1' 01-Jul-2011 01:50:50 Running update command again... 01-Jul-2011 01:50:50 pulling subrepo app/dolfants/common from /home/simon/Data/bamboo/xml-data/build-dir/_hg-repositories-cache/common 01-Jul-2011 01:50:50 Error occurred while executing the build for GA-T2-JOB1-5 : command [hg, update, --clean, --rev, ff5dd451c2c9c8662733d007ad840bdf748ef12b] failed. Working directory was `/home/simon/Data/bamboo/xml-data/build-dir/GA-T2-JOB1'. stderr: abort: repository /home/simon/Data/bamboo/xml-data/build-dir/_hg-repositories-cache/common not found! stdout: pulling subrepo app/dolfants/common from /home/simon/Data/bamboo/xml-data/build-dir/_hg-repositories-cache/common 01-Jul-2011 01:50:50 Running on server: post build plugin 'NCover Results Collector' 01-Jul-2011 01:50:50A Running on server: post build plugin 'Clover Delta Calculator' 01-Jul-2011 01:50:50 Running on server: post build plugin 'Build Hanging Detection Configuration' 01-Jul-2011 01:50:50 All post build plugins have finished 01-Jul-2011 01:50:50 Generating build results summary... 01-Jul-2011 01:50:50 Saving build results to disk... 01-Jul-2011 01:50:50 Indexing build results... 01-Jul-2011 01:50:50 Finished building GA-T2-JOB1-5.
        Hide
        Simon Stebbing added a comment -

        It's probably worth noting that my subrepositories are BitBucket friendly in that they exist outside the main repository and .hgsubbed into it.

        Show
        Simon Stebbing added a comment - It's probably worth noting that my subrepositories are BitBucket friendly in that they exist outside the main repository and .hgsubbed into it.
        Hide
        Lars Milland added a comment -

        Will a solution for this issue reach the 3.3 release of Bamboo. Since it would be critical for use of Bamboo with Mercurial in large project, that it can support subrepositories.

        Show
        Lars Milland added a comment - Will a solution for this issue reach the 3.3 release of Bamboo. Since it would be critical for use of Bamboo with Mercurial in large project, that it can support subrepositories.
        Hide
        Anton Mazkovoi [Atlassian] added a comment -

        Hi Lars,

        We are investigating the solution and are trying to make this work for Bmaboo 3.3. At the moment we cannot guarantee this just yet.

        Cheers,
        Anton

        Show
        Anton Mazkovoi [Atlassian] added a comment - Hi Lars, We are investigating the solution and are trying to make this work for Bmaboo 3.3. At the moment we cannot guarantee this just yet. Cheers, Anton
        Hide
        Marek Went [Atlassian] added a comment - - edited

        Really 2 tasks:

        • make caching optional (easy one)
        • investigate if we can support repo caching for sub repositories (python changes)

        Implement 1, timebox 2 to 4h

        Show
        Marek Went [Atlassian] added a comment - - edited Really 2 tasks: make caching optional (easy one) investigate if we can support repo caching for sub repositories (python changes) Implement 1, timebox 2 to 4h
        Hide
        Steffen Opel added a comment -

        This seems to affect me as well, albeit with a Git sub-repository within a Mercurial repository. The error signature pretty much resembles the one reported by Simon Stebbing, so I'll omit the log for the moment, let me know if you'd still like to see it.

        Alternatively you could test this yourself: The Mercurial repository in question is actually a Bamboo Plugin hosted at Bitbucket, the Git sub-repository has been introduced with release 1.4.0 and triggered the first failed build of this Bamboo plugin on our Bamboo server

        Is the fix mentioned in the initial comment a viable workaround I could integrate myself eventually, i.e. should I open a respective support request?

        Thanks much,
        Steffen

        Show
        Steffen Opel added a comment - This seems to affect me as well, albeit with a Git sub-repository within a Mercurial repository. The error signature pretty much resembles the one reported by Simon Stebbing , so I'll omit the log for the moment, let me know if you'd still like to see it. Alternatively you could test this yourself: The Mercurial repository in question is actually a Bamboo Plugin hosted at Bitbucket, the Git sub-repository has been introduced with release 1.4.0 and triggered the first failed build of this Bamboo plugin on our Bamboo server Is the fix mentioned in the initial comment a viable workaround I could integrate myself eventually, i.e. should I open a respective support request? Thanks much, Steffen
        Hide
        Matthias Luescher added a comment -

        If you have a fairly recent version of mercurial there is a simple workaround which you can add to your .hgsub file of the top level repository:

        MYSUBREPO = MYSUBREPO

        [subpaths]

        1. bamboo workaround (subrepositories do not get cached correctly)!
          .*/_hg-repositories-cache/[a-zA-Z0-9] {40}

          /(.*) = https://MYSERVER/\1

        Using this workaround the requests to the subrepositories will get redirected to the server instead of the cache.

        Show
        Matthias Luescher added a comment - If you have a fairly recent version of mercurial there is a simple workaround which you can add to your .hgsub file of the top level repository: MYSUBREPO = MYSUBREPO [subpaths] bamboo workaround (subrepositories do not get cached correctly)! .*/_hg-repositories-cache/ [a-zA-Z0-9] {40} /(.*) = https://MYSERVER/\1 Using this workaround the requests to the subrepositories will get redirected to the server instead of the cache.
        Hide
        Matthias Luescher added a comment -

        The previous comment with some formatting:
        If you have a fairly recent version of mercurial there is a simple workaround which you can add to your .hgsub file of the top level repository:

        MYSUBREPO = MYSUBREPO
        
        [subpaths]
        # bamboo workaround (subrepositories do not get cached correctly)!
        .*/_hg-repositories-cache/[a-zA-Z0-9]{40}/(.*) = https://MYSERVER/\1
        

        Using this workaround the requests to the subrepositories will get redirected to the server instead of the cache.

        Show
        Matthias Luescher added a comment - The previous comment with some formatting: If you have a fairly recent version of mercurial there is a simple workaround which you can add to your .hgsub file of the top level repository: MYSUBREPO = MYSUBREPO [subpaths] # bamboo workaround (subrepositories do not get cached correctly)! .*/_hg-repositories-cache/[a-zA-Z0-9]{40}/(.*) = https://MYSERVER/\1 Using this workaround the requests to the subrepositories will get redirected to the server instead of the cache.

          People

          • Votes:
            4 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:
              Last commented:
              2 years ago