• Icon: Bug Bug
    • Resolution: Obsolete
    • Icon: Medium Medium
    • None
    • None
    • Repository (Git)
    • None

      We are running Bamboo Elastic Agents on Windows Server 2012 AMIs and are finding poor execution time when using git checkout inside a bamboo script task. We run the exact identical script in RDP on the same instance, as the same user with the same parameters and it only take 1 minute while running the task from Bamboo takes 30minutes!!! There is no network traffic as we are checking out locally.

      Does the Java process lower the priority of the child process or limit the resources a task in child process can consume. We are close to abandoning Bamboo for our builds but are reaching out in a last stitch effort to make this work.

            [BAM-14880] Git checkout unreasonably slow

            The issue is related to msysgit which is obsolete.

            Marcin Gardias added a comment - The issue is related to msysgit which is obsolete.

            I was having a similar issue and solved it with a different workaround. I had a build plan that checked out 5 git repositories, and took 10 minutes to do so.

            Fix: I installed Git 2.7.2 on the Bamboo agent, and instructed Bamboo to use that git instead of the git packaged with Bamboo (Admin > Remote Agents > Agent > Capabilities > Git).

            After installing the new version of Git, checkouts took a matter of seconds.

            Brian Lachniet added a comment - I was having a similar issue and solved it with a different workaround. I had a build plan that checked out 5 git repositories, and took 10 minutes to do so. Fix : I installed Git 2.7.2 on the Bamboo agent, and instructed Bamboo to use that git instead of the git packaged with Bamboo (Admin > Remote Agents > Agent > Capabilities > Git). After installing the new version of Git, checkouts took a matter of seconds.

            I have run into a work-around that worked for me, but it also very strange.

            I run bamboo 5.9.1 on linux, and I have agents running on linux servers and Windows 2012R2 servers.

            For the Windows 2012 servers, I have a user that belongs to Administrators running the bamboo agent service (this is so far necessary to build MSIs)
            I had 50 second git checkouts/operations, because git was getting stuck on ssh.

            Here's what worked for me, and I have verified that it works on multiple servers, as strange as it is:

            Login as the bamboo user. Remove bamboo user from Admin group. Restart bamboo service.
            Add bamboo user to Admin group. Restart bamboo service.

            And just like that, on multiple servers, I get git clones in under 3 sec.

            I guess there is some sort of permission application order issue at play here, I just wish I knew which permissions and applied to what.

            George Lianeris added a comment - I have run into a work-around that worked for me, but it also very strange. I run bamboo 5.9.1 on linux, and I have agents running on linux servers and Windows 2012R2 servers. For the Windows 2012 servers, I have a user that belongs to Administrators running the bamboo agent service (this is so far necessary to build MSIs) I had 50 second git checkouts/operations, because git was getting stuck on ssh. Here's what worked for me, and I have verified that it works on multiple servers, as strange as it is: Login as the bamboo user. Remove bamboo user from Admin group. Restart bamboo service. Add bamboo user to Admin group. Restart bamboo service. And just like that, on multiple servers, I get git clones in under 3 sec. I guess there is some sort of permission application order issue at play here, I just wish I knew which permissions and applied to what.

            Here is an answer:

            http://stackoverflow.com/questions/3427224/hudson-windows-github-git-plugin-really-really-slow-fetch-stage

            Basically, it seems that msysgit's ssh tries to load something that is not available when running as a service and spends time waiting on that resource.
            The workaround is to set GIT_SSH to point at putty's plink.exe, and use this instead.

            You can create a session using putty or load pageant to load the key, either way will work.

            I just tested this work-around and it works.

            George Lianeris added a comment - Here is an answer: http://stackoverflow.com/questions/3427224/hudson-windows-github-git-plugin-really-really-slow-fetch-stage Basically, it seems that msysgit's ssh tries to load something that is not available when running as a service and spends time waiting on that resource. The workaround is to set GIT_SSH to point at putty's plink.exe, and use this instead. You can create a session using putty or load pageant to load the key, either way will work. I just tested this work-around and it works.

            Hi, I am having a similar issue.

            I'm not using Elastic Bamboo, the common denominator here though is Windows 2012 VM.

            Here's my setup:

            Bamboo 5.8.1 Agent running on Windows Server 2012 VM
            The Agent runs as a service that logs on as and runs as user 'build', with administrative privileges
            The agent home is in E:\build\bamboo-agent-home
            Msysgit 1.9.5 is installed on that machine and picked up correctly by the agent, it was automatically listed in the capabilitiles.
            VS2010 and VS2013 are also installed on this VM.

            The master server runs git 2.1.3 and it is a linux system.
            The Stash server also runs git 2.1.3.

            Our linux agents are NOT experiencing this problem.

            When I check out logged in as user build, the checkout is very fast both through tortoisegit and command line git, using SSH (specifically msysgit's SSH):

            Success (3953 ms @ 5/28/2015 2:05:59 PM) – and command line git is even faster.

            However when Bamboo is doing the check-out, I get this (notice this almost takes a whole minute):

            27-May-2015 11:19:19 Build ProjA - proj-a-rel - Release #1 (POC-PROJA-REL-1) started building on agent idaurcibmbwp01
            27-May-2015 11:19:19 Build working directory is E:\build\bamboo-agent-home\xml-data\build-dir\POC-PROJA-REL
            27-May-2015 11:19:19 Executing build IDS Entitlement - proj-a-rel - Release #1 (POC-PROJA-REL-1)
            27-May-2015 11:19:19 Starting task 'Checkout Default Repository' of type 'com.atlassian.bamboo.plugins.vcs:task.vcs.checkout'
            27-May-2015 11:19:19 Build always requires a clean checkout
            27-May-2015 11:19:19 Cleaning build directory 'E:\build\bamboo-agent-home\xml-data\build-dir\POC-PROJA-REL'
            27-May-2015 11:19:19 Updating source code to revision: 3b5a6ce50dbed2a6e4eb5adbb25683c691b839d5
            27-May-2015 11:19:19 Creating local git repository in 'E:\build\bamboo-agent-home\xml-data\build-dir\POC-PROJA-REL\.git'.
            27-May-2015 11:19:19 Initialized empty Git repository in E:/build/bamboo-agent-home/xml-data/build-dir/POC-PROJA-REL/.git/
            27-May-2015 11:19:44 Fetching 'refs/heads/master' from 'ssh://git@git.ids.cbn:7999/poc/proj-a.git'.
            27-May-2015 11:20:08 Warning: Permanently added '[127.0.0.1]:62670' (RSA) to the list of known hosts.
            27-May-2015 11:20:08 From ssh://127.0.0.1:62670/poc/proj-a
            27-May-2015 11:20:08 * [new branch] master -> master
            27-May-2015 11:20:08 * [new tag] 1.0.5 -> 1.0.5
            27-May-2015 11:20:08 Checking out revision 3b5a6ce50dbed2a6e4eb5adbb25683c691b839d5.
            27-May-2015 11:20:08 Already on 'master'
            27-May-2015 11:20:08 Updated source code to revision: 3b5a6ce50dbed2a6e4eb5adbb25683c691b839d5
            27-May-2015 11:20:08 Finished task 'Checkout Default Repository' with result: Success

            George Lianeris added a comment - Hi, I am having a similar issue. I'm not using Elastic Bamboo, the common denominator here though is Windows 2012 VM. Here's my setup: Bamboo 5.8.1 Agent running on Windows Server 2012 VM The Agent runs as a service that logs on as and runs as user 'build', with administrative privileges The agent home is in E:\build\bamboo-agent-home Msysgit 1.9.5 is installed on that machine and picked up correctly by the agent, it was automatically listed in the capabilitiles. VS2010 and VS2013 are also installed on this VM. The master server runs git 2.1.3 and it is a linux system. The Stash server also runs git 2.1.3. Our linux agents are NOT experiencing this problem. When I check out logged in as user build, the checkout is very fast both through tortoisegit and command line git, using SSH (specifically msysgit's SSH): Success (3953 ms @ 5/28/2015 2:05:59 PM) – and command line git is even faster. However when Bamboo is doing the check-out, I get this (notice this almost takes a whole minute): 27-May-2015 11:19:19 Build ProjA - proj-a-rel - Release #1 (POC-PROJA-REL-1) started building on agent idaurcibmbwp01 27-May-2015 11:19:19 Build working directory is E:\build\bamboo-agent-home\xml-data\build-dir\POC-PROJA-REL 27-May-2015 11:19:19 Executing build IDS Entitlement - proj-a-rel - Release #1 (POC-PROJA-REL-1) 27-May-2015 11:19:19 Starting task 'Checkout Default Repository' of type 'com.atlassian.bamboo.plugins.vcs:task.vcs.checkout' 27-May-2015 11:19:19 Build always requires a clean checkout 27-May-2015 11:19:19 Cleaning build directory 'E:\build\bamboo-agent-home\xml-data\build-dir\POC-PROJA-REL' 27-May-2015 11:19:19 Updating source code to revision: 3b5a6ce50dbed2a6e4eb5adbb25683c691b839d5 27-May-2015 11:19:19 Creating local git repository in 'E:\build\bamboo-agent-home\xml-data\build-dir\POC-PROJA-REL\.git'. 27-May-2015 11:19:19 Initialized empty Git repository in E:/build/bamboo-agent-home/xml-data/build-dir/POC-PROJA-REL/.git/ 27-May-2015 11:19:44 Fetching 'refs/heads/master' from 'ssh://git@git.ids.cbn:7999/poc/proj-a.git'. 27-May-2015 11:20:08 Warning: Permanently added ' [127.0.0.1] :62670' (RSA) to the list of known hosts. 27-May-2015 11:20:08 From ssh://127.0.0.1:62670/poc/proj-a 27-May-2015 11:20:08 * [new branch] master -> master 27-May-2015 11:20:08 * [new tag] 1.0.5 -> 1.0.5 27-May-2015 11:20:08 Checking out revision 3b5a6ce50dbed2a6e4eb5adbb25683c691b839d5. 27-May-2015 11:20:08 Already on 'master' 27-May-2015 11:20:08 Updated source code to revision: 3b5a6ce50dbed2a6e4eb5adbb25683c691b839d5 27-May-2015 11:20:08 Finished task 'Checkout Default Repository' with result: Success

            Also, what git client are you using?
            Is it exactly the same executable and ssh configuration as for RDP client? User? Environment?

            Marek Went (Inactive) added a comment - Also, what git client are you using? Is it exactly the same executable and ssh configuration as for RDP client? User? Environment?

            What protocol are you using? ssh or https
            What authentication method are you using for git checkout (is 1answer is ssh)?
            Did you try key without passphrase (if second answer is private key)?

            Marek Went (Inactive) added a comment - What protocol are you using? ssh or https What authentication method are you using for git checkout (is 1answer is ssh)? Did you try key without passphrase (if second answer is private key)?

              Unassigned Unassigned
              16e8fd2916d2 Wayne Allan
              Affected customers:
              1 This affects my team
              Watchers:
              5 Start watching this issue

                Created:
                Updated:
                Resolved: