Uploaded image for project: 'Bamboo Data Center'
  1. Bamboo Data Center
  2. BAM-4156

Build hangs saying that it is still building but the agent does not exist any more

    • Icon: Suggestion Suggestion
    • Resolution: Handled by Support
    • None
    • Agents, Builds
    • None
    • Our product teams collect and evaluate feedback from a number of different sources. To learn more about how we use customer feedback in the planning process, check out our new feature policy.

      Build status:

      Build Number: MYPROJECT-MYPROJECTTRUNK-70 (currently building)
      Building Started: 30/Jun/2009 12:35 PM
      Completed: 99%
      Time remaining: 861 minutes slower than usual
      Building on Agent: Elastic Agent on i-c1fcd0a8

      But in fact the agent is offline!!!

      Elastic Agent History >  Elastic Agent on i-c1fcd0a8 (elastic)
      Elastic agent on instance i-c1fcd0a8, configuration: 196609
      You can view historical information for this elastic agent in this page. This agent is offline and will never come back alive. 
      

      I was trying to press Stop Build button, but I only saw in UI's build log:

      Request to stop 'MyProject - MyProject trunk' received from Krystian Nowak
      

      and in Bamboo logs:

      2009-07-01 03:02:35,766 INFO [20715718@qtp-30289364-76] [CancelBuild] Request to stop 'MyProject - MyProject trunk' received from Krystian Nowak
      

        1. atlassian-bamboo-agent-core-3.0.2.jar
          79 kB
          Przemek Bruski
        2. atlassian-bamboo-agent-remote-3.0.2.jar
          53 kB
          Przemek Bruski
        3. atlassian-bamboo-api-3.0.2.jar
          400 kB
          Przemek Bruski
        4. atlassian-bamboo-core-3.0.2.jar
          1.48 MB
          Przemek Bruski

            [BAM-4156] Build hangs saying that it is still building but the agent does not exist any more

            MarkC added a comment -

            Please raise support requests on http://suppprt.atlassian.com if anyone see similar symptoms.

            MarkC added a comment - Please raise support requests on http://suppprt.atlassian.com if anyone see similar symptoms.

            SVN case: we won't be able to do much without access to thread dumps from the server (or the agent, if the problem occurred on the agent).

            Przemek Bruski added a comment - SVN case: we won't be able to do much without access to thread dumps from the server (or the agent, if the problem occurred on the agent).

            We saw this yesterday on two of our Bamboo plans running Bamboo 3.4.4 with SVN repositories.
            The builds both got stuck in the same "Updating repository" status.
            We ended up having to reboot the server to fix the issue. This is a really poor solution when we're trying to run 100 concurrent builds.

            David Corley added a comment - We saw this yesterday on two of our Bamboo plans running Bamboo 3.4.4 with SVN repositories. The builds both got stuck in the same "Updating repository" status. We ended up having to reboot the server to fix the issue. This is a really poor solution when we're trying to run 100 concurrent builds.

            Yes, that fixed it. The plan is past checkout and building now. Thanks for the pointer

            Jason Huntley added a comment - Yes, that fixed it. The plan is past checkout and building now. Thanks for the pointer

            Not really. Just reconfigure your plan and see if it helps.

            Przemek Bruski added a comment - Not really. Just reconfigure your plan and see if it helps.

            Hi Bruski,

            I'll try switching approaches. Is there any specific instructions you recommend following for switching from jgit to git?

            Jason Huntley added a comment - Hi Bruski, I'll try switching approaches. Is there any specific instructions you recommend following for switching from jgit to git?

            Przemek Bruski added a comment - - edited

            Jason, I think it's best if you open a support case. The issue you're seeing only looks the same as this one.
            Before that, you may want try switching between jgit and native git for your checkout.

            Przemek Bruski added a comment - - edited Jason, I think it's best if you open a support case . The issue you're seeing only looks the same as this one. Before that, you may want try switching between jgit and native git for your checkout.

            Nevermind, spoke too soon. It did get further, but only to the point of hanging again on the same checkout:

            26-Jan-2012 14:12:05 Updating source code to revision: 4a5e2953743fea6d16b9c262a1b732bbc8f46ecd
            26-Jan-2012 14:12:05 Creating local git repository in 'C:\development\bamboo-home\xml-data\build-dir_git-repositories-cache\618433d542edab33a64526aecf955033ec78bb13\.git'.
            26-Jan-2012 14:12:05 Fetching branch 'refs/heads/master' from 'C:\development\gitserver\repository\ha\archytas'.
            26-Jan-2012 14:12:05 Git: remote: Counting objects
            26-Jan-2012 14:12:05 Git: remote: Finding sources (8,084)
            26-Jan-2012 14:12:06 Git: remote: Getting sizes (4,263)
            26-Jan-2012 14:12:07 Git: remote: Compressing objects (4,258)
            26-Jan-2012 14:12:17 Git: 100% (4,258/4,258)
            26-Jan-2012 14:12:17 Git: Receiving objects (8,084)
            26-Jan-2012 14:12:27 Git: Resolving deltas (3,728)
            26-Jan-2012 14:12:37 Git: 81% (3,018/3,728)
            26-Jan-2012 14:12:40 Git: Updating references (1)
            26-Jan-2012 14:12:40 Git: Counting objects: 8084, done

            Total 8084 (delta 3644), reused 7930 (delta 3574)

            26-Jan-2012 14:12:40 Checking out revision 4a5e2953743fea6d16b9c262a1b732bbc8f46ecd.
            26-Jan-2012 14:12:40 Creating local git repository in 'C:\development\bamboo-home\xml-data\build-dir\ARCHYTAS-ARCHYTASGIT-JOB1\.git'.

            Jason Huntley added a comment - Nevermind, spoke too soon. It did get further, but only to the point of hanging again on the same checkout: 26-Jan-2012 14:12:05 Updating source code to revision: 4a5e2953743fea6d16b9c262a1b732bbc8f46ecd 26-Jan-2012 14:12:05 Creating local git repository in 'C:\development\bamboo-home\xml-data\build-dir_git-repositories-cache\618433d542edab33a64526aecf955033ec78bb13\.git'. 26-Jan-2012 14:12:05 Fetching branch 'refs/heads/master' from 'C:\development\gitserver\repository\ha\archytas'. 26-Jan-2012 14:12:05 Git: remote: Counting objects 26-Jan-2012 14:12:05 Git: remote: Finding sources (8,084) 26-Jan-2012 14:12:06 Git: remote: Getting sizes (4,263) 26-Jan-2012 14:12:07 Git: remote: Compressing objects (4,258) 26-Jan-2012 14:12:17 Git: 100% (4,258/4,258) 26-Jan-2012 14:12:17 Git: Receiving objects (8,084) 26-Jan-2012 14:12:27 Git: Resolving deltas (3,728) 26-Jan-2012 14:12:37 Git: 81% (3,018/3,728) 26-Jan-2012 14:12:40 Git: Updating references (1) 26-Jan-2012 14:12:40 Git: Counting objects: 8084, done Total 8084 (delta 3644), reused 7930 (delta 3574) 26-Jan-2012 14:12:40 Checking out revision 4a5e2953743fea6d16b9c262a1b732bbc8f46ecd. 26-Jan-2012 14:12:40 Creating local git repository in 'C:\development\bamboo-home\xml-data\build-dir\ARCHYTAS-ARCHYTASGIT-JOB1\.git'.

            I went back and turned off the "shallow copy" option and the build process stopped hanging.

            Jason Huntley added a comment - I went back and turned off the "shallow copy" option and the build process stopped hanging.

            I'm seeing this same issue in Bamboo 3.4.3 build 2811 - 11 Jan 12. The build hangs on the initial Git checkout and never returns. I try to cancel the build and bamboo doesn't respond. The process just seems stuck, no log entries, and the process is pegged.

            26-Jan-2012 14:01:49 Fetching branch 'refs/heads/master' from 'C:\development\gitserver\repository\ha\archytas'. Will try to do a shallow fetch.
            26-Jan-2012 14:01:49 Git: Updating references
            26-Jan-2012 14:01:49 Git:
            26-Jan-2012 14:01:49 Checking out revision 4a5e2953743fea6d16b9c262a1b732bbc8f46ecd.
            26-Jan-2012 14:01:49 Creating local git repository in 'C:\development\bamboo-home\xml-data\build-dir\ARCHYTAS-ARCHYTASGIT-JOB1\.git'.

            {qoute}

            Jason Huntley added a comment - I'm seeing this same issue in Bamboo 3.4.3 build 2811 - 11 Jan 12. The build hangs on the initial Git checkout and never returns. I try to cancel the build and bamboo doesn't respond. The process just seems stuck, no log entries, and the process is pegged. 26-Jan-2012 14:01:49 Fetching branch 'refs/heads/master' from 'C:\development\gitserver\repository\ha\archytas'. Will try to do a shallow fetch. 26-Jan-2012 14:01:49 Git: Updating references 26-Jan-2012 14:01:49 Git: 26-Jan-2012 14:01:49 Checking out revision 4a5e2953743fea6d16b9c262a1b732bbc8f46ecd. 26-Jan-2012 14:01:49 Creating local git repository in 'C:\development\bamboo-home\xml-data\build-dir\ARCHYTAS-ARCHYTASGIT-JOB1\.git'. {qoute}

              Unassigned Unassigned
              ac345ce11761 Krystian Nowak
              Votes:
              5 Vote for this issue
              Watchers:
              8 Start watching this issue

                Created:
                Updated:
                Resolved: