Uploaded image for project: 'Bamboo Data Center'
  1. Bamboo Data Center
  2. BAM-17299

Remote agent can't reconnect after a Bamboo server restart

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Medium
    • 5.10.3
    • 5.9.10, 5.10.2, 5.10.1.1, 7.2.1
    • Agents

    Description

      Summary

      Remote agent can't reconnect after a Bamboo server restart. Remote agent needs to be restarted.

      Steps to Reproduce

      1. Install and connect remote agent
      2. Restart Bamboo

      Expected Results

      Remote agent should automatically reconnect after Bamboo comes up

      Actual Results

      The below exception is thrown in the Bamboo server logs:

      2016-03-03 21:43:38,778 INFO [bambooHeartBeatMessageListenerConnector-1] [HeartbeatMessageListener] Agent sent an incorrect fingerprint in a JMS message Fingerprint[server: 5843657984118014849 run: 1631167983502063879], an attempt will be made to bring it down. Message:ActiveMQTextMessage {commandId = 79, responseRequired = true, messageId = ID:Derics-MBP.local-56638-1457011654725-1:1:4:1:7, originalDestination = null, originalTransactionId = null, producerId = ID:Derics-MBP.local-56638-1457011654725-1:1:4:1, destination = queue://com.atlassian.bamboo.heartbeatQueue, transactionId = null, expiration = 0, timestamp = 1457012618773, arrival = 0, brokerInTime = 1457012618774, brokerOutTime = 1457012618775, correlationId = null, replyTo = null, persistent = true, type = null, priority = 4, groupID = null, groupSequence = 0, targetConsumerId = null, compressed = false, userID = null, content = org.apache.activemq.util.ByteSequence@537c0a83, marshalledProperties = org.apache.activemq.util.ByteSequence@20fd4ce3, dataStructure = null, redeliveryCounter = 0, size = 2433, properties = {fingerprint=5843657984118014849, BambooUuidHash=725189461, instanceFingerprint=1631167983502063879}, readOnlyProperties = true, readOnlyBody = true, droppable = false, jmsXGroupFirstForConsumer = false, text = <com.atlassian.bamboo.v2.build.agent.messages...beatMessage>}
      

      The following is seen in the remote agent logs:

      2016-03-03 21:42:37,467 WARN [2-BAM::10.60.2.73::Agent:pool-5-thread-1] [DefaultBuildAgent] 1 attempts at building without a successful run. The agent will stop after reaching 10 exceptions.
      2016-03-03 21:42:37,471 INFO [2-BAM::10.60.2.73::Agent:pool-5-thread-1] [DefaultBuildAgent] Changing context: null -> null on 10.60.2.73/2e2d119e
      2016-03-03 21:42:37,473 INFO [2-BAM::10.60.2.73::Agent:pool-5-thread-1] [BuildAgentControllerImpl] Agent '10.60.2.73' ready to take build from queue...
      2016-03-03 21:42:37,473 INFO [2-BAM::10.60.2.73::Agent:pool-5-thread-1] [BuildAgentControllerImpl] Agent 393217 checking build queue for executables...
      2016-03-03 21:43:32,933 INFO [scheduler_Worker-3] [ClasslistCollectingConverter] Updating class list at remoting-whitelist.list
      2016-03-03 21:47:37,510 ERROR [2-BAM::10.60.2.73::Agent:pool-5-thread-1] [BuildAgentControllerImpl] Unknown exception occurred on '10.60.2.73'. Agent will attempt to recover its normal operation...
      org.springframework.remoting.RemoteAccessException: Could not access JMS invoker queue [queue://com.atlassian.bamboo.agentExecutableQueue]; nested exception is javax.jms.JMSException: A timeout has occurred during a remote call to the server.
      	at org.springframework.jms.remoting.JmsInvokerClientInterceptor.convertJmsInvokerAccessException(JmsInvokerClientInterceptor.java:433)
      	at org.springframework.jms.remoting.JmsInvokerClientInterceptor.invoke(JmsInvokerClientInterceptor.java:206)
      	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
      	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:207)
      	at com.sun.proxy.$Proxy15.takeBuildContext(Unknown Source)
      	at com.atlassian.bamboo.v2.build.agent.BuildAgentControllerImpl.getContextFromQueue(BuildAgentControllerImpl.java:222)
      	at com.atlassian.bamboo.v2.build.agent.BuildAgentControllerImpl.waitAndPerformBuild(BuildAgentControllerImpl.java:103)
      	at com.atlassian.bamboo.v2.build.agent.DefaultBuildAgent$1.run(DefaultBuildAgent.java:137)
      	at com.atlassian.bamboo.utils.BambooRunnables$1.run(BambooRunnables.java:49)
      	at com.atlassian.bamboo.security.ImpersonationHelper.runWith(ImpersonationHelper.java:31)
      	at com.atlassian.bamboo.security.ImpersonationHelper.runWithSystemAuthority(ImpersonationHelper.java:20)
      	at com.atlassian.bamboo.security.ImpersonationHelper$1.run(ImpersonationHelper.java:52)
      	at java.lang.Thread.run(Thread.java:745)
      Caused by: javax.jms.JMSException: A timeout has occurred during a remote call to the server.
      	at com.atlassian.bamboo.remoting.SpringRemotingMessageConverter.fromMessage(SpringRemotingMessageConverter.java:58)
      	at org.springframework.jms.remoting.JmsInvokerClientInterceptor.extractInvocationResult(JmsInvokerClientInterceptor.java:390)
      	at org.springframework.jms.remoting.JmsInvokerClientInterceptor.executeRequest(JmsInvokerClientInterceptor.java:254)
      	at org.springframework.jms.remoting.JmsInvokerClientInterceptor.invoke(JmsInvokerClientInterceptor.java:203)
      	... 11 more
      

      Remote agent does not appear to be online from Bamboo

      Workaround

      Restart remote agent

      Attachments

        Issue Links

          Activity

            People

              mgardias Marcin Gardias
              dlee@atlassian.com Deric Lee (Inactive)
              Votes:
              3 Vote for this issue
              Watchers:
              13 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: