We have occasional builds that hang (apparently) because the build logging information from the agent is no longer communicated to the server even though the build is still running, the agent is still connected, etc. We have long running builds and the lack of logging triggers the hung build event, but the build on the agent is actually still running and runs to completion. It even appears that the agent successfully runs the post-build tasks such as transferring artifacts, but the server status remains hung with no log updates and never "finishes" the build.

Last night we had this happen to four builds at once. In the attached bamboo.log file, at 2009/04/16 18:23:15 you can see four builds all trigger the hung build detection at the same time. We have a 60 minute limit on that, so it was about one hour earlier when the logging stopped, which we saw in the build activity pages and the download-data\build_logs. In the agent logs, you can see a long heartbeat message around 17:23, but our heartbeat timeout on the server is 240s, so the agents remained "attached".

It's probably worth pointing out that all of these machines are Hyper-V virtual machines and occasionally subject to strange behavior depending on the physical host. In this case, though, the agents that hung were spread across two different virtualization servers, so it seems more likely the problem originated on the server and/or its VM.

Please ignore the extra attachments (can't remove). Everything is in the zip file.