Highest This morning at 14 Oct 2013, 9:51:05 PM NTT time, I've installed a plugin via UPM (Bamboo continuous deployment plugin) to JBAC (5.2-m1 build 4000 - 16 Sep 13):
2013-10-14 21:51:03,558 INFO [pool-17-thread-3] [PluginSettingsAuditLogService] Mon Oct 14 21:51:03 UTC 2013 anonymous: Installed add-on Bamboonicorn (com.atlassian.bamboo.plugins.bamboonicorn version 1.5)
All good? Around 1pm Syndey time, someone reported a problem with a stale version of the plugin still being used by the remote agent. I had a quick look at the Bamboo agent, and indeed, it didn't contain the deployed version 1.5, only old versions:
root@agent:/opt/bamboo-agent# find . -name "*bamboonicorn*" ./caches/plugins/transformed-plugins/plugin_466280401245839467_bamboonicorn-1.5-SNAPSHOT_1381364553000.jar ./caches/plugins/transformed-plugins/plugin_7339490292427437586_bamboonicorn-1.0_1380754342000.jar
Looking at the agent logs, it didn't install the plugin at all ...
@40000000525c672322cbf774 2013-10-14 21:50:17,583 INFO [remoteEventRebroadcasterMessageListenerContainer-1] [ClasspathBuilder] Downloading resource: https://jira-bamboo.internal.atlassian.com/agentServer/plugins.zip?fingerprint=641810321678362483 @40000000525c672322cdfefc 2013-10-14 21:50:17,583 INFO [remoteEventRebroadcasterMessageListenerContainer-1] [ClasspathBuilder] Generating a list of jars locally available in [/opt/bamboo-agent/plugins/user-installed] @40000000525c67232e8a739c 2013-10-14 21:50:17,780 INFO [remoteEventRebroadcasterMessageListenerContainer-1] [ClasspathBuilder] 14 file(s) locally available. @40000000525c6723368b9d8c 2013-10-14 21:50:17,914 INFO [remoteEventRebroadcasterMessageListenerContainer-1] [ClasspathBuilder] Finished downloading resource 'https://jira-bamboo.internal.atlassian.com/agentServer/plugins.zip?fingerprint=641810321678362483' .... expecting install plugin message here ..... @40000000525c67280286e76c 2013-10-14 21:50:22,042 INFO [remoteEventRebroadcasterMessageListenerContainer-1] [RemoteEventRebroadcasterMessageListener] Rebroadcasting com.atlassian.bamboo.plugin.events.EnableRemotePluginModuleEvent[source=null]
I've tried to disable and enable the plugin on the server but it didn't help. But when dkovacs installed the nodejs plugin 1.2, something happened:
@40000000525ccbbb3169f4ac 2013-10-15 04:59:29,829 INFO [remoteEventRebroadcasterMessageListenerContainer-1] [ClasspathBuilder] Downloading resource: https://jira-bamboo.internal.atlassian.com/agentServer/plugins.zip?fingerprint=641810321678362483 @40000000525ccbbb316e1744 2013-10-15 04:59:29,829 INFO [remoteEventRebroadcasterMessageListenerContainer-1] [ClasspathBuilder] Generating a list of jars locally available in [/opt/bamboo-agent/plugins/user-installed] @40000000525ccbbc0040cbe4 2013-10-15 04:59:30,004 INFO [remoteEventRebroadcasterMessageListenerContainer-1] [ClasspathBuilder] 14 file(s) locally available. @40000000525ccbc11fc500d4 2013-10-15 04:59:35,532 INFO [remoteEventRebroadcasterMessageListenerContainer-1] [ClasspathBuilder] Finished downloading resource 'https://jira-bamboo.internal.atlassian.com/agentServer/plugins.zip?fingerprint=641810321678362483' @40000000525ccbc11fd77764 2013-10-15 04:59:35,534 INFO [remoteEventRebroadcasterMessageListenerContainer-1] [ClasspathBuilder] Removing '/opt/bamboo-agent/plugins/user-installed/plugin_6465874740736937153_bamboo-nodejs-plugin-1.1.jar' from Agent classpath @40000000525ccbc11fdd3424 2013-10-15 04:59:35,534 INFO [remoteEventRebroadcasterMessageListenerContainer-1] [ClasspathBuilder] Creating '/opt/bamboo-agent/plugins/user-installed/plugin_3878635317327925933_bamboonicorn-1.5.jar'
root@agent:/opt/bamboo-agent# find . -name "*bamboonicorn*" ./plugins/user-installed/plugin_3878635317327925933_bamboonicorn-1.5.jar ./caches/plugins/transformed-plugins/plugin_7339490292427437586_bamboonicorn-1.0_1380754342000.jar ./caches/plugins/transformed-plugins/plugin_3878635317327925933_bamboonicorn-1.5_1381813175000.jar ./caches/plugins/transformed-plugins/plugin_466280401245839467_bamboonicorn-1.5-SNAPSHOT_1381364554000.jar
So now the plugin is actually installed.
All good? Now, nodejs 1.2 is missing 
root@agent:/opt/bamboo-agent# find . -name "*nodejs*" ./.npm/jsdoc/3.2.0/package/nodejs ./caches/plugins/transformed-plugins/plugin_6465874740736937153_bamboo-nodejs-plugin-1.1_1369710860000.jar
Not sure if this is an important detail, but the fingerprint argument in the download links is the same. Also, I downloaded the file referenced by https://jira-bamboo.internal.atlassian.com/agentServer/plugins.zip?fingerprint=641810321678362483 and it actually contains the nodejs-1.2 plugin:
~/Desktop$ unzip -t plugins.zip
Archive: plugins.zip
Produced by Atlassian Bamboo 5.2-m1-build-4000
testing: MANIFEST.DAT OK
testing: plugin_5310425800806923919_continuous-plugin-deployment-2.0.3.jar OK
testing: plugin_4764185046397382113_bamboo-variable-inject-plugin-1.0.2.jar OK
testing: plugin.5933531017049701189.variable-updater-1.0-beta-5.jar OK
testing: plugin.4126629850893206131.bamboo-plan-variable-updater-1.0.jar OK
testing: plugin_2484334879914474669_bamboo-agent-tasks-1.6.jar OK
testing: plugin_3878635317327925933_bamboonicorn-1.5.jar OK
testing: plugin.1304645780829897086.bamboo-nodejs-plugin-1.2.jar OK
testing: plugin_1361021839046379921_bamboo-predator-plugin-1.4-SNAPSHOT.jar OK
testing: plugin_5696379602079403040_brokerurldiscovery-1.0-alpha-10-SNAPSHOT.jar OK
testing: plugin_256218540451899789_bamboo-sandbox-plugin-2.1-beta20-SNAPSHOT.jar OK
testing: plugin_6579196846879878830_requirementtask-1.0-SNAPSHOT.jar OK
testing: plugin_4397322126637599494_auto-restart-1.0-beta-3-SNAPSHOT.jar OK
testing: plugin_8831074448745406797_atlassian-bamboo-plugin-statsd-1.0-SNAPSHOT.jar OK
testing: plugin_1161525732867247763_bamboo-koto-plugin-1.2-20130417.034653-1.jar OK
testing: plugin_4729981394989838276_bamboo-plan-templates-1.0-SNAPSHOT.jar OK
No errors detected in compressed data of plugins.zip.
So it looks like, it is not only ignoring to install a new version of the plugin on the remote agent, but also removes the old version. Are we sure the Bamboo server has finished rebuilding the plugins.zip before the remote agents are notified about the plugin upgrade?
