Uploaded image for project: 'Confluence Data Center'
  1. Confluence Data Center
  2. CONFSERVER-23301

Database deadlock during upgrade of plugins

    XMLWordPrintable

Details

    Description

      As originally described in my comments to CONF-21986, Confluence has a bad habit of reporting deadlocks and failing to handle plugin installations gracefully.

      This was observed when trying to upgrade the Adaptavist Plugin Licensing Manager plugin to v0.6.1, while running a recently-upgraded Confluence 4.0 in a production environment:

      2011-09-23 11:51:39,838 INFO [Spring executor 11] [atlassian.plugin.manager.DefaultPluginManager] notifyPluginDisabled Disabling com.adaptavist.confluence.contentFormattingMacros
      2011-09-23 11:51:43,336 INFO [Spring executor 11] [atlassian.plugin.manager.DefaultPluginManager] disableDependentPlugins Found dependent enabled plugins for uninstalled plugin 'com.adaptavist.plm.plugin.plm-plugin': [].  Disabling...
      2011-09-23 11:51:43,337 INFO [Spring executor 11] [atlassian.plugin.manager.DefaultPluginManager] notifyPluginDisabled Disabling com.adaptavist.plm.plugin.plm-plugin
      2011-09-23 11:52:35,488 ERROR [Spring executor 11] [sf.hibernate.util.JDBCExceptionReporter] logExceptions Lock wait timeout exceeded; try restarting transaction
       -- referer: http://172.16.16.10:8090/confluence/plugins/servlet/upm | url: /confluence/rest/plugins/1.0/upgrades/all | userName: scott
      2011-09-23 11:52:35,490 ERROR [Spring executor 11] [sf.hibernate.impl.SessionImpl] execute Could not synchronize database state with session
       -- referer: http://172.16.16.10:8090/confluence/plugins/servlet/upm | url: /confluence/rest/plugins/1.0/upgrades/all | userName: scott
      2011-09-23 11:52:40,333 INFO [pool-6-thread-1] [atlassian.plugin.manager.DefaultPluginManager] notifyPluginDisabled Disabling com.adaptavist.plm.plugin.plm-plugin-installer
      

      We initially discovered what appears to be the same issue when developing one of our own plugins, which may shed some additional light on the problem. It seems that our plugin's Spring initialization is getting invoked while Confluence still holds a lock on some part of the plugin table. Our initialization code sometimes tries to modify other plugin modules (primarily, we call PluginController.disablePluginModule to disable incompatible modules of other plugins), which sometimes provokes a deadlock identical tot he above:

      2011-09-16 16:04:21,439 INFO [pool-6-thread-2] [atlassian.plugin.manager.DefaultPluginManager] disableDependentPlugins Found dependent enabled plugins for uninstalled plugin 'com.arsenale.plugins.lockpoint': []. Disabling...
      2011-09-16 16:04:21,441 INFO [pool-6-thread-2] [atlassian.plugin.manager.DefaultPluginManager] updatePlugin Updating plugin 'com.arsenale.plugins.lockpoint' to 'com.arsenale.plugins.lockpoint'
      2011-09-16 16:04:21,443 INFO [pool-6-thread-2] [atlassian.plugin.manager.DefaultPluginManager] notifyPluginDisabled Disabling com.arsenale.plugins.lockpoint
      2011-09-16 16:04:23,133 ERROR [http-8090-5] [sf.hibernate.util.JDBCExceptionReporter] logExceptions Deadlock found when trying to get lock; try restarting transaction
      - referer: http://172.16.16.10:8090/plugins/servlet/upm | url: /rest/plugins/1.0/pending/008decc2-f4fb-421f-a4f0-6d05489257d7 | userName: admin
      2011-09-16 16:04:23,134 ERROR [http-8090-5] [sf.hibernate.impl.SessionImpl] execute Could not synchronize database state with session
      - referer: http://172.16.16.10:8090/plugins/servlet/upm | url: /rest/plugins/1.0/pending/008decc2-f4fb-421f-a4f0-6d05489257d7 | userName: admin
      

      If we call the above method too soon after our plugin gets initialized, we deadlock. If we wait a while before trying to disable the other plugin modules, everything works correctly. One correlating factor is that if we have not seen our own PluginInstallEvent by the time we try to call disablePluginModule(), we seem to deadlock. But once we've received that event, we seem to be okay.

      In either case, the upgraded plugin is dead in the water after attempting the installation.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              7c60ab039b09 Scott Dudley [Inactive]
              Votes:
              6 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: