Uploaded image for project: 'Bitbucket Server'
  1. Bitbucket Server
  2. BSERV-12805

JMX monitoring can lead to heap exhaustion

    XMLWordPrintable

Details

    Description

      Issue Summary

      In Bitbucket Server/Data Center 7.5, the concept of asynchronous database migration was introduced as part of the new build status functionality. To allow administrators to track in-progress async migration, a JMX bean was added that they can query.

      However, each time that bean is queried, a third-party library which gets used can end up leaking a number of objects via a static map. One of the leaked objects has a reference to a JDBC Connection, which may also lead to unexpected connection issues in addition to heap exhaustion.

      Steps to Reproduce

      1. Enable JMX by setting jmx.enabled=true in bitbucket.properties
      2. Poll AsyncMigrationUpgrade via JMX while performing other work that uses database connections

      Expected Results

      The system is able to run indefinitely without heap exhaustion or any connection-related errors.

      Actual Results

      After some time (where "some time" can be on the order of weeks), the system will run out of memory. A heap dump will show a significant amount of memory retained by LockServiceFactory.

      Workaround

      Disable JMX. The code which triggers the resource leak is only accessible via JMX.

      Attachments

        Activity

          People

            bturner Bryan Turner (Inactive)
            bturner Bryan Turner (Inactive)
            Votes:
            1 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Backbone Issue Sync