Uploaded image for project: 'Bitbucket Data Center'
  1. Bitbucket Data Center
  2. BSERV-12805

JMX monitoring can lead to heap exhaustion

XMLWordPrintable

      Issue Summary

      In Bitbucket Server/Data Center 7.5, the concept of asynchronous database migration was introduced as part of the new build status functionality. To allow administrators to track in-progress async migration, a JMX bean was added that they can query.

      However, each time that bean is queried, a third-party library which gets used can end up leaking a number of objects via a static map. One of the leaked objects has a reference to a JDBC Connection, which may also lead to unexpected connection issues in addition to heap exhaustion.

      Steps to Reproduce

      1. Enable JMX by setting jmx.enabled=true in bitbucket.properties
      2. Poll AsyncMigrationUpgrade via JMX while performing other work that uses database connections

      Expected Results

      The system is able to run indefinitely without heap exhaustion or any connection-related errors.

      Actual Results

      After some time (where "some time" can be on the order of weeks), the system will run out of memory. A heap dump will show a significant amount of memory retained by LockServiceFactory.

      Workaround

      Disable JMX. The code which triggers the resource leak is only accessible via JMX.

              bturner Bryan Turner (Inactive)
              bturner Bryan Turner (Inactive)
              Votes:
              1 Vote for this issue
              Watchers:
              6 Start watching this issue

                Created:
                Updated:
                Resolved: