Uploaded image for project: 'Jira Server and Data Center'
  1. Jira Server and Data Center
  2. JRASERVER-72615

Jira 8.18.0 is unresponsive after around 4 hours due to stuck MpacApp cache

    XMLWordPrintable

    Details

      Description

      Issue Summary

      After upgrading to Jira 8.18, threads are stuck while the UPM is on the cache. The specific method com.atlassian.upm.pac.PacClientImpl.mpacApp is stuck on the cache.

      java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
      java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
      java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireShared(AbstractQueuedSynchronizer.java:967)
      java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(AbstractQueuedSynchronizer.java:1283)
      java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.lock(ReentrantReadWriteLock.java:727)
      net.sf.ehcache.concurrent.ReadWriteLockSync.lock(ReadWriteLockSync.java:50)
      net.sf.ehcache.Cache.acquireLockOnKey(Cache.java:3950)
      net.sf.ehcache.Cache.acquireReadLockOnKey(Cache.java:3964)
      net.sf.ehcache.constructs.EhcacheDecoratorAdapter.acquireReadLockOnKey(EhcacheDecoratorAdapter.java:744)
      net.sf.ehcache.constructs.EhcacheDecoratorAdapter.acquireReadLockOnKey(EhcacheDecoratorAdapter.java:744)
      com.atlassian.cache.ehcache.DelegatingCachedReference.getIfPresent(DelegatingCachedReference.java:123)
      com.atlassian.upm.pac.PacClientImpl.mpacApp(PacClientImpl.java:202)
      com.atlassian.upm.pac.PacClientImpl.unknownAppVersion(PacClientImpl.java:206)
      com.atlassian.upm.pac.PacClientImpl.isUnknownProductVersion(PacClientImpl.java:198)
      com.atlassian.upm.permission.UpmVisibilityImpl.isKnownOrDevelopmentVersion(UpmVisibilityImpl.java:72)
      com.atlassian.upm.permission.UpmVisibilityImpl.isInstallVisible(UpmVisibilityImpl.java:45)
      com.atlassian.upm.conditions.IsPermittedToView.shouldDisplay(IsPermittedToView.java:61)
      ...
      

      REST API requests are processed successfully. The stuck threads identified are any user facing pages. All the stuck threads were rendering the header and footer (all threads were on this after the servlet filter stack):

      ...
      com.atlassian.jira.plugin.navigation.HeaderFooterRendering$7.visit(HeaderFooterRendering.java:354)
      com.atlassian.jira.plugin.navigation.HeaderFooterRendering$7.visit(HeaderFooterRendering.java:346)
      com.atlassian.ozymandias.SafePluginPointAccess.invokeModule(SafePluginPointAccess.java:644)
      com.atlassian.ozymandias.SafePluginPointAccess.visitPluginPointImpl(SafePluginPointAccess.java:325)
      com.atlassian.ozymandias.SafePluginPointAccess.descriptors(SafePluginPointAccess.java:213)
      com.atlassian.jira.plugin.navigation.HeaderFooterRendering.includeTopNavigation(HeaderFooterRendering.java:346)
      com.atlassian.jira.plugin.navigation.HeaderFooterRendering$1.run(HeaderFooterRendering.java:151)
      com.atlassian.ozymandias.SafePluginPointAccess.runnable(SafePluginPointAccess.java:404)
      com.atlassian.jira.plugin.navigation.HeaderFooterRendering.includeTopNavigation(HeaderFooterRendering.java:148)
      jsp.decorators.general_jsp._jspService(general_jsp.java:291)
      ...
      

      The specific UPM version 5.0.0-2ee9e3d93d is delivered with 8.18.0:

               Atlassian Universal Plugin Manager Plugin     : com.atlassian.upm.atlassian-universal-plugin-manager-plugin
                    Version                                       : 5.0.0-2ee9e3d93d
                    Status                                        : enabled
                    Vendor                                        : Atlassian
                    Description                                   : This is the plugin that provides the Atlassian Universal Plugin Manager.
      

      Steps to Reproduce

      1. Upgrading to Jira 8.18 includes the UPM version 5.0.0-2ee9e3d93d
      2. Start Jira and wait for around 4 hours

      Expected Results

      Jira should be stable and not have a problem loading pages.

      Actual Results

      Around 4 hours after starting Jira, customer facing pages are hanging/unresponsive.

      Workaround

      • The PacClient hangs trying to load the version. Workaround is to add the JVM argument below and restart Jira. This will disable the PacClient that hangs, and is not critical for Jira to run properly. The operation needs to be applied to every Jira DC node individually.
        -Dupm.pac.disable=true
        

        Note: After disabling the PacClient, you won’t be able to install or update apps directly from the Atlassian Marketplace. Those operations will still be available thought manual upload of the apps to your Jira instance via "Manage apps" page.

      • Same may be done via the Jira UI: System Administration -> Manage addons -> Settings (bottom of the page) -> Uncheck "Connect to the Atlassian Marketplace" checkbox -> Apply changes. Change is cluster-wide.

       

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              pczuj Przemyslaw Czuj
              Reporter:
              6d6f6a289aac Konde
              Votes:
              9 Vote for this issue
              Watchers:
              39 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: