Uploaded image for project: 'Jira Server and Data Center'
  1. Jira Server and Data Center
  2. JRASERVER-64908

UPM actions may flush internal caches, leading to performance problems

    XMLWordPrintable

    Details

    • Introduced in Version:
      6.04
    • Support reference count:
      142
    • Symptom Severity:
      Severity 1 - Critical
    • UIS:
      2,510
    • Current Status:
      Hide
      Atlassian Update – 26 April 2019

      Dear Jira users,

      we’re glad to announce that this issue will be addressed in our upcoming 8.2.0 release!

      We’ve fixed several issues that caused performance degradation in large Jira instances when installing or updating apps. If you’ve encountered this problem before, it’s solved, so feel free to update your apps anytime. For technical details please go this comment.

      We plan to back port this fix to version 7.13 (our latest Enterprise Release version). Please watch the fix versions in this issue for further updates.

      Looking forward for your comments.

      Thank you,
      Grazyna Kaszkur

      Product Manager,
      Jira Server and Data Center

      Show
      Atlassian Update – 26 April 2019 Dear Jira users, we’re glad to announce that this issue will be addressed in our upcoming 8.2.0 release! We’ve fixed several issues that caused performance degradation in large Jira instances when installing or updating apps. If you’ve encountered this problem before, it’s solved, so feel free to update your apps anytime. For technical details please go this comment. We plan to back port this fix to version 7.13 (our latest Enterprise Release version). Please watch the fix versions in this issue for further updates. Looking forward for your comments. Thank you, Grazyna Kaszkur
 Product Manager, Jira Server and Data Center

      Description

      Summary

      During plugin (add-on) installs/updates (including license updates) /delete/disable action UPM will flush the large number of caches. Under high load that will cause significant JIRA performance degradation and might cause JIRA to freeze/stall for a long time.

      Steps to Reproduce

      1. Apply high load to JIRA
      2. Upgrade any plugin.

      Expected Results

      • JIRA performs fast, could be slight performance degradation.
      • JIRA admin is notified about the impact

      Actual Results

      • JIRA admin is not notified about the impact
      • Thread dump might show the following thread UpmAsynchronousTaskManager:thread
        "UpmAsynchronousTaskManager:thread-1" #38891 prio=5 os_prio=0 tid=0x00007fbda8047800 nid=0x6712 waiting on condition [0x00007fbcf9cd5000]
           java.lang.Thread.State: WAITING (parking)
        	at sun.misc.Unsafe.park(Native Method)
        	- parking to wait for  <0x000000056c734418> (a java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync)
        ...
        	at net.sf.ehcache.concurrent.ReadWriteLockSync.lock(ReadWriteLockSync.java:50)
        	at com.atlassian.cache.ehcache.LoadingCache.remove(LoadingCache.java:190)
        	at com.atlassian.cache.ehcache.DelegatingCachedReference.reset(DelegatingCachedReference.java:79)
        	at com.atlassian.jira.issue.search.managers.DefaultSearchHandlerManager.refresh(DefaultSearchHandlerManager.java:290)
        	at com.atlassian.jira.issue.search.managers.DefaultIssueSearcherManager.refresh(DefaultIssueSearcherManager.java:46)
        	at com.atlassian.jira.issue.fields.DefaultFieldManager.refreshSearchersAndIndexers(DefaultFieldManager.java:804)
        	at com.atlassian.jira.issue.fields.DefaultFieldManager.refresh(DefaultFieldManager.java:706)
        	at com.atlassian.jira.plugin.JiraCacheResetter$Delegate.resetCaches(JiraCacheResetter.java:83)
        	at com.atlassian.jira.plugin.JiraCacheResetter$Delegate.onPluginModuleDisabled(JiraCacheResetter.java:53)
        ...
        	at com.atlassian.plugin.event.impl.DefaultPluginEventManager.broadcast(DefaultPluginEventManager.java:73)
        	at com.atlassian.plugin.manager.DefaultPluginManager.broadcastIgnoreError(DefaultPluginManager.java:2081)
        	at com.atlassian.plugin.manager.DefaultPluginManager.publishModuleDisabledEvents(DefaultPluginManager.java:1955)
        	at com.atlassian.plugin.manager.DefaultPluginManager.disablePluginModuleNoPersist(DefaultPluginManager.java:1910)
        	at com.atlassian.plugin.manager.DefaultPluginManager.disablePluginModules(DefaultPluginManager.java:1904)
        	at com.atlassian.plugin.manager.DefaultPluginManager.disablePluginWithModuleEvents(DefaultPluginManager.java:1860)
        	at com.atlassian.plugin.manager.DefaultPluginManager.notifyPluginDisabled(DefaultPluginManager.java:1888)
        	at com.atlassian.plugin.manager.DefaultPluginManager.unloadPlugin(DefaultPluginManager.java:1022)
        	at com.atlassian.plugin.manager.DefaultPluginManager.uninstallNoEvent(DefaultPluginManager.java:981)
        	at com.atlassian.plugin.manager.DefaultPluginManager.updatePlugin(DefaultPluginManager.java:1343)
        	at com.atlassian.plugin.manager.DefaultPluginManager.addPlugins(DefaultPluginManager.java:1133)
        	at com.atlassian.jira.plugin.JiraPluginManager.addPlugins(JiraPluginManager.java:150)
        	at com.atlassian.plugin.manager.DefaultPluginManager.scanForNewPlugins(DefaultPluginManager.java:903)
        	at com.atlassian.plugin.manager.DefaultPluginManager.installPlugins(DefaultPluginManager.java:821)
        	at com.atlassian.jira.plugin.JiraPluginManager.installPlugins(JiraPluginManager.java:168)
        ...
        

      Notes

      Problem is amplified at JIRA Datacenter. Cache flush is triggered during node startup as UPM needs to register and load modules from plugins. See JRASERVER-66839

      Workaround

      Please plan any plugin actions updates/delete/disable (UPM actions) during low peak hours or maintenance windows. See Best Practices for Managing JIRA Application Add-ons for recommendations.

        Attachments

        1. plugin_disable_80.png
          plugin_disable_80.png
          341 kB
        2. plugin_disable_82.png
          plugin_disable_82.png
          133 kB

          Issue Links

            Activity

              People

              Assignee:
              mswinarski Maciej Swinarski
              Reporter:
              ayakovlev@atlassian.com Andriy Yakovlev
              Votes:
              82 Vote for this issue
              Watchers:
              124 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: