Uploaded image for project: 'Jira Server and Data Center'
  1. Jira Server and Data Center
  2. JRASERVER-64908

UPM action will flush the cache and cause JIRA to stall

    XMLWordPrintable

    Details

    • Introduced in Version:
      6.04
    • Support reference count:
      142
    • Symptom Severity:
      Severity 1 - Critical
    • Occurrence Factor:
      50%
    • UIS:
      2,510
    • Current Status:
      Hide
      Atlassian Update – 26 April 2019

      Dear Jira users,

      we’re glad to announce that this issue will be addressed in our upcoming 8.2.0 release!

      We’ve fixed several issues that caused performance degradation in large Jira instances when installing or updating apps. If you’ve encountered this problem before, it’s solved, so feel free to update your apps anytime. For technical details please go this comment.

      We plan to back port this fix to version 7.13 (our latest Enterprise Release version). Please watch the fix versions in this issue for further updates.

      Looking forward for your comments.

      Thank you,
      Grazyna Kaszkur

      Product Manager,
      Jira Server and Data Center

      Show
      Atlassian Update – 26 April 2019 Dear Jira users, we’re glad to announce that this issue will be addressed in our upcoming 8.2.0 release! We’ve fixed several issues that caused performance degradation in large Jira instances when installing or updating apps. If you’ve encountered this problem before, it’s solved, so feel free to update your apps anytime. For technical details please go this comment. We plan to back port this fix to version 7.13 (our latest Enterprise Release version). Please watch the fix versions in this issue for further updates. Looking forward for your comments. Thank you, Grazyna Kaszkur
 Product Manager, Jira Server and Data Center

      Description

      Summary

      During plugin (add-on) installs/updates (including license updates) /delete/disable action UPM will flush the large number of caches. Under high load that will cause significant JIRA performance degradation and might cause JIRA to freeze/stall for a long time.

      Steps to Reproduce

      1. Apply high load to JIRA
      2. Upgrade any plugin.

      Expected Results

      • JIRA performs fast, could be slight performance degradation.
      • JIRA admin is notified about the impact

      Actual Results

      • JIRA admin is not notified about the impact
      • Thread dump might show the following thread UpmAsynchronousTaskManager:thread
        "UpmAsynchronousTaskManager:thread-1" #38891 prio=5 os_prio=0 tid=0x00007fbda8047800 nid=0x6712 waiting on condition [0x00007fbcf9cd5000]
           java.lang.Thread.State: WAITING (parking)
        	at sun.misc.Unsafe.park(Native Method)
        	- parking to wait for  <0x000000056c734418> (a java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync)
        ...
        	at net.sf.ehcache.concurrent.ReadWriteLockSync.lock(ReadWriteLockSync.java:50)
        	at com.atlassian.cache.ehcache.LoadingCache.remove(LoadingCache.java:190)
        	at com.atlassian.cache.ehcache.DelegatingCachedReference.reset(DelegatingCachedReference.java:79)
        	at com.atlassian.jira.issue.search.managers.DefaultSearchHandlerManager.refresh(DefaultSearchHandlerManager.java:290)
        	at com.atlassian.jira.issue.search.managers.DefaultIssueSearcherManager.refresh(DefaultIssueSearcherManager.java:46)
        	at com.atlassian.jira.issue.fields.DefaultFieldManager.refreshSearchersAndIndexers(DefaultFieldManager.java:804)
        	at com.atlassian.jira.issue.fields.DefaultFieldManager.refresh(DefaultFieldManager.java:706)
        	at com.atlassian.jira.plugin.JiraCacheResetter$Delegate.resetCaches(JiraCacheResetter.java:83)
        	at com.atlassian.jira.plugin.JiraCacheResetter$Delegate.onPluginModuleDisabled(JiraCacheResetter.java:53)
        ...
        	at com.atlassian.plugin.event.impl.DefaultPluginEventManager.broadcast(DefaultPluginEventManager.java:73)
        	at com.atlassian.plugin.manager.DefaultPluginManager.broadcastIgnoreError(DefaultPluginManager.java:2081)
        	at com.atlassian.plugin.manager.DefaultPluginManager.publishModuleDisabledEvents(DefaultPluginManager.java:1955)
        	at com.atlassian.plugin.manager.DefaultPluginManager.disablePluginModuleNoPersist(DefaultPluginManager.java:1910)
        	at com.atlassian.plugin.manager.DefaultPluginManager.disablePluginModules(DefaultPluginManager.java:1904)
        	at com.atlassian.plugin.manager.DefaultPluginManager.disablePluginWithModuleEvents(DefaultPluginManager.java:1860)
        	at com.atlassian.plugin.manager.DefaultPluginManager.notifyPluginDisabled(DefaultPluginManager.java:1888)
        	at com.atlassian.plugin.manager.DefaultPluginManager.unloadPlugin(DefaultPluginManager.java:1022)
        	at com.atlassian.plugin.manager.DefaultPluginManager.uninstallNoEvent(DefaultPluginManager.java:981)
        	at com.atlassian.plugin.manager.DefaultPluginManager.updatePlugin(DefaultPluginManager.java:1343)
        	at com.atlassian.plugin.manager.DefaultPluginManager.addPlugins(DefaultPluginManager.java:1133)
        	at com.atlassian.jira.plugin.JiraPluginManager.addPlugins(JiraPluginManager.java:150)
        	at com.atlassian.plugin.manager.DefaultPluginManager.scanForNewPlugins(DefaultPluginManager.java:903)
        	at com.atlassian.plugin.manager.DefaultPluginManager.installPlugins(DefaultPluginManager.java:821)
        	at com.atlassian.jira.plugin.JiraPluginManager.installPlugins(JiraPluginManager.java:168)
        ...
        

      Notes

      Problem is amplified at JIRA Datacenter. Cache flush is triggered during node startup as UPM needs to register and load modules from plugins. See JRASERVER-66839

      Workaround

      Please plan any plugin actions updates/delete/disable (UPM actions) during low peak hours or maintenance windows. See Best Practices for Managing JIRA Application Add-ons for recommendations.

        Attachments

        1. plugin_disable_80.png
          plugin_disable_80.png
          341 kB
        2. plugin_disable_82.png
          plugin_disable_82.png
          133 kB

          Issue Links

            Activity

              People

              • Assignee:
                mswinarski Maciej Swinarski
                Reporter:
                ayakovlev@atlassian.com Andriy Yakovlev
              • Votes:
                82 Vote for this issue
                Watchers:
                116 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: