Uploaded image for project: 'Jira Data Center'
  1. Jira Data Center
  2. JRASERVER-60114

CachingOfBizPropertyEntryStore cache failed to start after Restore System

    XMLWordPrintable

Details

    Description

      Summary

      After running a system Restore into a single JIRA DC node, com.atlassian.jira.propertyset.CachingOfBizPropertyEntryStore cache failes to start.

      Environment

      This is intermittent problem, it doesn't reproduce all the time. So far problem was observed only with Jira Datacenter.

      Steps to Reproduce

      1. Start nodes
      2. Restore full backup.

      Expected Results

      All works fine and JIRA node re-initialise itself after import and run re-index.

      Actual Results

      JIRA node tries to re-initialise itself after import, during that step CachingOfBizPropertyEntryStore cache failed to start and JIRA doesn't run re-index at the end.

      The below exception is thrown in the atlassian-jira.log file:

      2016-03-07 16:06:52,345 JiraImportTaskExecutionThread-1 INFO admin 934x1687x1 8hen3m 127.0.0.1 /secure/admin/XmlRestore.jspa [jira.bc.dataimport.DefaultDataImportService] Importing data is 96% complete...
      2016-03-07 16:06:52,346 JiraImportTaskExecutionThread-1 WARN admin 934x1687x1 8hen3m 127.0.0.1 /secure/admin/XmlRestore.jspa [jira.config.properties.ApplicationPropertiesStore] Exception getting property 'jira.webresource.flushcounter' from database. Using default
      2016-03-07 16:06:52,347 JiraImportTaskExecutionThread-1 WARN admin 934x1687x1 8hen3m 127.0.0.1 /secure/admin/XmlRestore.jspa [jira.config.properties.ApplicationPropertiesStore] Exception getting property 'jira.webresource.flushcounter' from database. Using default
      2016-03-07 16:06:52,350 JiraImportTaskExecutionThread-1 ERROR admin 934x1687x1 8hen3m 127.0.0.1 /secure/admin/XmlRestore.jspa [jira.bc.dataimport.DefaultDataImportService] Error importing data: com.atlassian.cache.CacheException: java.lang.IllegalStateException: The com.atlassian.jira.propertyset.CachingOfBizPropertyEntryStore.entries Cache is not alive (STATUS_SHUTDOWN)
      com.atlassian.cache.CacheException: java.lang.IllegalStateException: The com.atlassian.jira.propertyset.CachingOfBizPropertyEntryStore.entries Cache is not alive (STATUS_SHUTDOWN)
      	at com.atlassian.cache.ehcache.DelegatingCache.remove(DelegatingCache.java:136)
      	at com.atlassian.jira.propertyset.CachingOfBizPropertyEntryStore.setEntry(CachingOfBizPropertyEntryStore.java:145)
      	at com.atlassian.jira.propertyset.CachingOfBizPropertySet.setImpl(CachingOfBizPropertySet.java:195)
      	at com.opensymphony.module.propertyset.AbstractPropertySet.set(AbstractPropertySet.java:502)
      	at com.opensymphony.module.propertyset.AbstractPropertySet.setString(AbstractPropertySet.java:300)
      	at com.atlassian.jira.config.properties.ApplicationPropertiesStore.setString(ApplicationPropertiesStore.java:237)
      	at com.atlassian.jira.config.properties.ApplicationPropertiesImpl.setString(ApplicationPropertiesImpl.java:118)
      	at com.atlassian.jira.config.properties.LookAndFeelBean.updateFlushCounter(LookAndFeelBean.java:109)
      	at com.atlassian.jira.config.properties.LookAndFeelBean.updateVersion(LookAndFeelBean.java:147)
      	at com.atlassian.jira.bc.dataimport.DefaultDataImportService.updateLookAndFeel(DefaultDataImportService.java:891)
      	at com.atlassian.jira.bc.dataimport.DefaultDataImportService.performImport(DefaultDataImportService.java:808)
      	at com.atlassian.jira.bc.dataimport.DefaultDataImportService.doImport(DefaultDataImportService.java:332)
      	at com.atlassian.jira.web.action.setup.DataImportAsyncCommand.call(DataImportAsyncCommand.java:65)
      	at com.atlassian.jira.web.action.setup.DataImportAsyncCommand.call(DataImportAsyncCommand.java:28)
      	at com.atlassian.jira.task.ImportTaskManagerImpl$TaskCallableDecorator.call(ImportTaskManagerImpl.java:177)
      	at com.atlassian.jira.task.ImportTaskManagerImpl$TaskCallableDecorator.call(ImportTaskManagerImpl.java:148)
      	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
      	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      	at java.lang.Thread.run(Thread.java:745)
      Caused by: java.lang.IllegalStateException: The com.atlassian.jira.propertyset.CachingOfBizPropertyEntryStore.entries Cache is not alive (STATUS_SHUTDOWN)
      	at net.sf.ehcache.Cache$CacheStatus.checkAlive(Cache.java:3946)
      	at net.sf.ehcache.Cache.checkStatus(Cache.java:2664)
      	at net.sf.ehcache.Cache.removeInternal(Cache.java:2288)
      	at net.sf.ehcache.Cache.remove(Cache.java:2207)
      	at net.sf.ehcache.Cache.remove(Cache.java:2125)
      	at com.atlassian.cache.ehcache.LoadingCache.remove(LoadingCache.java:205)
      	at com.atlassian.cache.ehcache.DelegatingCache.remove(DelegatingCache.java:132)
      	... 21 more
      2016-03-07 16:06:52,351 JiraImportTaskExecutionThread-1 INFO admin 934x1687x1 8hen3m 127.0.0.1 /secure/admin/XmlRestore.jspa [jira.index.ha.DefaultIndexCopyService] Index backup started. Requesting node: ANY
      2016-03-07 16:06:52,352 JiraImportTaskExecutionThread-1 WARN admin 934x1687x1 8hen3m 127.0.0.1 /secure/admin/XmlRestore.jspa [jira.index.ha.DefaultIndexCopyService] Index backup failed - latest index operation not found. Requesting node: ANY
      2016-03-07 16:06:52,354 JiraImportTaskExecutionThread-1 WARN admin 934x1687x1 8hen3m 127.0.0.1 /secure/admin/XmlRestore.jspa [dvcs.dao.impl.CachingOrganizationDaoImpl] onImportCompleted - clearing cache !!
      2016-03-07 16:06:52,354 JiraImportTaskExecutionThread-1 INFO admin 934x1687x1 8hen3m 127.0.0.1 /secure/admin/XmlRestore.jspa [jira.bc.dataimport.DefaultDataImportService] Importing data is 100% complete...
      2016-03-07 16:06:52,354 JiraImportTaskExecutionThread-1 INFO admin 934x1687x1 8hen3m 127.0.0.1 /secure/admin/XmlRestore.jspa [jira.bc.dataimport.DefaultDataImportService] JIRA Data Import has finished.
      

      Notes

      The result is that the indexing job did not kick off after importing the data. With that, the data has been successfully imported, but the index still needs updating.

      Workaround

      1. Restart node
      2. Run Full re-index

      Attachments

        Issue Links

          Activity

            People

              jreczycki Jakub Reczycki
              ayakovlev@atlassian.com Andriy Yakovlev [Atlassian]
              Votes:
              15 Vote for this issue
              Watchers:
              39 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: