Details
-
Bug
-
Resolution: Fixed
-
Low
-
6.2.5, 6.4.6, 6.4.7, 6.4.8
-
6.02
-
Description
Description:
When using the Background Reindex or the Index Recovery functionality, users will encounter a 500 error page which will display a stack trace similar to the following:
Referer URL: http://jira-dev/secure/admin/jira/IndexProgress.jspa?taskId=15203
java.lang.NullPointerException java.lang.NullPointerException
at com.atlassian.jira.web.action.admin.index.IndexAdminImpl.doProgress(IndexAdminImpl.java:395)
at sun.reflect.GeneratedMethodAccessor1683.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at webwork.util.InjectionUtils$DefaultInjectionImpl.invoke(InjectionUtils.java:70)
at webwork.util.InjectionUtils.invoke(InjectionUtils.java:56)
at webwork.action.ActionSupport.invokeCommand(ActionSupport.java:438)
at webwork.action.ActionSupport.execute(ActionSupport.java:157)
at com.atlassian.jira.action.JiraActionSupport.execute(JiraActionSupport.java:88)
at webwork.interceptor.DefaultInterceptorChain.proceed(DefaultInterceptorChain.java:39)
at webwork.interceptor.NestedInterceptorChain.proceed(NestedInterceptorChain.java:31)
at webwork.interceptor.ChainedInterceptor.intercept(ChainedInterceptor.java:16)
at webwork.interceptor.DefaultInterceptorChain.proceed(DefaultInterceptorChain.java:35)
at webwork.dispatcher.GenericDispatcher.executeAction(GenericDispatcher.java:225)
at webwork.dispatcher.GenericDispatcher.executeAction(GenericDispatcher.java:154)
at com.atlassian.jira.web.dispatcher.JiraWebworkActionDispatcher.service(JiraWebworkActionDispatcher.java:152)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
This bug also causes index replication on JIRA Data Center to fail with the stack trace below:
2015-08-01 18:20:17,174 ClusterMessageHandlerServiceThread:thread-1 INFO [jira.index.ha.DefaultIndexCopyService] Index restore started. Snapshot file: IndexSnapshot_13203.zip 2015-08-01 18:21:20,011 ClusterMessageHandlerServiceThread:thread-1 INFO [jira.index.ha.DefaultIndexRecoveryManager] Recovering search indexes - 1% complete... 2015-08-01 18:21:21,375 ClusterMessageHandlerServiceThread:thread-1 DEBUG [jira.issue.index.DefaultIndexManager] Activating indexes in '/apps/instances/jira_12000/home/caches/indexes'. 2015-08-01 18:23:18,071 ClusterMessageHandlerServiceThread:thread-1 INFO [jira.index.ha.DefaultIndexRecoveryManager] Recovering search indexes - 20% complete... 2015-08-01 18:23:18,071 ClusterMessageHandlerServiceThread:thread-1 ERROR [atlassian.jira.cluster.OfBizMessageHandlerService] There was a problem handling a cluster message java.lang.NullPointerException at com.atlassian.jira.index.ha.ReindexMetadata.getIndexStartTime(ReindexMetadata.java:96) at com.atlassian.jira.index.ha.DefaultIndexRecoveryManager$ReplaceIndexRunner.calculateDurationToRecover(DefaultIndexRecoveryManager.java:399) at com.atlassian.jira.index.ha.DefaultIndexRecoveryManager$ReplaceIndexRunner.run(DefaultIndexRecoveryManager.java:384) at com.atlassian.jira.issue.index.DefaultIndexManager.withReindexLock(DefaultIndexManager.java:382) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) <+2> (NativeMethodAccessorImpl.java:62) (DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:483) at com.atlassian.util.profiling.object.ObjectProfiler.profiledInvoke(ObjectProfiler.java:83) at com.atlassian.jira.config.component.SwitchingInvocationHandler.invoke(SwitchingInvocationHandler.java:28) at com.sun.proxy.$Proxy22.withReindexLock(Unknown Source) at com.atlassian.jira.index.ha.DefaultIndexRecoveryManager.recoverIndexFromBackup(DefaultIndexRecoveryManager.java:124) (...) at java.lang.Thread.run(Thread.java:745)
It seems like the actual indexing process (for Background Reindex or Index Recovery) completes in the logs:
Example from Background Reindex
2015-06-19 09:11:38,563 ISSUE-indexQueue:thread-1 INFO SPNG3 763x115x1 1g85w3l 10.0.8.177 /secure/admin/IndexReIndex.jspa [action.admin.index.IndexAdminImpl] Re-indexing is 97% complete. Current index: Issue 2015-06-19 09:11:43,912 JiraTaskExectionThread-4 INFO SPNG3 533x85327x2 bn3lrn 10.0.8.177 /secure/admin/IndexReIndex.jspa [jira.issue.index.DefaultIndexManager] 79795 issues reindexed in the background, in 1078825 millis. 2015-06-19 09:11:43,929 JiraTaskExectionThread-4 ERROR SPNG3 533x85327x2 bn3lrn 10.0.8.177 /secure/admin/IndexReIndex.jspa [jira.util.index.CompositeIndexLifecycleManager] Reindex All In Background FAILED. Indexer: DefaultIndexManager: paths: [/data/jira/caches/indexes/comments, /data/jira/caches/indexes/issues, /data/jira/caches/indexes/changes, /data/jira/caches/indexes/worklogs] java.lang.NullPointerException at com.atlassian.jira.issue.index.DefaultIssueIndexer$3.consume(DefaultIssueIndexer.java:337)... 2015-06-19 09:11:43,931 JiraTaskExectionThread-4 INFO SPNG3 533x85327x2 bn3lrn 10.0.8.177 /secure/admin/IndexReIndex.jspa [action.admin.index.IndexAdminImpl] Re-indexing finished
Example from Index Recovery
2015-07-01 07:46:54,003 JiraTaskExectionThread-1 INFO spjsf 466x624023x1 17ake2e 10.0.8.78 /secure/admin/jira/IndexRecover.jspa [action.admin.index.IndexAdminImpl] Re-indexing started 2015-07-01 07:46:58,918 JiraTaskExectionThread-1 INFO spjsf 466x624023x1 17ake2e 10.0.8.78 /secure/admin/jira/IndexRecover.jspa [jira.index.ha.DefaultIndexRecoveryManager] Recovering search indexes - 1% complete... 2015-07-01 07:47:05,305 JiraTaskExectionThread-1 INFO spjsf 466x624023x1 17ake2e 10.0.8.78 /secure/admin/jira/IndexRecover.jspa [jira.index.ha.DefaultIndexRecoveryManager] Recovering search indexes - 20% complete... 2015-07-01 07:47:05,341 JiraTaskExectionThread-1 INFO spjsf 466x624023x1 17ake2e 10.0.8.78 /secure/admin/jira/IndexRecover.jspa [action.admin.index.IndexAdminImpl] Index Recovery finished
Steps to Reproduce
I have not been able to reproduce this locally with Background Reindexing, but I HAVE been able to reproduce locally using the Index Recovery functionality. These steps refer to that process
- Set up Index Recovery
- Allow for the Index Recovery process to run at the appointed time (for testing, you can set the time for 5 or 10 minutes in the future)
- Once an index recovery backup zip is created, enter in the appropriate zip file name into the UI and click Recover.
- Near the end of the recovery process, note the 500 error page.
Workaround
Since the actual indexing process seems to work fine, a workaround may not be needed. However, users can run a full, locking index in lieu of either process to rebuild their indexes from the database if their situations warrant it.