Uploaded image for project: 'Jira Data Center'
  1. Jira Data Center
  2. JRASERVER-70327

ResultSet Closed when doing a lock and reindex on Jira 8.0+

XMLWordPrintable

      Atlassian Update - 9 October 2020

      We found out that the ResultSet closed error is caused by a misalignment of various properties that control batch and buffer sizes during indexing.

      We aligned these properties in the upcoming Jira 8.14.0 release.

      If this problem affects your instance running Jira < 8.14.0, you can apply the fix yourself by setting the following values in jira-config.properties:

      jira.index.issue.maxqueuesize=4000
jira.index.sharedentity.maxqueuesize=4000

      To be double sure, you can ensure that your instance does use the default value of jira.index.background.batch.size = 4000. You can check this on the Advanced Settings page (requires Jira System Administrator rights).

       

      Issue Summary

      Customers are experiencing upon upgrading to Jira 8.0+ from 7.0+ that full lock and re-index runs into ResultSet closed error from their database (happens on SqlServer and Postgres so far).

      The trend from customers is that most of the time their dbconfig.xml has default max pool size of 20 threads, and Jira 8.0+ has default re-index thread of 20.

      Steps to Reproduce

      Unable to reproduce, requires data and load.

      Expected Results

      Reindex completes successfully.

      Actual Results

      Reindex fails with the error message -

      2019-11-13 15:57:44,805 JiraTaskExecutionThread-4 ERROR xxxx xxxxxxx 297u5a xx.xxx.xx.xx,127.0.0.1 /secure/admin/IndexReIndex!reindex.jspa [c.a.j.util.index.CompositeIndexLifecycleManager] Reindex All FAILED.  Indexer: DefaultIndexManager: paths: [/srv/jira/data/caches/indexesV1/comments, /srv/jira/data/caches/indexesV1/issues, /srv/jira/data/caches/indexesV1/changes, /srv/jira/data/caches/indexesV1/worklogs]
org.ofbiz.core.util.GeneralRuntimeException: Error getting the next result (This ResultSet is closed.)
	at org.ofbiz.core.entity.EntityListIterator.next(EntityListIterator.java:272)
	at com.atlassian.jira.ofbiz.DefaultOfBizListIterator.next(DefaultOfBizListIterator.java:100)
	at com.atlassian.jira.ofbiz.WrappingOfBizListIterator.next(WrappingOfBizListIterator.java:68)
	at com.atlassian.jira.issue.util.DatabaseIssuesIterator.pullNextIssue(DatabaseIssuesIterator.java:97)
	at com.atlassian.jira.issue.util.DatabaseIssuesIterator.populateNextIssueIfNull(DatabaseIssuesIterator.java:68)
	at com.atlassian.jira.issue.util.DatabaseIssuesIterator.hasNext(DatabaseIssuesIterator.java:51)
	at com.atlassian.jira.issue.util.DatabaseIssuesIterable.foreach(DatabaseIssuesIterable.java:55)
	at com.atlassian.jira.issue.index.DefaultIssueIndexer.perform(DefaultIssueIndexer.java:335)
	at com.atlassian.jira.issue.index.DefaultIssueIndexer.indexIssuesBatchMode(DefaultIssueIndexer.java:182)
	at com.atlassian.jira.issue.index.DefaultIssueIndexer.indexIssuesBatchMode(DefaultIssueIndexer.java:198)
	at com.atlassian.jira.issue.index.DefaultIndexManager.doIndexIssuesInBatchMode(DefaultIndexManager.java:1023)
	at com.atlassian.jira.issue.index.DefaultIndexManager.doStopTheWorldReindex(DefaultIndexManager.java:1002)
	at com.atlassian.jira.issue.index.DefaultIndexManager.lambda$reIndexAll$0(DefaultIndexManager.java:331)
	at com.atlassian.jira.issue.index.DefaultIndexManager.withReindexLock(DefaultIndexManager.java:378)
	at com.atlassian.jira.issue.index.DefaultIndexManager.reIndexAll(DefaultIndexManager.java:331)
	at com.atlassian.jira.issue.index.DefaultIndexManager.reIndexAll(DefaultIndexManager.java:264)
	... 3 filtered
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.atlassian.jira.config.component.SwitchingInvocationHandler.invoke(SwitchingInvocationHandler.java:38)
	at com.sun.proxy.$Proxy9.reIndexAll(Unknown Source)
	at com.atlassian.jira.util.index.CompositeIndexLifecycleManager.reIndexAll(CompositeIndexLifecycleManager.java:66)
	at com.atlassian.jira.util.index.CompositeIndexLifecycleManager.reIndexAll(CompositeIndexLifecycleManager.java:49)
	at com.atlassian.jira.web.action.admin.index.ReIndexAsyncIndexerCommand.doReindex(ReIndexAsyncIndexerCommand.java:27)
	at com.atlassian.jira.web.action.admin.index.AbstractAsyncIndexerCommand.call(AbstractAsyncIndexerCommand.java:63)
	at com.atlassian.jira.web.action.admin.index.ReIndexAsyncIndexerCommand.call(ReIndexAsyncIndexerCommand.java:18)
	at com.atlassian.jira.web.action.admin.index.AbstractAsyncIndexerCommand.call(AbstractAsyncIndexerCommand.java:26)
	at com.atlassian.jira.task.TaskManagerImpl$TaskCallableDecorator.call(TaskManagerImpl.java:537)
	at com.atlassian.jira.task.TaskManagerImpl$TaskCallableDecorator.call(TaskManagerImpl.java:495)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at com.atlassian.jira.task.ForkedThreadExecutor$ForkedRunnableDecorator.run(ForkedThreadExecutor.java:216)
	at java.lang.Thread.run(Thread.java:748)
Caused by: org.postgresql.util.PSQLException: This ResultSet is closed.
	at org.postgresql.jdbc.PgResultSet.checkClosed(PgResultSet.java:2696)
	at org.postgresql.jdbc.PgResultSet.next(PgResultSet.java:1769)
	at org.apache.commons.dbcp2.DelegatingResultSet.next(DelegatingResultSet.java:191)
	at org.apache.commons.dbcp2.DelegatingResultSet.next(DelegatingResultSet.java:191)
	at org.ofbiz.core.entity.EntityListIterator.next(EntityListIterator.java:266)

      The following might also be found in the logs too, showing that the pool sized has maxed out

      Dangerous use of multiple connections: taken => count=2; marks=[1-0]; pool=20/20

      Workaround

      If this problem affects your instance running Jira < 8.14.0, you can apply the fix yourself by setting the following values in jira-config.properties:

      jira.index.issue.maxqueuesize=4000
jira.index.sharedentity.maxqueuesize=4000

      To be double sure, you can ensure that your instance does use the default value of jira.index.background.batch.size = 4000. You can check this on the Advanced Settings page (requires Jira System Administrator rights).

      September 2020 Update:

      We are closely monitoring JRASERVER-70327, and, as it is a complex issue, we’ve decided to address it in parts. As a first step, we released a fix for JRASERVER-65457 that should should mitigate some of the problem. We will continue to closely follow the current ticket and will take further steps based on how it develops.

        1. screenshot-1.png
          screenshot-1.png
          1.01 MB

              Unassigned Unassigned
              dung@atlassian.com Dave (Inactive)
              Votes:
              15 Vote for this issue
              Watchers:
              46 Start watching this issue

                Created:
                Updated:
                Resolved: