Loading...

XML

Word

Printable

Details

Type: Bug
Resolution: Fixed
Priority: Highest
Fix Version/s: 8.14.0
Affects Version/s: 8.0.0, 8.10.0, 8.5.8, 8.13.0
Component/s: Indexing
Labels:
- l1l2
- lts-backport
- lts-top-ask
- lts813
- pse-request
- warranty

Introduced in Version:
8
Support reference count:
71
Symptom Severity:
Severity 3 - Minor
UIS:
83
Bug Fix Policy:
View Atlassian Server bug fix policy

Description

Atlassian Update - 9 October 2020

We found out that the ResultSet closed error is caused by a misalignment of various properties that control batch and buffer sizes during indexing.

We aligned these properties in the upcoming Jira 8.14.0 release.

If this problem affects your instance running Jira < 8.14.0, you can apply the fix yourself by setting the following values in jira-config.properties:

jira.index.issue.maxqueuesize=4000
jira.index.sharedentity.maxqueuesize=4000

To be double sure, you can ensure that your instance does use the default value of jira.index.background.batch.size = 4000. You can check this on the Advanced Settings page (requires Jira System Administrator rights).

Issue Summary

Customers are experiencing upon upgrading to Jira 8.0+ from 7.0+ that full lock and re-index runs into ResultSet closed error from their database (happens on SqlServer and Postgres so far).

The trend from customers is that most of the time their dbconfig.xml has default max pool size of 20 threads, and Jira 8.0+ has default re-index thread of 20.

Steps to Reproduce

Unable to reproduce, requires data and load.

Expected Results

Reindex completes successfully.

Actual Results

Reindex fails with the error message -

2019-11-13 15:57:44,805 JiraTaskExecutionThread-4 ERROR xxxx xxxxxxx 297u5a xx.xxx.xx.xx,127.0.0.1 /secure/admin/IndexReIndex!reindex.jspa [c.a.j.util.index.CompositeIndexLifecycleManager] Reindex All FAILED.  Indexer: DefaultIndexManager: paths: [/srv/jira/data/caches/indexesV1/comments, /srv/jira/data/caches/indexesV1/issues, /srv/jira/data/caches/indexesV1/changes, /srv/jira/data/caches/indexesV1/worklogs]
org.ofbiz.core.util.GeneralRuntimeException: Error getting the next result (This ResultSet is closed.)
	at org.ofbiz.core.entity.EntityListIterator.next(EntityListIterator.java:272)
	at com.atlassian.jira.ofbiz.DefaultOfBizListIterator.next(DefaultOfBizListIterator.java:100)
	at com.atlassian.jira.ofbiz.WrappingOfBizListIterator.next(WrappingOfBizListIterator.java:68)
	at com.atlassian.jira.issue.util.DatabaseIssuesIterator.pullNextIssue(DatabaseIssuesIterator.java:97)
	at com.atlassian.jira.issue.util.DatabaseIssuesIterator.populateNextIssueIfNull(DatabaseIssuesIterator.java:68)
	at com.atlassian.jira.issue.util.DatabaseIssuesIterator.hasNext(DatabaseIssuesIterator.java:51)
	at com.atlassian.jira.issue.util.DatabaseIssuesIterable.foreach(DatabaseIssuesIterable.java:55)
	at com.atlassian.jira.issue.index.DefaultIssueIndexer.perform(DefaultIssueIndexer.java:335)
	at com.atlassian.jira.issue.index.DefaultIssueIndexer.indexIssuesBatchMode(DefaultIssueIndexer.java:182)
	at com.atlassian.jira.issue.index.DefaultIssueIndexer.indexIssuesBatchMode(DefaultIssueIndexer.java:198)
	at com.atlassian.jira.issue.index.DefaultIndexManager.doIndexIssuesInBatchMode(DefaultIndexManager.java:1023)
	at com.atlassian.jira.issue.index.DefaultIndexManager.doStopTheWorldReindex(DefaultIndexManager.java:1002)
	at com.atlassian.jira.issue.index.DefaultIndexManager.lambda$reIndexAll$0(DefaultIndexManager.java:331)
	at com.atlassian.jira.issue.index.DefaultIndexManager.withReindexLock(DefaultIndexManager.java:378)
	at com.atlassian.jira.issue.index.DefaultIndexManager.reIndexAll(DefaultIndexManager.java:331)
	at com.atlassian.jira.issue.index.DefaultIndexManager.reIndexAll(DefaultIndexManager.java:264)
	... 3 filtered
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.atlassian.jira.config.component.SwitchingInvocationHandler.invoke(SwitchingInvocationHandler.java:38)
	at com.sun.proxy.$Proxy9.reIndexAll(Unknown Source)
	at com.atlassian.jira.util.index.CompositeIndexLifecycleManager.reIndexAll(CompositeIndexLifecycleManager.java:66)
	at com.atlassian.jira.util.index.CompositeIndexLifecycleManager.reIndexAll(CompositeIndexLifecycleManager.java:49)
	at com.atlassian.jira.web.action.admin.index.ReIndexAsyncIndexerCommand.doReindex(ReIndexAsyncIndexerCommand.java:27)
	at com.atlassian.jira.web.action.admin.index.AbstractAsyncIndexerCommand.call(AbstractAsyncIndexerCommand.java:63)
	at com.atlassian.jira.web.action.admin.index.ReIndexAsyncIndexerCommand.call(ReIndexAsyncIndexerCommand.java:18)
	at com.atlassian.jira.web.action.admin.index.AbstractAsyncIndexerCommand.call(AbstractAsyncIndexerCommand.java:26)
	at com.atlassian.jira.task.TaskManagerImpl$TaskCallableDecorator.call(TaskManagerImpl.java:537)
	at com.atlassian.jira.task.TaskManagerImpl$TaskCallableDecorator.call(TaskManagerImpl.java:495)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at com.atlassian.jira.task.ForkedThreadExecutor$ForkedRunnableDecorator.run(ForkedThreadExecutor.java:216)
	at java.lang.Thread.run(Thread.java:748)
Caused by: org.postgresql.util.PSQLException: This ResultSet is closed.
	at org.postgresql.jdbc.PgResultSet.checkClosed(PgResultSet.java:2696)
	at org.postgresql.jdbc.PgResultSet.next(PgResultSet.java:1769)
	at org.apache.commons.dbcp2.DelegatingResultSet.next(DelegatingResultSet.java:191)
	at org.apache.commons.dbcp2.DelegatingResultSet.next(DelegatingResultSet.java:191)
	at org.ofbiz.core.entity.EntityListIterator.next(EntityListIterator.java:266)

The following might also be found in the logs too, showing that the pool sized has maxed out

Dangerous use of multiple connections: taken => count=2; marks=[1-0]; pool=20/20

Workaround

If this problem affects your instance running Jira < 8.14.0, you can apply the fix yourself by setting the following values in jira-config.properties:

jira.index.issue.maxqueuesize=4000
jira.index.sharedentity.maxqueuesize=4000

September 2020 Update:

We are closely monitoring ~~JRASERVER-70327~~, and, as it is a complex issue, we’ve decided to address it in parts. As a first step, we released a fix for ~~JRASERVER-65457~~ that should should mitigate some of the problem. We will continue to closely follow the current ticket and will take further steps based on how it develops.

Attachments

- Sort By Name
- Sort By Date
- Ascending
- Descending
- Thumbnails
- List

screenshot-1.png
1.01 MB
18/Dec/2019 8:59 AM

Issue Links

is related to

JRASERVER-68484 As a Jira Admin I want to configure a tolerate threshold for amount of errors during Jira reindexing

Gathering Interest

Mentioned in

CSS Top Asks - Jira On Prem: JRASERVER-70327 | ResultSet Closed when doing a lock and reindex on Jira 8.0+

mentioned in: Page Loading...; Page Loading...; Page Loading...; Page Loading...; Page Loading...; Page Loading...; Page Loading...; Page Loading...; Page Loading...; Page Loading...; Page Loading...; Page Loading...; Page Loading...; Page Loading...; Page Loading...; Page Loading...; Page Loading...; Page Loading...; Page Loading...; Page Loading...; Page Loading...

(18 mentioned in)

Activity

People

Assignee:: Unassigned

Reporter:: Dave (Inactive)

Votes:: 15 Vote for this issue

Watchers:: 46 Start watching this issue

Dates

Created:: 28/Nov/2019 12:45 AM

Updated:: 30/Mar/2023 1:59 PM

Resolved:: 23/Nov/2020 2:39 PM