[JRASERVER-72688] Reindexing in jira + Postgres frequently fails with : "Cannot change transaction isolation level in the middle of a transaction"

Type: Bug
Resolution: Fixed
Priority: Medium (View bug fix roadmap)
Fix Version/s: 10.2.0
Affects Version/s: 8.17.0, 8.13.8, 9.4.0, 9.4.2
Component/s: Data Center - Other
Labels:
- database
- delta
- whl-fy24q3
- whl-fy24q4
- whl-fy25q1
- whl-fy25q2

Introduced in Version:
8.13
Support reference count:
59
Symptom Severity:
Severity 2 - Major
UIS:
27
Bug Fix Policy:
View Atlassian Server bug fix policy
Current Status:

Hide

Atlassian Update – 5 November 2024

Dear Customers,

We're happy to announce that this issue has been fixed in the 10.2.0 release. At this point, we cannot say if it will be possible to backport this fix to the LTS (9.4.x and 9.12.x). Please continue watching this ticket for future updates on the possibility of backporting.

Summary of the problem

The root cause of the issue was tracked to the commons-dbcp2 library. The details of the issue can be found at https://issues.apache.org/jira/browse/DBCP-585. In short, the library permitted concurrent access to a database connection, which was supported in the past by database vendors. However, that support was dropped at various points in time and can lead to potential complications. For Postgres specifically, the issue presented itself in Jira as ~~JRASERVER-72688~~.

We implemented a fix in the library that allows disabling of Connection instance-level JMX tracking while still maintaining other types of tracking, including tracking at the Connections level (see https://issues.apache.org/jira/browse/DBCP-585?focusedCommentId=17516575&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17516575 for details). Consequently, we have utilized that flag in Jira to disable the problematic functionality.

As a result, the issue is fixed and we plan to close the ticket with the release of version 10.2.0.

New behavior after the change

For the reasons described above, we no longer expose Connection instance-level metrics through JMX Exporter in Jira while still maintaining other types of tracking, including tracking at the Connections level.

To envision the difference, compare the screenshots linked below from JavaMissionControl. Up until now, we exposed Connection-level metrics together with the Connection instance-level metrics. We no longer expose the latter while still maintaining all the information from Connection-level metrics.

If you need to restore the previous behavior, you can use the parameter -Ddbcp.registerConnectionMBean=true to override those settings. However, it's important to note that this workaround is not recommended, as it may bring back symptoms fixed in ~~JRASERVER-72688~~ or cause other unknown problems resulting from the thread races in the underlying library (see link for details).

if the absence of the Connection instance-level metrics breaks your monitoring solution, please consider raising a separate ticket.

Best regards

Jacek Foremski

Jira DC Principal Software Engineer

Show
Atlassian Update – 5 November 2024 Dear Customers, We're happy to announce that this issue has been fixed in the 10.2.0 release. At this point, we cannot say if it will be possible to backport this fix to the LTS (9.4.x and 9.12.x). Please continue watching this ticket for future updates on the possibility of backporting. Summary of the problem The root cause of the issue was tracked to the commons-dbcp2 library. The details of the issue can be found at https://issues.apache.org/jira/browse/DBCP-585 . In short, the library permitted concurrent access to a database connection, which was supported in the past by database vendors. However, that support was dropped at various points in time and can lead to potential complications. For Postgres specifically, the issue presented itself in Jira as JRASERVER-72688 . We implemented a fix in the library that allows disabling of Connection instance-level JMX tracking while still maintaining other types of tracking, including tracking at the Connections level (see https://issues.apache.org/jira/browse/DBCP-585?focusedCommentId=17516575&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17516575 for details). Consequently, we have utilized that flag in Jira to disable the problematic functionality. As a result, the issue is fixed and we plan to close the ticket with the release of version 10.2.0 . New behavior after the change For the reasons described above, we no longer expose Connection instance-level metrics through JMX Exporter in Jira while still maintaining other types of tracking, including tracking at the Connections level. To envision the difference, compare the screenshots linked below from JavaMissionControl. Up until now, we exposed Connection-level metrics together with the Connection instance-level metrics . We no longer expose the latter while still maintaining all the information from Connection-level metrics . If you need to restore the previous behavior, you can use the parameter -Ddbcp.registerConnectionMBean=true to override those settings. However, it's important to note that this workaround is not recommended, as it may bring back symptoms fixed in JRASERVER-72688 or cause other unknown problems resulting from the thread races in the underlying library (see link for details). if the absence of the Connection instance-level metrics breaks your monitoring solution, please consider raising a separate ticket. Best regards Jacek Foremski Jira DC Principal Software Engineer

Issue Summary

While reindexing Jira(which is configured with Postgres mainly), it fails with : "Cannot change transaction isolation level in the middle of a transaction", somewhere at the end of whole process.

Steps to Reproduce

Install Jira 8.13 and above with Postgres.
Populate set of data on this environment or restore back up from existing reproducible environment.
Re-index.

Expected Results

Reindex should succeed.

Actual Results

It fails with following error:

Caused by: com.atlassian.jira.transaction.TransactionRuntimeException: org.ofbiz.core.entity.GenericTransactionException: Error occurred while starting transaction. (Cannot change transaction isolation level in the middle of a transaction.)
	at com.atlassian.jira.transaction.TransactionSupportImpl.beginTxn(TransactionSupportImpl.java:54)
	at com.atlassian.jira.transaction.TransactionSupportImpl.startedTransaction(TransactionSupportImpl.java:46)
	at com.atlassian.jira.transaction.TransactionSupportImpl.begin(TransactionSupportImpl.java:31)
[....clipped....]
	at com.atlassian.jira.versioning.EntityVersioningManagerImpl.getRelatedCommentVersions(EntityVersioningManagerImpl.java:137)
[....clipped....]
Caused by: org.ofbiz.core.entity.GenericTransactionException: Error occurred while starting transaction. (Cannot change transaction isolation level in the middle of a transaction.)
	at org.ofbiz.core.entity.TransactionUtil.beginLocalTransaction(TransactionUtil.java:296)
	at com.atlassian.core.ofbiz.util.CoreTransactionUtil.begin(CoreTransactionUtil.java:54)
	at com.atlassian.jira.transaction.TransactionSupportImpl.beginTxn(TransactionSupportImpl.java:52)
	... 22 more
Caused by: org.postgresql.util.PSQLException: Cannot change transaction isolation level in the middle of a transaction.
	at org.apache.commons.dbcp2.DelegatingConnection.setTransactionIsolation(DelegatingConnection.java:573)
	at org.apache.commons.dbcp2.DelegatingConnection.setTransactionIsolation(DelegatingConnection.java:573)
	at org.ofbiz.core.entity.jdbc.interceptors.connection.DelegatingConnection.setTransactionIsolation(DelegatingConnection.java:93)
	at com.atlassian.jira.ofbiz.sql.ConnectionWrapper.setTransactionIsolation(ConnectionWrapper.java:112)
	at org.ofbiz.core.entity.TransactionUtil.beginLocalTransaction(TransactionUtil.java:289)

Workaround

The issues is triggered by JMX queries to MBeans with naming pattern "com.atlassian.jira:connectionpool=connections,connection=X,name=BasicDataSource", where "X" changes with each new connection creation. This behaviour is due to DBCP-585. In majority of cases, we notice that this query is initiated via "Prometheus JMX Exporter" as it queries all MBeans with default configuration. This exporter may be configured via command line, or may be shipped inside the plugin Prometheus Experter for Jira. Removing exporter and restarting the server resolves the issue.

If unable to remove Prometheus another work around would be adding the startup parameter -Ddbcp.jmx=false to the setenv.sh script then restart Jira.

is related to

JRASERVER-68484 As a Jira Admin I want to configure a tolerate threshold for amount of errors during Jira reindexing

Gathering Interest

mentioned in: Page Failed to load; Page Failed to load; Page Failed to load; Page Failed to load; Page Loading...; Page Loading...; Page Loading...

(3 mentioned in)

Jacek Foremski added a comment - 05/Nov/2024 3:04 PM - edited

Atlassian Update – 5 November 2024

Dear Customers,

We're happy to announce that this issue has been fixed in the 10.2.0 release. At this point, we cannot say if it will be possible to backport this fix to the LTS (9.4.x and 9.12.x). Please continue watching this ticket for future updates on the possibility of backporting.

Summary of the problem

The root cause of the issue was tracked to the commons-dbcp2 library. The details of the issue can be found at https://issues.apache.org/jira/browse/DBCP-585. In short, the library permitted concurrent access to a database connection, which was supported in the past by database vendors. However, that support was dropped at various points in time and can lead to potential complications. For Postgres specifically, the issue presented itself in Jira as ~~JRASERVER-72688~~.

We implemented a fix in the library that allows disabling of Connection instance-level JMX tracking while still maintaining other types of tracking, including tracking at the Connections level (see https://issues.apache.org/jira/browse/DBCP-585?focusedCommentId=17516575&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17516575 for details). Consequently, we have utilized that flag in Jira to disable the problematic functionality.

As a result, the issue is fixed and we plan to close the ticket with the release of version 10.2.0.

New behavior after the change

For the reasons described above, we no longer expose Connection instance-level metrics through JMX Exporter in Jira while still maintaining other types of tracking, including tracking at the Connections level.

To envision the difference, compare the screenshots linked below from JavaMissionControl. Up until now, we exposed Connection-level metrics together with the Connection instance-level metrics. We no longer expose the latter while still maintaining all the information from Connection-level metrics.

If you need to restore the previous behavior, you can use the parameter -Ddbcp.registerConnectionMBean=true to override those settings. However, it's important to note that this workaround is not recommended, as it may bring back symptoms fixed in ~~JRASERVER-72688~~ or cause other unknown problems resulting from the thread races in the underlying library (see link for details).

if the absence of the Connection instance-level metrics breaks your monitoring solution, please consider raising a separate ticket.

Best regards

Jacek Foremski

Jira DC Principal Software Engineer

Jacek Foremski added a comment - 05/Nov/2024 3:04 PM - edited Atlassian Update – 5 November 2024 Dear Customers, We're happy to announce that this issue has been fixed in the 10.2.0 release. At this point, we cannot say if it will be possible to backport this fix to the LTS (9.4.x and 9.12.x). Please continue watching this ticket for future updates on the possibility of backporting. Summary of the problem The root cause of the issue was tracked to the commons-dbcp2 library. The details of the issue can be found at https://issues.apache.org/jira/browse/DBCP-585 . In short, the library permitted concurrent access to a database connection, which was supported in the past by database vendors. However, that support was dropped at various points in time and can lead to potential complications. For Postgres specifically, the issue presented itself in Jira as JRASERVER-72688 . We implemented a fix in the library that allows disabling of Connection instance-level JMX tracking while still maintaining other types of tracking, including tracking at the Connections level (see https://issues.apache.org/jira/browse/DBCP-585?focusedCommentId=17516575&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17516575 for details). Consequently, we have utilized that flag in Jira to disable the problematic functionality. As a result, the issue is fixed and we plan to close the ticket with the release of version 10.2.0 . New behavior after the change For the reasons described above, we no longer expose Connection instance-level metrics through JMX Exporter in Jira while still maintaining other types of tracking, including tracking at the Connections level. To envision the difference, compare the screenshots linked below from JavaMissionControl. Up until now, we exposed Connection-level metrics together with the Connection instance-level metrics . We no longer expose the latter while still maintaining all the information from Connection-level metrics . If you need to restore the previous behavior, you can use the parameter -Ddbcp.registerConnectionMBean=true to override those settings. However, it's important to note that this workaround is not recommended, as it may bring back symptoms fixed in JRASERVER-72688 or cause other unknown problems resulting from the thread races in the underlying library (see link for details). if the absence of the Connection instance-level metrics breaks your monitoring solution, please consider raising a separate ticket. Best regards Jacek Foremski Jira DC Principal Software Engineer

Alex [Atlassian,PSE] added a comment - 06/Dec/2023 3:49 AM

this got fixed in upstream 2.10

Jira 9.4 uses version 2.1
Jira 9.12 uses version 2.9

Alex [Atlassian,PSE] added a comment - 06/Dec/2023 3:49 AM this got fixed in upstream 2.10 Jira 9.4 uses version 2.1 Jira 9.12 uses version 2.9

Александр Никига added a comment - 16/Dec/2022 1:26 AM

... 25 more
at org.ofbiz.core.entity.TransactionUtil.beginLocalTransaction(TransactionUtil.java:289)
at com.atlassian.jira.ofbiz.sql.ConnectionWrapper.setTransactionIsolation(ConnectionWrapper.java:112)
at org.ofbiz.core.entity.jdbc.interceptors.connection.DelegatingConnection.setTransactionIsolation(DelegatingConnection.java:93)
at org.apache.commons.dbcp2.DelegatingConnection.setTransactionIsolation(DelegatingConnection.java:573)
at org.apache.commons.dbcp2.DelegatingConnection.setTransactionIsolation(DelegatingConnection.java:573)
Caused by: org.postgresql.util.PSQLException: Cannot change transaction isolation level in the middle of a transaction.
... 23 more
at com.atlassian.jira.transaction.TransactionSupportImpl.beginTxn(TransactionSupportImpl.java:52)
at com.atlassian.core.ofbiz.util.CoreTransactionUtil.begin(CoreTransactionUtil.java:54)
at org.ofbiz.core.entity.TransactionUtil.beginLocalTransaction(TransactionUtil.java:296)
Caused by: org.ofbiz.core.entity.GenericTransactionException: Error occurred while starting transaction. (Cannot change transaction isolation level in the middle of a transaction.)
... 10 more
at com.atlassian.jira.issue.index.DefaultIssueIndexer$EntityOperation.perform(DefaultIssueIndexer.java:847)
at com.atlassian.jira.issue.index.DefaultIssueIndexer$IssuesOperation.createDocuments(DefaultIssueIndexer.java:982)
at com.atlassian.jira.issue.index.DefaultIssueIndexer$DefaultDocumentCreationStrategy.get(DefaultIssueIndexer.java:1320)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1541)
at com.atlassian.jira.issue.index.DefaultIssueIndexer$DefaultDocumentCreationStrategy.lambda$get$1(DefaultIssueIndexer.java:1328)
at com.atlassian.jira.issue.index.DefaultWorklogRetriever.retrieve(DefaultWorklogRetriever.java:27)
at com.atlassian.jira.versioning.EntityVersioningManagerWithStats.getRelatedWorklogVersions(EntityVersioningManagerWithStats.java:240)
at com.atlassian.jira.versioning.EntityVersioningManagerImpl.getRelatedWorklogVersions(EntityVersioningManagerImpl.java:146)
at com.atlassian.jira.versioning.TransactionSupportHelper.executeWithRequired(TransactionSupportHelper.java:46)
at com.atlassian.jira.transaction.TransactionSupport.execute(TransactionSupport.java:26)
at com.atlassian.jira.transaction.TransactionSupportImpl.begin(TransactionSupportImpl.java:31)
at com.atlassian.jira.transaction.TransactionSupportImpl.startedTransaction(TransactionSupportImpl.java:46)
at com.atlassian.jira.transaction.TransactionSupportImpl.beginTxn(TransactionSupportImpl.java:54)
Caused by: com.atlassian.jira.transaction.TransactionRuntimeException: org.ofbiz.core.entity.GenericTransactionException: Error occurred while starting transaction. (Cannot change transaction isolation level in the middle of a transaction.)
at java.base/java.lang.Thread.run(Thread.java:834)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at com.atlassian.jira.util.concurrent.BoundedExecutor$2.call(BoundedExecutor.java:68)
at com.atlassian.jira.index.MultiThreadedIndexingStrategy$1.call(MultiThreadedIndexingStrategy.java:43)
at com.atlassian.jira.index.MultiThreadedIndexingStrategy$1.call(MultiThreadedIndexingStrategy.java:47)
at com.atlassian.jira.index.SimpleIndexingStrategy.apply(SimpleIndexingStrategy.java:5)
at com.atlassian.jira.index.SimpleIndexingStrategy.apply(SimpleIndexingStrategy.java:7)
at com.atlassian.jira.issue.index.DefaultIssueIndexer.lambda$processBatch$2(DefaultIssueIndexer.java:334)
at com.atlassian.jira.issue.index.DefaultIssueIndexer$EntityOperation.perform(DefaultIssueIndexer.java:858)
at java.base/java.util.HashMap.forEach(HashMap.java:1337)
at com.atlassian.jira.issue.index.DefaultIssueIndexer$EntityOperation.lambda$perform$6(DefaultIssueIndexer.java:858)
at com.atlassian.jira.index.DefaultIndex$Failure.<init>(DefaultIndex.java:100)
java.lang.RuntimeException: com.atlassian.jira.transaction.TransactionRuntimeException: org.ofbiz.core.entity.GenericTransactionException: Error occurred while starting transaction. (Cannot change transaction isolation level in the middle of a transaction.)
2022-12-16 02:02:03,624+0200 IssueIndexer:thread-12 WARN nikiga 84x77x1 10t7gzd 192.168.76.146 /secure/admin/IndexReIndex!reindex.jspa [c.a.jira.index.AccumulatingResultBuilder] com.atlassian.jira.transaction.TransactionRuntimeException: org.ofbiz.core.entity.GenericTransactionException: Error occurred while starting transaction. (Cannot change transaction isolation level in the middle of a transaction.)
2022-12-16 02:01:49,878+0200 IssueIndexer:thread-11 INFO nikiga 84x77x1 10t7gzd 192.168.76.146 /secure/admin/IndexReIndex!reindex.jspa [c.a.j.w.a.admin.index.IndexAdminImpl] Re-indexing is 69% complete. Current index: Issue
... 66 more
at org.ofbiz.core.entity.TransactionUtil.beginLocalTransaction(TransactionUtil.java:289)
at com.atlassian.jira.ofbiz.sql.ConnectionWrapper.setTransactionIsolation(ConnectionWrapper.java:112)
at org.ofbiz.core.entity.jdbc.interceptors.connection.DelegatingConnection.setTransactionIsolation(DelegatingConnection.java:93)
at org.apache.commons.dbcp2.DelegatingConnection.setTransactionIsolation(DelegatingConnection.java:573)
at org.apache.commons.dbcp2.DelegatingConnection.setTransactionIsolation(DelegatingConnection.java:573)
Caused by: org.postgresql.util.PSQLException: Cannot change transaction isolation level in the middle of a transaction.
... 64 more
at com.atlassian.jira.transaction.TransactionSupportImpl.beginTxn(TransactionSupportImpl.java:52)
at com.atlassian.core.ofbiz.util.CoreTransactionUtil.begin(CoreTransactionUtil.java:54)
at org.ofbiz.core.entity.TransactionUtil.beginLocalTransaction(TransactionUtil.java:296)
Caused by: org.ofbiz.core.entity.GenericTransactionException: Error occurred while starting transaction. (Cannot change transaction isolation level in the middle of a transaction.)
... 42 more
at com.atlassian.servicedesk.internal.sla.customfield.SLACFType.getRawSlaFieldValue(SLACFType.java:337)
at com.atlassian.servicedesk.internal.sla.customfield.SLACFType.getValuesFromDatabase(SLACFType.java:493)
at com.atlassian.servicedesk.internal.sla.customfield.JIRACustomFieldValueStore.getAllSLATextValues(JIRACustomFieldValueStore.java:34)
at com.atlassian.pocketknife.internal.querydsl.DatabaseAccessorImpl.runInTransaction(DatabaseAccessorImpl.java:43)
at com.atlassian.pocketknife.internal.querydsl.DatabaseAccessorImpl.execute(DatabaseAccessorImpl.java:67)
at com.atlassian.sal.core.rdbms.DefaultTransactionalExecutor.execute(DefaultTransactionalExecutor.java:40)
at com.atlassian.sal.jira.rdbms.JiraHostConnectionAccessor.execute(JiraHostConnectionAccessor.java:60)
at com.atlassian.sal.jira.rdbms.JiraHostConnectionAccessor.runInStartedOrExistingTransaction(JiraHostConnectionAccessor.java:130)
at com.sun.proxy.$Proxy347.runInManagedTransaction(Unknown Source)
at com.atlassian.plugin.osgi.bridge.external.HostComponentFactoryBean$DynamicServiceInvocationHandler.invoke(HostComponentFactoryBean.java:130)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at jdk.internal.reflect.GeneratedMethodAccessor467.invoke(Unknown Source)
at com.sun.proxy.$Proxy347.runInManagedTransaction(Unknown Source)
at com.atlassian.plugin.util.ContextClassLoaderSettingInvocationHandler.invoke(ContextClassLoaderSettingInvocationHandler.java:26)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at com.atlassian.jira.database.DatabaseAccessorImpl.runInManagedTransaction(DatabaseAccessorImpl.java:128)
at com.atlassian.jira.transaction.Txn.begin(Txn.java:17)
at com.atlassian.jira.transaction.TransactionSupportImpl.begin(TransactionSupportImpl.java:31)
at com.atlassian.jira.transaction.TransactionSupportImpl.startedTransaction(TransactionSupportImpl.java:46)
at com.atlassian.jira.transaction.TransactionSupportImpl.beginTxn(TransactionSupportImpl.java:54)
Caused by: com.atlassian.jira.transaction.TransactionRuntimeException: org.ofbiz.core.entity.GenericTransactionException: Error occurred while starting transaction. (Cannot change transaction isolation level in the middle of a transaction.)
at java.base/java.lang.Thread.run(Thread.java:834)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at com.atlassian.jira.util.concurrent.BoundedExecutor$2.call(BoundedExecutor.java:68)
at com.atlassian.jira.index.MultiThreadedIndexingStrategy$1.call(MultiThreadedIndexingStrategy.java:43)
at com.atlassian.jira.index.MultiThreadedIndexingStrategy$1.call(MultiThreadedIndexingStrategy.java:47)
at com.atlassian.jira.index.SimpleIndexingStrategy.apply(SimpleIndexingStrategy.java:5)
at com.atlassian.jira.index.SimpleIndexingStrategy.apply(SimpleIndexingStrategy.java:7)
at com.atlassian.jira.issue.index.DefaultIssueIndexer.lambda$processBatch$2(DefaultIssueIndexer.java:334)
at com.atlassian.jira.issue.index.DefaultIssueIndexer$EntityOperation.perform(DefaultIssueIndexer.java:847)
at com.atlassian.jira.issue.index.DefaultIssueIndexer$IssuesOperation.createDocuments(DefaultIssueIndexer.java:982)
at com.atlassian.jira.issue.index.DefaultIssueIndexer$DefaultDocumentCreationStrategy.get(DefaultIssueIndexer.java:1332)
at com.atlassian.jira.issue.index.DefaultIssueDocumentFactory.createDocuments(DefaultIssueDocumentFactory.java:120)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1541)
at com.atlassian.jira.issue.index.DefaultIssueDocumentFactory.lambda$createDocuments$1(DefaultIssueDocumentFactory.java:121)
at com.atlassian.jira.issue.index.DefaultIssueDocumentFactory.getDocument(DefaultIssueDocumentFactory.java:137)
at com.atlassian.jira.issue.index.DefaultIssueDocumentFactory$Builder.addAllWithPrefetchedData(DefaultIssueDocumentFactory.java:350)
at com.atlassian.jira.issue.index.DefaultIssueDocumentFactory$Builder.add(DefaultIssueDocumentFactory.java:365)
at com.atlassian.jira.issue.index.indexers.FieldIndexerWithStats.addIndex(FieldIndexerWithStats.java:57)
at com.atlassian.jira.issue.index.managers.CustomFieldIndexerAdapter.addIndex(CustomFieldIndexerAdapter.java:39)
at com.atlassian.jira.issue.index.indexers.FieldIndexer.addIndex(FieldIndexer.java:114)
at com.qoti.rfsdd.sla.index.SlaIndexerJira8.addIndex(SlaIndexerJira8.java:45)
at com.qoti.rfsdd.sla.index.SlaIndexer.indexIssue(SlaIndexer.java:365)
at java.base/java.util.Optional.ifPresent(Optional.java:183)
at com.qoti.rfsdd.sla.index.SlaIndexer.lambda$indexIssue$8(SlaIndexer.java:369)
at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:497)
at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
at java.base/java.util.HashMap$KeySpliterator.forEachRemaining(HashMap.java:1620)
at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:177)
at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
at com.qoti.rfsdd.sla.index.SlaIndexer.lambda$indexIssue$7(SlaIndexer.java:370)
at com.qoti.rfsdd.sla.index.SlaIndexer.lambda$getValueFunc$0(SlaIndexer.java:97)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at jdk.internal.reflect.GeneratedMethodAccessor1344.invoke(Unknown Source)
java.lang.reflect.InvocationTargetException
2022-12-16 02:01:48,526+0200 IssueIndexer:thread-12 ERROR nikiga 84x77x1 10t7gzd 192.168.76.146 /secure/admin/IndexReIndex!reindex.jspa [c.q.r.sla.index.SlaIndexer] Error extracting value from SLA field

Александр Никига added a comment - 16/Dec/2022 1:26 AM ... 25 more at org.ofbiz.core.entity.TransactionUtil.beginLocalTransaction(TransactionUtil.java:289) at com.atlassian.jira.ofbiz.sql.ConnectionWrapper.setTransactionIsolation(ConnectionWrapper.java:112) at org.ofbiz.core.entity.jdbc.interceptors.connection.DelegatingConnection.setTransactionIsolation(DelegatingConnection.java:93) at org.apache.commons.dbcp2.DelegatingConnection.setTransactionIsolation(DelegatingConnection.java:573) at org.apache.commons.dbcp2.DelegatingConnection.setTransactionIsolation(DelegatingConnection.java:573) Caused by: org.postgresql.util.PSQLException: Cannot change transaction isolation level in the middle of a transaction. ... 23 more at com.atlassian.jira.transaction.TransactionSupportImpl.beginTxn(TransactionSupportImpl.java:52) at com.atlassian.core.ofbiz.util.CoreTransactionUtil.begin(CoreTransactionUtil.java:54) at org.ofbiz.core.entity.TransactionUtil.beginLocalTransaction(TransactionUtil.java:296) Caused by: org.ofbiz.core.entity.GenericTransactionException: Error occurred while starting transaction. (Cannot change transaction isolation level in the middle of a transaction.) ... 10 more at com.atlassian.jira.issue.index.DefaultIssueIndexer$EntityOperation.perform(DefaultIssueIndexer.java:847) at com.atlassian.jira.issue.index.DefaultIssueIndexer$IssuesOperation.createDocuments(DefaultIssueIndexer.java:982) at com.atlassian.jira.issue.index.DefaultIssueIndexer$DefaultDocumentCreationStrategy.get(DefaultIssueIndexer.java:1320) at java.base/java.util.ArrayList.forEach(ArrayList.java:1541) at com.atlassian.jira.issue.index.DefaultIssueIndexer$DefaultDocumentCreationStrategy.lambda$get$1(DefaultIssueIndexer.java:1328) at com.atlassian.jira.issue.index.DefaultWorklogRetriever.retrieve(DefaultWorklogRetriever.java:27) at com.atlassian.jira.versioning.EntityVersioningManagerWithStats.getRelatedWorklogVersions(EntityVersioningManagerWithStats.java:240) at com.atlassian.jira.versioning.EntityVersioningManagerImpl.getRelatedWorklogVersions(EntityVersioningManagerImpl.java:146) at com.atlassian.jira.versioning.TransactionSupportHelper.executeWithRequired(TransactionSupportHelper.java:46) at com.atlassian.jira.transaction.TransactionSupport.execute(TransactionSupport.java:26) at com.atlassian.jira.transaction.TransactionSupportImpl.begin(TransactionSupportImpl.java:31) at com.atlassian.jira.transaction.TransactionSupportImpl.startedTransaction(TransactionSupportImpl.java:46) at com.atlassian.jira.transaction.TransactionSupportImpl.beginTxn(TransactionSupportImpl.java:54) Caused by: com.atlassian.jira.transaction.TransactionRuntimeException: org.ofbiz.core.entity.GenericTransactionException: Error occurred while starting transaction. (Cannot change transaction isolation level in the middle of a transaction.) at java.base/java.lang. Thread .run( Thread .java:834) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at com.atlassian.jira.util.concurrent.BoundedExecutor$2.call(BoundedExecutor.java:68) at com.atlassian.jira.index.MultiThreadedIndexingStrategy$1.call(MultiThreadedIndexingStrategy.java:43) at com.atlassian.jira.index.MultiThreadedIndexingStrategy$1.call(MultiThreadedIndexingStrategy.java:47) at com.atlassian.jira.index.SimpleIndexingStrategy.apply(SimpleIndexingStrategy.java:5) at com.atlassian.jira.index.SimpleIndexingStrategy.apply(SimpleIndexingStrategy.java:7) at com.atlassian.jira.issue.index.DefaultIssueIndexer.lambda$processBatch$2(DefaultIssueIndexer.java:334) at com.atlassian.jira.issue.index.DefaultIssueIndexer$EntityOperation.perform(DefaultIssueIndexer.java:858) at java.base/java.util.HashMap.forEach(HashMap.java:1337) at com.atlassian.jira.issue.index.DefaultIssueIndexer$EntityOperation.lambda$perform$6(DefaultIssueIndexer.java:858) at com.atlassian.jira.index.DefaultIndex$Failure.<init>(DefaultIndex.java:100) java.lang.RuntimeException: com.atlassian.jira.transaction.TransactionRuntimeException: org.ofbiz.core.entity.GenericTransactionException: Error occurred while starting transaction. (Cannot change transaction isolation level in the middle of a transaction.) 2022-12-16 02:02:03,624+0200 IssueIndexer:thread-12 WARN nikiga 84x77x1 10t7gzd 192.168.76.146 /secure/admin/IndexReIndex!reindex.jspa [c.a.jira.index.AccumulatingResultBuilder] com.atlassian.jira.transaction.TransactionRuntimeException: org.ofbiz.core.entity.GenericTransactionException: Error occurred while starting transaction. (Cannot change transaction isolation level in the middle of a transaction.) 2022-12-16 02:01:49,878+0200 IssueIndexer:thread-11 INFO nikiga 84x77x1 10t7gzd 192.168.76.146 /secure/admin/IndexReIndex!reindex.jspa [c.a.j.w.a.admin.index.IndexAdminImpl] Re-indexing is 69% complete. Current index: Issue ... 66 more at org.ofbiz.core.entity.TransactionUtil.beginLocalTransaction(TransactionUtil.java:289) at com.atlassian.jira.ofbiz.sql.ConnectionWrapper.setTransactionIsolation(ConnectionWrapper.java:112) at org.ofbiz.core.entity.jdbc.interceptors.connection.DelegatingConnection.setTransactionIsolation(DelegatingConnection.java:93) at org.apache.commons.dbcp2.DelegatingConnection.setTransactionIsolation(DelegatingConnection.java:573) at org.apache.commons.dbcp2.DelegatingConnection.setTransactionIsolation(DelegatingConnection.java:573) Caused by: org.postgresql.util.PSQLException: Cannot change transaction isolation level in the middle of a transaction. ... 64 more at com.atlassian.jira.transaction.TransactionSupportImpl.beginTxn(TransactionSupportImpl.java:52) at com.atlassian.core.ofbiz.util.CoreTransactionUtil.begin(CoreTransactionUtil.java:54) at org.ofbiz.core.entity.TransactionUtil.beginLocalTransaction(TransactionUtil.java:296) Caused by: org.ofbiz.core.entity.GenericTransactionException: Error occurred while starting transaction. (Cannot change transaction isolation level in the middle of a transaction.) ... 42 more at com.atlassian.servicedesk.internal.sla.customfield.SLACFType.getRawSlaFieldValue(SLACFType.java:337) at com.atlassian.servicedesk.internal.sla.customfield.SLACFType.getValuesFromDatabase(SLACFType.java:493) at com.atlassian.servicedesk.internal.sla.customfield.JIRACustomFieldValueStore.getAllSLATextValues(JIRACustomFieldValueStore.java:34) at com.atlassian.pocketknife.internal.querydsl.DatabaseAccessorImpl.runInTransaction(DatabaseAccessorImpl.java:43) at com.atlassian.pocketknife.internal.querydsl.DatabaseAccessorImpl.execute(DatabaseAccessorImpl.java:67) at com.atlassian.sal.core.rdbms.DefaultTransactionalExecutor.execute(DefaultTransactionalExecutor.java:40) at com.atlassian.sal.jira.rdbms.JiraHostConnectionAccessor.execute(JiraHostConnectionAccessor.java:60) at com.atlassian.sal.jira.rdbms.JiraHostConnectionAccessor.runInStartedOrExistingTransaction(JiraHostConnectionAccessor.java:130) at com.sun.proxy.$Proxy347.runInManagedTransaction(Unknown Source) at com.atlassian.plugin.osgi.bridge.external.HostComponentFactoryBean$DynamicServiceInvocationHandler.invoke(HostComponentFactoryBean.java:130) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at jdk.internal.reflect.GeneratedMethodAccessor467.invoke(Unknown Source) at com.sun.proxy.$Proxy347.runInManagedTransaction(Unknown Source) at com.atlassian.plugin.util.ContextClassLoaderSettingInvocationHandler.invoke(ContextClassLoaderSettingInvocationHandler.java:26) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at com.atlassian.jira.database.DatabaseAccessorImpl.runInManagedTransaction(DatabaseAccessorImpl.java:128) at com.atlassian.jira.transaction.Txn.begin(Txn.java:17) at com.atlassian.jira.transaction.TransactionSupportImpl.begin(TransactionSupportImpl.java:31) at com.atlassian.jira.transaction.TransactionSupportImpl.startedTransaction(TransactionSupportImpl.java:46) at com.atlassian.jira.transaction.TransactionSupportImpl.beginTxn(TransactionSupportImpl.java:54) Caused by: com.atlassian.jira.transaction.TransactionRuntimeException: org.ofbiz.core.entity.GenericTransactionException: Error occurred while starting transaction. (Cannot change transaction isolation level in the middle of a transaction.) at java.base/java.lang. Thread .run( Thread .java:834) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at com.atlassian.jira.util.concurrent.BoundedExecutor$2.call(BoundedExecutor.java:68) at com.atlassian.jira.index.MultiThreadedIndexingStrategy$1.call(MultiThreadedIndexingStrategy.java:43) at com.atlassian.jira.index.MultiThreadedIndexingStrategy$1.call(MultiThreadedIndexingStrategy.java:47) at com.atlassian.jira.index.SimpleIndexingStrategy.apply(SimpleIndexingStrategy.java:5) at com.atlassian.jira.index.SimpleIndexingStrategy.apply(SimpleIndexingStrategy.java:7) at com.atlassian.jira.issue.index.DefaultIssueIndexer.lambda$processBatch$2(DefaultIssueIndexer.java:334) at com.atlassian.jira.issue.index.DefaultIssueIndexer$EntityOperation.perform(DefaultIssueIndexer.java:847) at com.atlassian.jira.issue.index.DefaultIssueIndexer$IssuesOperation.createDocuments(DefaultIssueIndexer.java:982) at com.atlassian.jira.issue.index.DefaultIssueIndexer$DefaultDocumentCreationStrategy.get(DefaultIssueIndexer.java:1332) at com.atlassian.jira.issue.index.DefaultIssueDocumentFactory.createDocuments(DefaultIssueDocumentFactory.java:120) at java.base/java.util.ArrayList.forEach(ArrayList.java:1541) at com.atlassian.jira.issue.index.DefaultIssueDocumentFactory.lambda$createDocuments$1(DefaultIssueDocumentFactory.java:121) at com.atlassian.jira.issue.index.DefaultIssueDocumentFactory.getDocument(DefaultIssueDocumentFactory.java:137) at com.atlassian.jira.issue.index.DefaultIssueDocumentFactory$Builder.addAllWithPrefetchedData(DefaultIssueDocumentFactory.java:350) at com.atlassian.jira.issue.index.DefaultIssueDocumentFactory$Builder.add(DefaultIssueDocumentFactory.java:365) at com.atlassian.jira.issue.index.indexers.FieldIndexerWithStats.addIndex(FieldIndexerWithStats.java:57) at com.atlassian.jira.issue.index.managers.CustomFieldIndexerAdapter.addIndex(CustomFieldIndexerAdapter.java:39) at com.atlassian.jira.issue.index.indexers.FieldIndexer.addIndex(FieldIndexer.java:114) at com.qoti.rfsdd.sla.index.SlaIndexerJira8.addIndex(SlaIndexerJira8.java:45) at com.qoti.rfsdd.sla.index.SlaIndexer.indexIssue(SlaIndexer.java:365) at java.base/java.util.Optional.ifPresent(Optional.java:183) at com.qoti.rfsdd.sla.index.SlaIndexer.lambda$indexIssue$8(SlaIndexer.java:369) at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:497) at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173) at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150) at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) at java.base/java.util.HashMap$KeySpliterator.forEachRemaining(HashMap.java:1620) at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195) at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:177) at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195) at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183) at com.qoti.rfsdd.sla.index.SlaIndexer.lambda$indexIssue$7(SlaIndexer.java:370) at com.qoti.rfsdd.sla.index.SlaIndexer.lambda$getValueFunc$0(SlaIndexer.java:97) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at jdk.internal.reflect.GeneratedMethodAccessor1344.invoke(Unknown Source) java.lang.reflect.InvocationTargetException 2022-12-16 02:01:48,526+0200 IssueIndexer:thread-12 ERROR nikiga 84x77x1 10t7gzd 192.168.76.146 /secure/admin/IndexReIndex!reindex.jspa [c.q.r.sla.index.SlaIndexer] Error extracting value from SLA field

Daniel Zauner added a comment - 08/Dec/2021 1:29 PM

we increased the DB-Connection pool which helped reduce the issues, however we're not out of the woods yet; the issue still occurs, now it seems to appear only in a Jira Service Management context:

2021-12-07 07:47:55,327+0100 automation-rule-executor:thread-6 INFO jira-automation     [c.a.j.r.v2.issue.IssueBeanBuilder2] Cannot get value from RestAwareField customfield_12110, exception: 'org.ofbiz.core.entity.GenericTransactionException: Error occurred while starting transaction. (Cannot change transaction isolation level in the middle of a transaction.)'

customfield_12110 = Customer Request Type

Daniel Zauner added a comment - 08/Dec/2021 1:29 PM we increased the DB-Connection pool which helped reduce the issues, however we're not out of the woods yet; the issue still occurs, now it seems to appear only in a Jira Service Management context: 2021-12-07 07:47:55,327+0100 automation-rule-executor:thread-6 INFO jira-automation [c.a.j.r.v2.issue.IssueBeanBuilder2] Cannot get value from RestAwareField customfield_12110, exception: 'org.ofbiz.core.entity.GenericTransactionException: Error occurred while starting transaction. (Cannot change transaction isolation level in the middle of a transaction.)' customfield_12110 = Customer Request Type

Jira Data Center

Details

Summary of the problem

New behavior after the change

Description

Issue Summary

Steps to Reproduce

Expected Results

Actual Results

Workaround

Attachments

Issue Links

Forms

Activity

Collapse comment: Jacek Foremski added a comment - 05/Nov/2024 3:04 PM, Edited by Jacek Foremski - 06/Nov/2024 11:31 AM

Summary of the problem

New behavior after the change

Expand comment: Jacek Foremski added a comment - 05/Nov/2024 3:04 PM, Edited by Jacek Foremski - 06/Nov/2024 11:31 AM

Collapse comment: Alex [Atlassian,PSE] added a comment - 06/Dec/2023 3:49 AM

Expand comment: Alex [Atlassian,PSE] added a comment - 06/Dec/2023 3:49 AM

Collapse comment: Александр Никига added a comment - 16/Dec/2022 1:26 AM

Expand comment: Александр Никига added a comment - 16/Dec/2022 1:26 AM

Collapse comment: Daniel Zauner added a comment - 08/Dec/2021 1:29 PM

Expand comment: Daniel Zauner added a comment - 08/Dec/2021 1:29 PM

People

Dates