Uploaded image for project: 'Jira Data Center'
  1. Jira Data Center
  2. JRASERVER-72688

Reindexing in jira + Postgres frequently fails with : "Cannot change transaction isolation level in the middle of a transaction"

    XMLWordPrintable

Details

    Description

      Issue Summary

      While reindexing Jira(which is configured with Postgres mainly), it fails with : "Cannot change transaction isolation level in the middle of a transaction", somewhere at the end of whole process.

      Steps to Reproduce

      • Install Jira 8.13 and above with Postgres.
      • Populate set of data on this environment or restore back up from existing reproducible environment.
      • Re-index.

      Expected Results

      Reindex should succeed.

      Actual Results

      It fails with following error:

      Caused by: com.atlassian.jira.transaction.TransactionRuntimeException: org.ofbiz.core.entity.GenericTransactionException: Error occurred while starting transaction. (Cannot change transaction isolation level in the middle of a transaction.)
      	at com.atlassian.jira.transaction.TransactionSupportImpl.beginTxn(TransactionSupportImpl.java:54)
      	at com.atlassian.jira.transaction.TransactionSupportImpl.startedTransaction(TransactionSupportImpl.java:46)
      	at com.atlassian.jira.transaction.TransactionSupportImpl.begin(TransactionSupportImpl.java:31)
      [....clipped....]
      	at com.atlassian.jira.versioning.EntityVersioningManagerImpl.getRelatedCommentVersions(EntityVersioningManagerImpl.java:137)
      [....clipped....]
      Caused by: org.ofbiz.core.entity.GenericTransactionException: Error occurred while starting transaction. (Cannot change transaction isolation level in the middle of a transaction.)
      	at org.ofbiz.core.entity.TransactionUtil.beginLocalTransaction(TransactionUtil.java:296)
      	at com.atlassian.core.ofbiz.util.CoreTransactionUtil.begin(CoreTransactionUtil.java:54)
      	at com.atlassian.jira.transaction.TransactionSupportImpl.beginTxn(TransactionSupportImpl.java:52)
      	... 22 more
      Caused by: org.postgresql.util.PSQLException: Cannot change transaction isolation level in the middle of a transaction.
      	at org.apache.commons.dbcp2.DelegatingConnection.setTransactionIsolation(DelegatingConnection.java:573)
      	at org.apache.commons.dbcp2.DelegatingConnection.setTransactionIsolation(DelegatingConnection.java:573)
      	at org.ofbiz.core.entity.jdbc.interceptors.connection.DelegatingConnection.setTransactionIsolation(DelegatingConnection.java:93)
      	at com.atlassian.jira.ofbiz.sql.ConnectionWrapper.setTransactionIsolation(ConnectionWrapper.java:112)
      	at org.ofbiz.core.entity.TransactionUtil.beginLocalTransaction(TransactionUtil.java:289)
      

      Workaround

      The issues is triggered by JMX queries to MBeans with naming pattern "com.atlassian.jira:connectionpool=connections,connection=X,name=BasicDataSource", where "X" changes with each new connection creation. This behaviour is due to DBCP-585. In majority of cases, we notice that this query is initiated via "Prometheus JMX Exporter" as it queries all MBeans with default configuration. This exporter may be configured via command line, or may be shipped inside the plugin Prometheus Experter for Jira. Removing exporter and restarting the server resolves the issue.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              vpandey2@atlassian.com Vikas Pandey (Inactive)
              Votes:
              10 Vote for this issue
              Watchers:
              31 Start watching this issue

              Dates

                Created:
                Updated: