Uploaded image for project: 'Jira Data Center'
  1. Jira Data Center
  2. JRASERVER-78739

Transitioning an issue during database contention produces a stale webhook in Jira Data Center

XMLWordPrintable

      Issue Summary

      In all Jira 9 versions using asynchronous webhooks and affected Jira 10 versions, it's possible for a webhook to get fired with the previous modification's change history and custom field values. This occurs due to a race condition between the workflow transition's database transaction and the webhook dispatcher's payload generation.

      The payload generation happens outside of the database transaction. Since the transaction uses the "read committed" isolation level, dirty reads are disallowed, and the webhook dispatcher reads stale data. This problem is more likely to occur if the database is slow to process and commit the changes produced by the workflow transition.

      Steps to Reproduce

      Due to a race condition, this bug isn't consistently reproducible without specialized tooling.

      1. Configure a webhook to listen to issue events.
      2. Make Modification A to an issue within the webhook's scope.
        • Example: lower priority from "High" to "Low".
      3. Introduce a database performance bottleneck, specifically on transaction COMMIT.
      4. Transition the issue and update a custom field from the transition screen (Modification B).
        • Example: transition "To Do"->"In Progress" and increase Story Points (1->2).

      Expected Results

      Two unique webhooks are produced – one containing Modification A and the other containing Modification B. All custom field values are consistent.

      Actual Results

      Two webhooks with Modification A's change history are produced. The custom field value is stale in the latter.

      After Step 2

      {
        "timestamp": 1744324207702,
        "webhookEvent": "jira:issue_updated",
        "issue_event_type_name": "issue_generic",
        "issue": {
            "key": "APP-5",
            "fields": {
                "priority": {
                    "name": "Low"
                },
                "updated": "2025-04-10T15:30:07.696-0700",
                "status": {
                    "name": "To Do"
                },
                "customfield_10107": 1,
            }
        },
        "changelog": {
            "id": "10158",
            "author": null,
            "created": null,
            "items": [
                {
                    "field": "priority",
                    "fieldtype": "jira",
                    "from": "2",
                    "fromString": "High",
                    "to": "4",
                    "toString": "Low"
                }
            ]
        }
      }
      

      After Step 4

      {
        "timestamp": 1744324227414,
        "webhookEvent": "jira:issue_updated",
        "issue_event_type_name": "issue_generic",
        "issue": {
            "key": "APP-5",
            "fields": {
                "priority": {
                    "name": "Low"
                },
                "updated": "2025-04-10T15:30:22.813-0700",
                "status": {
                    "name": "In Progress"
                },
                "customfield_10107": 1,
            }
        },
        "changelog": {
            "id": "10158",
            "author": null,
            "created": null,
            "items": [
                {
                    "field": "priority",
                    "fieldtype": "jira",
                    "from": "2",
                    "fromString": "High",
                    "to": "4",
                    "toString": "Low"
                }
            ]
        }
      }
      

      Observe that the same change ID (10158) appears in both payloads. As a result, the change history for Modification B is never delivered. Furthermore, the value of customfield_10107 (Story Points) is stale in the latter webhook's issue body.

      Workaround

      1. Improve database transaction commit time performance.
      2. Switch to synchronous webhooks (Jira 9 only).

      Supplemental logging

      In Jira 10.3.6 / 10.6.1 and later, logging was added to capture this edge case:

      2025-04-15 14:30:00,000+0000 webhook-dispatcher:thread-1 ERROR admin     [c.a.j.p.w.serializer.issue.IssueEventSerializer] For issue with ID: 5375659 we were unable to retrieve the change history group with ID: 230178 from database. The webhook will be sent with the most recent change history item instead.
      

      Downstream integrations can reprocess the change identified using the /rest/api/2/issue/{issueId}?expand=changelog REST API endpoint.

              c05631dc8c5f Tomasz Ziółkowski
              1353e2e9fd2f Benjamin S
              Votes:
              17 Vote for this issue
              Watchers:
              30 Start watching this issue

                Created:
                Updated:
                Resolved: