Uploaded image for project: 'Jira Platform Cloud'
  1. Jira Platform Cloud
  2. JRACLOUD-78542

When fetching an issue in response to an issue create webhook the Issue may not be found

      Issue Summary

      Issue create webhooks may be sent prematurely in some circumstances, leading to the issue not being fully created by the time an integrator calls back to Jira for the issue details.

      Steps to Reproduce

      1. Register a Webhook to receive "jira:issue_created" events
      2. Create an Issue in Jira
      3. Immediately after receiving the webhook in your application, using the issueId contained within the webhook payload, attempt to fetch the issue via the REST api  (eg GET /rest/api/3/issue/{issueIdOrKey} )

      Expected Results

      The issue can be successfully retrieved, eg a 200 response from GET /rest/api/3/issue/{issueIdOrKey}

      Actual Results

      A 404 response may be returned, indicating the issue does not exist. 

      Workaround

      Introduce a delay or a retry in response to 404's when fetching a recently created issue. 

          Form Name

            [JRACLOUD-78542] When fetching an issue in response to an issue create webhook the Issue may not be found

            Hi e555ca5e1233,
            I guess the problem is indeed resolved. Thanks!

            David Fischer added a comment - Hi e555ca5e1233 , I guess the problem is indeed resolved. Thanks!

            Hi 94298a523126, I've checked what happened with 24971d3a15fd7888 and in this case, the issue was deleted by Jira Automation. We received a DELETE request on 2022-09-23T12:02:30,451Z, so it's expected the app got 404 for the following requests:

            • 8d39bfa17e4d3934    
            • 0f52d689c7d3e65c
            • 6e11d1c4ecac5b1a
            • 1a9e379d504c24b4

            Since the requests you recently mentioned are in fact not related to the initial reported issue, we decided to close this ticket.

            Bartosz Robakowski added a comment - Hi 94298a523126 , I've checked what happened with  24971d3a15fd7888  and in this case, the issue was deleted by Jira Automation. We received a  DELETE  request on  2022-09-23T12:02:30,451Z , so it's expected the app got  404  for the following requests: 8d39bfa17e4d3934     0f52d689c7d3e65c 6e11d1c4ecac5b1a 1a9e379d504c24b4 Since the requests you recently mentioned are in fact not related to the initial reported issue, we decided to close this ticket.

            David Fischer added a comment - - edited

            Hi e555ca5e1233b71be3adc7ae,

            oops, sorry about that. Unfortunately, we can't easily filter out requests like you asked. But I found this other error for which we do not have any trace of the issue being deleted (but of course if the issue was deleted by another app or by Jira Automation, we wouldn't know):

            "atl-traceid":"6e11d1c4ecac5b1a","x-arequestid":"0a47784c-00ed-4da1-b053-6f6896c99e2b"

            What's interesting about that particular error is that it happens after a few successful attempts to retrieve the same issue through the exact same REST API call (we don't cache them because they're run on different VMs). Maybe you can see in the Jira logs something we can't see?

            Anyway, wouldn't it be easier to discuss this in a private space where I can share more information? You can ask Andrew Califano (our TPM) to connect us.

            David Fischer added a comment - - edited Hi e555ca5e1233 ,  b71be3adc7ae , oops, sorry about that. Unfortunately, we can't easily filter out requests like you asked. But I found this other error for which we do not have any trace of the issue being deleted (but of course if the issue was deleted by another app or by Jira Automation, we wouldn't know): "atl-traceid" : "6e11d1c4ecac5b1a" , "x-arequestid" : "0a47784c-00ed-4da1-b053-6f6896c99e2b" What's interesting about that particular error is that it happens after a few successful attempts to retrieve the same issue through the exact same REST API call (we don't cache them because they're run on different VMs). Maybe you can see in the Jira logs something we can't see? Anyway, wouldn't it be easier to discuss this in a private space where I can share more information? You can ask Andrew Califano (our TPM) to connect us.

            Bartosz Robakowski added a comment - - edited

            Hi 94298a523126, I've searched logs for the mentioned error, and in this case, 404 was an expected response. That's because your app deleted the requested issue beforehand. We started processing DELETE request 616144ed740c6c0d on 2022-09-22T22:41:06.43553696Z, which happened before GET request we received on 2022-09-22T22:41:09.440138112Z.

            Actually, I see a total of 7 requests ending in 404 for this particular issue:

            • 5df81a48a0a9bdc9
            • 4909a24c8a96f0d7
            • 2f2c54d6dd90ba16
            • 18596621f3e598f3
            • 616144ed740c6c0d
            • 3d7edc3b48688239
            • a7c7823bca76a746

            which might add up to the reported number of similar cases (700 per day). Is each of them a result of the issue_created webhook? Can you by any chance filter requests ending by 404 that were preceded by DELETE and give us this number?

            Bartosz Robakowski added a comment - - edited Hi 94298a523126 , I've searched logs for the mentioned error, and in this case, 404 was an expected response. That's because your app deleted the requested issue beforehand. We started processing DELETE request 616144ed740c6c0d on 2022-09-22T22:41:06.43553696Z , which happened before GET request we received on 2022-09-22T22:41:09.440138112Z . Actually, I see a total of 7 requests ending in 404 for this particular issue: 5df81a48a0a9bdc9 4909a24c8a96f0d7 2f2c54d6dd90ba16 18596621f3e598f3 616144ed740c6c0d 3d7edc3b48688239 a7c7823bca76a746 which might add up to the reported number of similar cases (700 per day). Is each of them a result of the issue_created webhook? Can you by any chance filter requests ending by 404 that were preceded by DELETE and give us this number?

            Hi b71be3adc7ae,

            I do see a lower number of these errors since roughly Sept 20, but they're not gone entirely. We still get around 700 errors per day (compared to 4000/day last week).

            I can share this from such an error today, if it helps:

            "atl-traceid":"616144ed740c6c0d","x-arequestid":"c7f96c3d-0497-4da6-b05f-6e2d8a5e156e" 

            You can reach out to me directly if you need more details.

            Cheers,

            David

            David Fischer added a comment - Hi  b71be3adc7ae , I do see a lower number of these errors since roughly Sept 20, but they're not gone entirely. We still get around 700 errors per day (compared to 4000/day last week). I can share this from such an error today, if it helps: "atl-traceid" : "616144ed740c6c0d" , "x-arequestid" : "c7f96c3d-0497-4da6-b05f-6e2d8a5e156e" You can reach out to me directly if you need more details. Cheers, David

            Hi 94298a523126,

            I started rolling out a potential fix for this issue a few days ago.
            It's currently rolled out on the whole production, so the issue shouldn't occur anymore. If you could let me know if you can notice any improvement when it comes to fetching recently created issues that would be great. 

            Cheers, Maciej

            Maciej Stanuch added a comment - Hi 94298a523126 , I started rolling out a potential fix for this issue a few days ago. It's currently rolled out on the whole production, so the issue shouldn't occur anymore. If you could let me know if you can notice any improvement when it comes to fetching recently created issues that would be great.  Cheers, Maciej

            Hi Bartosz,

            I checked our stats over the last 2 weeks and didn't notice a major change in the number of 404 errors while fetching the issue during a Create transition. In fact, last Friday (May 27) saw a huge peak of such errors coming from 3 different tenants.

            David Fischer added a comment - Hi Bartosz, I checked our stats over the last 2 weeks and didn't notice a major change in the number of 404 errors while fetching the issue during a Create transition. In fact, last Friday (May 27) saw a huge peak of such errors coming from 3 different tenants.

            Then I'll check back in a week

            David Fischer added a comment - Then I'll check back in a week

            Hi 94298a523126,

            the change was rolled out on Friday, May 20.

            Bartosz Robakowski added a comment - Hi 94298a523126 , the change was rolled out on Friday, May 20.

            David Fischer added a comment - - edited

            Hi Bartosz,

            when did you roll out the change?

            It's a little hard for us to evaluate these errors, but here's a chart of the number of 404 errors fetching the issue during a Create transition, from May 11 to May 23:

            (ok, can't add attachments so can't share here)

            David Fischer added a comment - - edited Hi Bartosz, when did you roll out the change? It's a little hard for us to evaluate these errors, but here's a chart of the number of 404 errors fetching the issue during a Create transition, from May 11 to May 23: (ok, can't add attachments so can't share here)

              Unassigned Unassigned
              gtaylor@atlassian.com Gene Taylor
              Affected customers:
              5 This affects my team
              Watchers:
              14 Start watching this issue

                Created:
                Updated:
                Resolved: