Uploaded image for project: 'Jira Server and Data Center'
  1. Jira Server and Data Center
  2. JRASERVER-61937

Atlassian HTTP client might stop working at high load




      NOTE: This bug report is for JIRA Server. Using JIRA Cloud? See the corresponding bug report.


      Atlassian HTTP Client's callbacks' ThreadPoolExecutor doesn't queue callbacks, so it might reject tasks submitted to it during high load (when no threads available right at the moment of submitting a task).
      In such cases it throws java.util.concurrent.RejectedExecutionException from the execute method in a submitting thread.
      Apache HTTP Client doesn't expect that, so it throws org.apache.http.nio.reactor.IOReactorException when catching RejectedExecutionException and changes IOReactor status to STOPPED therefore preventing further requests being sent by this client bean.
      One of the consequences is that WebHooks stop working after it happens.


      Any JIRA up to 7.2.0

      Steps to Reproduce

      1. Enable DEBUG logging for com.atlassian.httpclient
      2. Set up an HTTP endpoint (preferably having a timeout of 30+ sec before responding)
      3. Configure a WebHook against it
      4. Make a bulk edit of 1000 issues
      5. Cross the fingers
      6. Repeat bulk edits until you see java.util.concurrent.RejectedExecutionException in the logs.

      Expected Results

      • No java.util.concurrent.RejectedExecutionException thrown
      • No org.apache.http.nio.reactor.IOReactorException thrown
      • All JIRA's outgoing HTTP requests (including webhooks) continue working

      Actual Results

      • Apache HTTP client's reactor status is STOPPED
      • Webhooks (as well as some other functionality) do not work


      That might only decrease the probability of this issue, but not fix it completely:

      • Decrease the number of webhooks
      • Decrease the number of errors when calling webhooks (e.g. get rid of webhooks often throwing TimeoutExceptions, etc.)
      • Better hardware might also help slightly

      Actions that help if the issue actually happened:

      • Disabling/enabling the Atlassian HTTP Client plugin
      • Restarting JIRA.


      • Disabling/enabling the Atlassian HTTP Client Plugin will trigger other plugins to be restarted. Schedule downtime to make sure it is not affecting users that using JIRA.
      • Disabling/enabling the Atlassian HTTP Client Plugin will disable other Jira apps (Jira Software, Jira Service Desk). You need to reenable them manually from Administrations > Add-ons > Manage add-ons after that.


        Issue Links



              apawelczyk Artur Pawelczyk
              ibruzgin Ivan Bruzgin (Inactive)
              11 Vote for this issue
              29 Start watching this issue