Uploaded image for project: 'Jira Service Management Data Center'
  1. Jira Service Management Data Center
  2. JSDSERVER-6979

JSD automation webhooks are not being sent due to no socket timeout defined

    XMLWordPrintable

Details

    • Bug
    • Resolution: Unresolved
    • Low
    • 4.5.8, 4.12.1
    • 4.5.3, 4.12.0, 4.5.7, 4.11.1
    • Automation

    Description

      Issue Summary

      JSD automation webhooks appear to be executed with no defined socket timeouts (note, this is a different implementation from the core Jira). Consequently, a poorly behaving receiver can prevent JSD automation webhooks from being sent, as PsmqAsyncExecutors-then threads become stuck in the following stack:

      java.lang.Thread.State: RUNNABLE
        at java.net.SocketInputStream.socketRead0(Native Method)
        at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
        at java.net.SocketInputStream.read(SocketInputStream.java:171)
        at java.net.SocketInputStream.read(SocketInputStream.java:141)
        at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137)
        at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:153)
        at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:282)
        at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138)
        at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
        at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
        at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
        at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165)
        at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
        at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
        at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
        at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
        at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
        at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111)
        at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:108)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
        at com.atlassian.servicedesk.plugins.automation.webhook.executor.WebhookExecutorImpl.executeWebhook(WebhookExecutorImpl.java:91)
      
        • Note the flow: WebhookExecutorImpl.executeWebhook -> CloseableHttpClient.execute -> SessionInputBufferImpl.streamRead -> SocketInputStream.socketRead0
      • Other PsmqAsyncExecutors-then threads are stuck in waiting for a HTTP connection from the pool, since they all use the same pool:
        java.lang.Thread.State: WAITING (parking)   java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x000000000496db71> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) 
          at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) 
          at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) 
          at org.apache.http.pool.AbstractConnPool.getPoolEntryBlocking(AbstractConnPool.java:380)
        

        Steps to Reproduce

        Challenging to reproduce, but you can mimic a misbehaving receiver by configuring a tarpit.

      Expected Results

      The connection should timeout after being stuck in SocketRead() for some time.

      Actual Results

      The connection never seems to be closed when a receiver is misbehaving and not terminating the connection from its end.

      Workaround

      • Restart Jira to unblock the PsmqAsyncExecutors-then thread
      • Currently there is no known workaround without downtime for this behavior.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              troyall TJ Royall
              Votes:
              1 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

                Created:
                Updated:

                Backbone Issue Sync