Uploaded image for project: 'Jira Data Center'
  1. Jira Data Center
  2. JRASERVER-43143

Webhooks failing to bypass proxy when receiving URL is configured as a nonProxyHost in JVM configuration

XMLWordPrintable

    • 6.03
    • 57
    • Severity 2 - Major
    • 216
    • Hide
      Atlassian Update – 22 March 2018

      Hello everyone,

      Pleased to announce that fix has been committed and now awaits a release. We are planning to release 7.8.2 and 7.9.0 versions in few weeks.
      Fix would be available for Jira 7.6.5 too, which is current Jira Enterprise release, yet the release date is unknown as of today.

      Details of the fix

      As it was mentioned - the issue was isolated in an internal component AHTTP-36.
      Upgrading the component inside the product resolves the issue. Fix has been tested Jira versions 7.6.5, 7.7.3, 7.8.2, 7.9.0.

       

      Upgrading
      While it is possible to upgrade a component out of band - Atlassian recommends upgrading to nearest bugfix release to get the fix.

       Out-of-band component upgrades may lead to support team reject support cases until Jira code and plugins is in original versions.

       Also out-of-band deployments also may have a critical impact on instance stability and performance because of JRASERVER-64908.

      In any case - I'm attaching the mentioned version of the component that contain the fix - atlassian-httpclient-plugin-1.0.1.jar, which may be upgraded if the mentioned risks are tolerable.

       

      Configuration

      Please mind that there are different proxy configuration options available for http and https (http.proxyHost, https.proxyHost). All the details are available in https://confluence.atlassian.com/jirakb/how-to-configure-an-outbound-http-and-https-proxy-for-jira-applications-247857187.html

       

      Wish you a smooth upgrades.

      Cheers,
      Ignat Alexeyenko
      Jira Bugmaster

      Show
      Atlassian Update – 22 March 2018 Hello everyone, Pleased to announce that fix has been committed and now awaits a release. We are planning to release 7.8.2 and 7.9.0 versions in few weeks. Fix would be available for Jira 7.6.5 too, which is current Jira Enterprise release, yet the release date is unknown as of today. Details of the fix As it was mentioned - the issue was isolated in an internal component AHTTP-36 . Upgrading the component inside the product resolves the issue. Fix has been tested Jira versions 7.6.5, 7.7.3, 7.8.2, 7.9.0.   Upgrading While it is possible to upgrade a component out of band - Atlassian recommends upgrading to nearest bugfix release to get the fix.  Out-of-band component upgrades may lead to support team reject support cases until Jira code and plugins is in original versions.  Also out-of-band deployments also may have a critical impact on instance stability and performance because of JRASERVER-64908 . In any case - I'm attaching the mentioned version of the component that contain the fix - atlassian-httpclient-plugin-1.0.1.jar, which may be upgraded if the mentioned risks are tolerable.   Configuration Please mind that there are different proxy configuration options available for http and https (http.proxyHost, https.proxyHost). All the details are available in https://confluence.atlassian.com/jirakb/how-to-configure-an-outbound-http-and-https-proxy-for-jira-applications-247857187.html   Wish you a smooth upgrades. Cheers, Ignat Alexeyenko Jira Bugmaster

      Steps to reproduce the problem

      1. Install a Charles Proxy
      2. Create a HTTP proxy settings via Charles - see http-proxy.png
      3. Configure the parameters below to JVM argument via $JIRA_INSTALL/bin/setenv file: 
        -Dhttp.proxyHost=127.0.0.1 -Dhttp.proxyPort=8888 -Dhttp.nonProxyHosts=requestb.in
      4. Save it and start JIRA.
      5. Configure a webhook to http://requestb.in/ for example like this test-webhook-642.png
      6. Create a new issue to fire a webhook.
      7. The http://requestb.in URL is showing in the Charles traffic (which mean -Dhttp.nonProxyHosts is not working).

      Expected Result

      The -Dhttp.nonProxyHosts is respected and the traffic is not directed to the proxy. The Request Bin URL shouldn't be showed in the traffic if the -Dhttp.nonProxyHosts is working.

      Actual Result

      The WebHook goes through the proxy.

      From the logs :

      2016-01-19 13:25:00,973 httpclient-callbacks:thread-23 WARN anonymous     [atlassian.webhooks.plugin.PublishTaskFactoryImpl$PublishTaskImpl] Client error - 400 when posting to web hook at 'http://requestb.in/xxxxx'

      Additional Information

      • Tested on 6.4.2 and 6.4.3
      • The problem is not occurring in JIRA 6.3.15, thus this is a regression.

      Workaround

      See "Upgrading" section from Current status

        1. atlassian-httpclient-plugin-1.0.1.jar
          1.88 MB
        2. atlassian-httpclient-plugin-1.0.1.jar.md5
          0.0 kB
        3. http-proxy.png
          http-proxy.png
          27 kB
        4. test-webhook.png
          test-webhook.png
          22 kB
        5. test-webhook-642.png
          test-webhook-642.png
          29 kB

            [JRASERVER-43143] Webhooks failing to bypass proxy when receiving URL is configured as a nonProxyHost in JVM configuration

            Jean-François FORGET added a comment -

            Answer from Klaus Wening tested and fully approved on Jira 7.8 and 7.4.2 instances.

            I totally agree with Piotr, this issue is not only webhook-related, but affects many functions within a proxied network setup.

            Nevertheless we found a workaround that fits our needs (JIRA only comunicating within our network despite Atlassian Marketplace).

            It seems that UPM has another proxy-implementation (pac) to communicate with the marketplace. So we made the following changes to our JIRA proxy-setup:

            • disable all proxy-settings within catalina.properties (http.proxyHost, http.proxyPort, ...) and / or setenv.sh
            • add http.pac.proxyHost, http.pac.proxyPort (http.pac.proxyUser and http.pac.proxyPassword if needed) to catalina.properties
            • restart JIRA

            This disables all proxy-functionality for webhooks, linked instances,... but enables proxy for the marketplace.

            Thanks a lot Klaus !

            Jean-François FORGET added a comment - Answer from Klaus Wening tested and fully approved on Jira 7.8 and 7.4.2 instances. I totally agree with Piotr, this issue is not only webhook-related, but affects many functions within a proxied network setup. Nevertheless we found a workaround that fits our needs (JIRA only comunicating within our network despite Atlassian Marketplace). It seems that UPM has another proxy-implementation (pac) to communicate with the marketplace. So we made the following changes to our JIRA proxy-setup: disable all proxy-settings within catalina.properties (http.proxyHost, http.proxyPort, ...) and / or setenv.sh add  http.pac.proxyHost, http.pac.proxyPort  (http.pac.proxyUser and http.pac.proxyPassword if needed) to catalina.properties restart JIRA This disables all proxy-functionality for webhooks, linked instances,... but enables proxy for the marketplace. Thanks a lot Klaus !

            Deekshith Musthyala added a comment -

            Deekshith Musthyala added a comment -

            Ignat (Inactive) added a comment -

            Hi deekshith.musthyala1,

            Thanks for the confirmation  I'm glad the case is resolved for you!

            Cheers,
            Ignat Alexeyenko
            Jira Bugmaster

            Ignat (Inactive) added a comment - Hi deekshith.musthyala1 , Thanks for the confirmation   I'm glad the case is resolved for you! Cheers, Ignat Alexeyenko Jira Bugmaster

            Deekshith Musthyala added a comment -

            Yes fix from newer version atlassian-httpclient-plugin worked on 7.3x version of Jira too.

            Deekshith Musthyala added a comment - Yes fix from newer version atlassian-httpclient-plugin worked on 7.3x version of Jira too.

            Piotr Sojka added a comment -

            This is great. Paid application gets community support because the actual support does not bother to actually solve a basic issue .

            By the way, the workaround mentioned by Klaus on 23/Nov/2016 8:53 AM totally works with us too.

            Piotr Sojka added a comment - This is great. Paid application gets community support because the actual support does not bother to actually solve a basic issue . By the way, the workaround mentioned by Klaus on 23/Nov/2016 8:53 AM totally works with us too.

            Gino Melone added a comment -

            We've been experimenting and found that Atlassian's Public Maven had newer versions of atlassian-httpclient-plugin and swapping in the latest JAR (1.0.1) seems to fix this issue.  We're still testing, but this looks promising.

            Can someone from Atlassian comment on this approach?

            Gino Melone added a comment - We've been experimenting and found that Atlassian's Public Maven had newer versions of atlassian-httpclient-plugin and swapping in the latest JAR (1.0.1) seems to fix this issue.  We're still testing, but this looks promising. Can someone from Atlassian comment on this approach?

            Balint Zsolt added a comment - - edited

            The error is in the "atlassian-httpclient-plugin" plugin. The plugin uses the apache httpclient library and it is not initialized properly.

            In the class com.atlassian.httpclient.apache.httpcomponents.ApacheAsyncHttpClient the proxies are set explicitly (line 206).

            The problem with this is that it will use the DefaultRouter when resolving connections.

            If they don't set the proxy but call the clientBuilder.useSystemProperties() instead will result in SystemDefaultRouter that will check for the nonproxy settings as well.

            So the whole fix is just remove the set proxy code starting in line 206 (ProxyConfigFactory.getProxyHost ....)  and replace it with "clientBuilder.useSystemProperties()".

            I tested this with Jira 6.4 and it's working .. I presume that this will fix every application that uses this plugin for http clienting.

            If somebody wants to apply this fix the source can be found in the maven repo, just make sure that you use the version your application relies on.

            https://mvnrepository.com/artifact/com.atlassian.httpclient/atlassian-httpclient-plugin/0.23.2

             

            Balint Zsolt added a comment - - edited The error is in the "atlassian-httpclient-plugin" plugin. The plugin uses the apache httpclient library and it is not initialized properly. In the class com.atlassian.httpclient.apache.httpcomponents.ApacheAsyncHttpClient the proxies are set explicitly (line 206). The problem with this is that it will use the DefaultRouter when resolving connections. If they don't set the proxy but call the clientBuilder.useSystemProperties() instead will result in SystemDefaultRouter that will check for the nonproxy settings as well. So the whole fix is just remove the set proxy code starting in line 206 (ProxyConfigFactory.getProxyHost ....)  and replace it with "clientBuilder.useSystemProperties()". I tested this with Jira 6.4 and it's working .. I presume that this will fix every application that uses this plugin for http clienting. If somebody wants to apply this fix the source can be found in the maven repo, just make sure that you use the version your application relies on. https://mvnrepository.com/artifact/com.atlassian.httpclient/atlassian-httpclient-plugin/0.23.2  

            Gino Melone added a comment -

            Will the fix mentioned in https://jira.atlassian.com/browse/BSERV-10441 be applied to JIRA?

            Gino Melone added a comment - Will the fix mentioned in https://jira.atlassian.com/browse/BSERV-10441 be applied to JIRA?

            Michal Sladek added a comment -

            Hi, We get the same error with 7.5.0 Server and 7.6.2 Server. Nice if it could be fixed

            Michal Sladek added a comment - Hi, We get the same error with 7.5.0 Server and 7.6.2 Server. Nice if it could be fixed

            Tony Choplin added a comment -

            hi very interested as I wnat to hook with MS teams and have proxy so what do you mean with 

            I fix this problem after install proper connector in Microsoft Teams and now everything is working.

            ?

            Tony Choplin added a comment - hi very interested as I wnat to hook with MS teams and have proxy so what do you mean with  I fix this problem after install proper connector in Microsoft Teams and now everything is working. ?

              ialexeyenko Ignat (Inactive)
              ckimloong John Chin
              Affected customers:
              131 This affects my team
              Watchers:
              125 Start watching this issue

                Created:
                Updated:
                Resolved: