Uploaded image for project: 'Bitbucket Data Center'
  1. Bitbucket Data Center
  2. BSERV-7741

Secure LDAP connections are broken when using Java 1.8u51+, 1.7.0_85+ and 1.6.0_101+

    • Icon: Bug Bug
    • Resolution: Won't Fix
    • Icon: Medium Medium
    • None
    • 3.11.2
    • Crowd
    • Oracle Java 1.8u51
      Oracle Java 1.7u85

      Summary

      As of Java 1.8u51+ (also 1.7.0_85+ and 1.6.0_101+) making SSL connections to an IP address is no longer allowed. Previously the JVM would do a reverse lookup of the hostname using the IP then complete the connection using that hostname. This was a security vulnerability because an attacker that gained control of DNS
      could route traffic to their own server. This change is a security feature of Java.

      We have found there is a problem with the way Java creates LDAPS connections. The way Socket.createSocket() is used results in an SSLSocket without the hostname set in it. This results in use of an IP address to make the connection and since this is no longer allowed the connection fails. We have reported this problem to Oracle and also forwarded the information to the Java security dev list: http://mail.openjdk.java.net/pipermail/security-dev/2015-September/012845.html .

      Because of this existing LDAPS connections are broken and users from that user directory are unable to login including non-local administrators.

      This will affect any customer using secure LDAP that upgrades the JVM. Secure LDAP is very common to protect passwords on the internal network, particularly in enterprise environments.

      Environment

      • Any version of Stash using one of the noted Java versions.

      Steps to Reproduce

      1. Configure a user directory using a secure LDAP connection
      2. Upgrade the JDK to a noted Java version.

      Expected Results

      Logins and LDAP syncs should work as normal.

      Actual Results

      Users will be unable to authenticate and LDAP syncs will fail.

      Errors like this will occur in the logs:

      2015-07-22 12:43:05,896 WARN  [http-bio-443-exec-1] @1SDVDDx763x1x0 XXXXX "POST /j_stash_security_check HTTP/1.1" c.a.s.i.s.s.PluginAuthenticationProvider Could not authenticate XXXXX; authentication by com.atlassian.stash.stash-authentication:crowdHttpAuthHandler failed
      com.atlassian.stash.user.AuthenticationSystemException: The remote authentication server is not available. Please try again later.
      	at com.atlassian.stash.internal.crowd.RiotPolice.authenticate(RiotPolice.java:113) ~[stash-service-impl-3.11.1.jar:na]
      	at com.atlassian.stash.internal.user.DefaultUserService.authenticate(DefaultUserService.java:108) ~[stash-service-impl-3.11.1.jar:na]
      ...
      	... 21 common frames omitted
      Caused by: org.springframework.transaction.CannotCreateTransactionException: Could not create DirContext instance for transaction; nested exception is org.springframework.ldap.CommunicationException: <DNS.NAME>:636; nested exception is javax.naming.CommunicationException: <DNS.NAME>:636 [Root exception is javax.net.ssl.SSLHandshakeException: java.security.cert.CertificateException: No subject alternative names matching IP address XXXXX found]
      	at org.springframework.transaction.compensating.support.AbstractCompensatingTransactionManagerDelegate.doBegin(AbstractCompensatingTransactionManagerDelegate.java:90) ~[spring-ldap-core-2.0.2.RELEASE.jar:2.0.2.RELEASE]
      ...
      	... 22 common frames omitted
      Caused by: org.springframework.ldap.CommunicationException: <DNS.NAME>:636; nested exception is javax.naming.CommunicationException: <DNS.NAME>:636 [Root exception is javax.net.ssl.SSLHandshakeException: java.security.cert.CertificateException: No subject alternative names matching IP address XXXXX found]
      	at org.springframework.ldap.support.LdapUtils.convertLdapException(LdapUtils.java:108)
      ...
      	... 39 common frames omitted
      Caused by: javax.naming.CommunicationException: <DNS.NAME>:636
      	at com.sun.jndi.ldap.Connection.<init>(Unknown Source) ~[na:1.8.0_51]
      	at com.sun.jndi.ldap.LdapClient.<init>(Unknown Source) ~[na:1.8.0_51]
      ...
      	... 43 common frames omitted
      Caused by: javax.net.ssl.SSLHandshakeException: java.security.cert.CertificateException: No subject alternative names matching IP address XXXXX found
      	at sun.security.ssl.Alerts.getSSLException(Unknown Source) ~[na:1.8.0_51]
      	at sun.security.ssl.SSLSocketImpl.fatal(Unknown Source) ~[na:1.8.0_51]
      ...
      	... 58 common frames omitted
      Caused by: java.security.cert.CertificateException: No subject alternative names matching IP address XXXXX found
      	at sun.security.util.HostnameChecker.matchIP(Unknown Source) ~[na:1.8.0_51]
      ...
      	... 67 common frames omitted
      

      Workaround

      Option 1: Add JVM startup parameter -Djdk.tls.trustNameService=true

      This reverts back to the older Java behavior while preserving all the other security related changes in the newer Java release.

      Your LDAP data will still be encrypted. The risk is described by Oracle in their explanation of the change at http://www.oracle.com/technetwork/java/javase/8u51-relnotes-2587590.html :

      If an application does need to perform reverse name lookup for raw IP addresses in SSL/TLS connections, and encounter endpoint identification compatibility issue, System property "jdk.tls.trustNameService" can be used to switch on reverse name lookup. Note that if the name service is not trustworthy, enabling reverse name lookup may be susceptible to MITM attacks.

      Option 2: Under the LDAP connection "Advanced Settings" set "Secure SSL" to disabled

      As stated in the UI this will disable all verification of the SSL certificate. Your LDAP data will still be encrypted.

      This workaround may represent a greater vulnerability as any SSL cert could be substituted with no need to compromise a DNS server.

          Form Name

            [BSERV-7741] Secure LDAP connections are broken when using Java 1.8u51+, 1.7.0_85+ and 1.6.0_101+

            danno added a comment -

            Same problem 6 weeks later. Jira 7.0.5 shipping with oracle JRE 51. Thanks for wasting two hours of my life.

            danno added a comment - Same problem 6 weeks later. Jira 7.0.5 shipping with oracle JRE 51. Thanks for wasting two hours of my life.

            I couldn't agree more with Mike, we had the same issue with Bitbucket and Jira, installed with the latest version. How can you justify that your customers must work around this kind of issue?

            Jérémie Grauer added a comment - I couldn't agree more with Mike, we had the same issue with Bitbucket and Jira, installed with the latest version. How can you justify that your customers must work around this kind of issue?

            Mike Friedrich added a comment - - edited

            That is not my point, i undertand above workaround. But as long as atlassian products com with JRE, then its Atlassian's reponsibility to intergrate the patchers into their product. Even if that just means repackaging with a newer JRE.

            We just installed Confluence 5.8.15 a few days ago, seems to be newest version right now and still comes with a buggy JRE!
            When installing a new system why do i have to deal with these type of issues at all? This is close since October, almost 2 months where Atlasssian could have fixed the issue - and a very simple fix too.

            If you ask me, this ticket should be reopened.

            Mike Friedrich added a comment - - edited That is not my point, i undertand above workaround. But as long as atlassian products com with JRE, then its Atlassian's reponsibility to intergrate the patchers into their product. Even if that just means repackaging with a newer JRE. We just installed Confluence 5.8.15 a few days ago, seems to be newest version right now and still comes with a buggy JRE! When installing a new system why do i have to deal with these type of issues at all? This is close since October, almost 2 months where Atlasssian could have fixed the issue - and a very simple fix too. If you ask me, this ticket should be reopened.

            Hey Mike,
            I have found that if I download Oracle JDK 1.8.0_65 that is solves the problem. Once I upgrade and add the cert that it then will connect. SSLPoke is a good tool to test without taking your products up and down.

            Chris Whitten [Comskil] added a comment - Hey Mike, I have found that if I download Oracle JDK 1.8.0_65 that is solves the problem. Once I upgrade and add the cert that it then will connect. SSLPoke is a good tool to test without taking your products up and down.

            Mike Friedrich added a comment - - edited

            This issue seems to effect all Atlassian products in which configure LDAPS. Whats the solution? Is Atlassian repackaging all products to include patched JRE's? I have this problem with confluence LDAP setup.

            Mike Friedrich added a comment - - edited This issue seems to effect all Atlassian products in which configure LDAPS. Whats the solution? Is Atlassian repackaging all products to include patched JRE's? I have this problem with confluence LDAP setup.

            Daniel R added a comment - - edited

            Hi Chris,

            The link above had an erroneous colon ( : ) – the correct link takes you to the same bug that is linked below the description above:

            https://jira.atlassian.com/browse/CWD-4444

            This Java update is in regard to Stash/Bitbucket Server and doesn't have anything to do with the Crowd standalone application.

            Daniel R added a comment - - edited Hi Chris, The link above had an erroneous colon ( : ) – the correct link takes you to the same bug that is linked below the description above: https://jira.atlassian.com/browse/CWD-4444 This Java update is in regard to Stash/Bitbucket Server and doesn't have anything to do with the Crowd standalone application.

            Dear Kristy,
            First of all I can't access that ticket to see Oliver's comment, it says the ticket doesn't exist (am I doing something wrong?). Second, Atlassian documents describe to use Oracle JDK over Open JDK and for the latest crowd it only supports 1.7 according to the supported platform docs (https://confluence.atlassian.com/display/CROWD/Supported+Platforms). Is this still the consensus for crowd by Atlassian or are you now condoning OpenJDK 1.8? THanks

            Chris Whitten [Comskil] added a comment - Dear Kristy, First of all I can't access that ticket to see Oliver's comment, it says the ticket doesn't exist (am I doing something wrong?). Second, Atlassian documents describe to use Oracle JDK over Open JDK and for the latest crowd it only supports 1.7 according to the supported platform docs ( https://confluence.atlassian.com/display/CROWD/Supported+Platforms ). Is this still the consensus for crowd by Atlassian or are you now condoning OpenJDK 1.8? THanks

            Kristy added a comment - - edited

            Please see osenn's comment on https://jira.atlassian.com/browse/CWD-4444 :

            The release of Java JDK version 8u65 contains a fix (https://bugs.openjdk.java.net/browse/JDK-8135194) which avoids the kind of problem described in this issue. Since a JDK is available which fixes the underlying issue directly, we will not implement a custom fix in Crowd itself. Therefore, we recommend to update to to a version of the JDK equal or newer to 8u65.

            Kristy added a comment - - edited Please see osenn 's comment on https://jira.atlassian.com/browse/CWD-4444 : The release of Java JDK version 8u65 contains a fix ( https://bugs.openjdk.java.net/browse/JDK-8135194 ) which avoids the kind of problem described in this issue. Since a JDK is available which fixes the underlying issue directly, we will not implement a custom fix in Crowd itself. Therefore, we recommend to update to to a version of the JDK equal or newer to 8u65.

            UPDATE: We have found there is a problem with the way Java creates LDAPS connections. The way Socket.createSocket() is used results in an SSLSocket without the hostname set in it. This results in use of an IP address to make the connection and since this is no longer allowed for security reasons the connection fails. We have reported this problem to Oracle and also forwarded the information to the Java security dev list: http://mail.openjdk.java.net/pipermail/security-dev/2015-September/012845.html .

            Morgan Knicely (Inactive) added a comment - UPDATE: We have found there is a problem with the way Java creates LDAPS connections. The way Socket.createSocket() is used results in an SSLSocket without the hostname set in it. This results in use of an IP address to make the connection and since this is no longer allowed for security reasons the connection fails. We have reported this problem to Oracle and also forwarded the information to the Java security dev list: http://mail.openjdk.java.net/pipermail/security-dev/2015-September/012845.html .

              khughes@atlassian.com Kristy
              chparker ♚♛♜♝♞♟♔♕♖♗♘♙ (Inactive)
              Affected customers:
              8 This affects my team
              Watchers:
              23 Start watching this issue

                Created:
                Updated:
                Resolved: