Uploaded image for project: 'Crowd Data Center'
  1. Crowd Data Center
  2. CWD-4444

Secure LDAP connections are broken when using Java 1.8u51, 1.8u60, 1.7.0_85+ and 1.6.0_101+

    • Icon: Bug Bug
    • Resolution: Won't Fix
    • Icon: Medium Medium
    • None
    • 2.7, 2.8, 2.8.3
    • None
    • Oracle Java 1.8u51
      Oracle Java 1.7u85

      Summary

      As of Java 1.8u51+ (also 1.7.0_85+ and 1.6.0_101+) making SSL connections to an IP address is no longer allowed. Previously the JVM would do a reverse lookup of the hostname using the IP then complete the connection using that hostname. This was a security vulnerability because an attacker that gained control of DNS
      could route traffic to their own server. This change is a security feature of Java.

      We have found there is a problem with the way Java creates LDAPS connections. The way Socket.createSocket() is used results in an SSLSocket without the hostname set in it. This results in use of an IP address to make the connection and since this is no longer allowed the connection fails. We have reported this problem to Oracle and also forwarded the information to the Java security dev list: http://mail.openjdk.java.net/pipermail/security-dev/2015-September/012845.html .

      Because of this existing LDAPS connections are broken and users from that user directory are unable to login including non-local administrators.

      This will affect any customer using secure LDAP that upgrades the JVM. Secure LDAP is very common to protect passwords on the internal network, particularly in enterprise environments.

      Environment

      • Any version of Crowd using one of the noted Java versions.

      Steps to Reproduce

      1. Configure a user directory using a secure LDAP connection
      2. Upgrade the JDK to a noted Java version.

      Expected Results

      Logins and LDAP syncs should work as normal.

      Actual Results

      Users will be unable to authenticate and LDAP syncs will fail.

      Errors like this will occur in the logs:

      2015-07-22 12:43:05,896 WARN  [http-bio-443-exec-1] @1SDVDDx763x1x0 XXXXX "POST /j_stash_security_check HTTP/1.1" c.a.s.i.s.s.PluginAuthenticationProvider Could not authenticate XXXXX; authentication by com.atlassian.stash.stash-authentication:crowdHttpAuthHandler failed
      com.atlassian.stash.user.AuthenticationSystemException: The remote authentication server is not available. Please try again later.
      	at com.atlassian.stash.internal.crowd.RiotPolice.authenticate(RiotPolice.java:113) ~[stash-service-impl-3.11.1.jar:na]
      	at com.atlassian.stash.internal.user.DefaultUserService.authenticate(DefaultUserService.java:108) ~[stash-service-impl-3.11.1.jar:na]
      ...
      	... 21 common frames omitted
      Caused by: org.springframework.transaction.CannotCreateTransactionException: Could not create DirContext instance for transaction; nested exception is org.springframework.ldap.CommunicationException: <DNS.NAME>:636; nested exception is javax.naming.CommunicationException: <DNS.NAME>:636 [Root exception is javax.net.ssl.SSLHandshakeException: java.security.cert.CertificateException: No subject alternative names matching IP address XXXXX found]
      	at org.springframework.transaction.compensating.support.AbstractCompensatingTransactionManagerDelegate.doBegin(AbstractCompensatingTransactionManagerDelegate.java:90) ~[spring-ldap-core-2.0.2.RELEASE.jar:2.0.2.RELEASE]
      ...
      	... 22 common frames omitted
      Caused by: org.springframework.ldap.CommunicationException: <DNS.NAME>:636; nested exception is javax.naming.CommunicationException: <DNS.NAME>:636 [Root exception is javax.net.ssl.SSLHandshakeException: java.security.cert.CertificateException: No subject alternative names matching IP address XXXXX found]
      	at org.springframework.ldap.support.LdapUtils.convertLdapException(LdapUtils.java:108)
      ...
      	... 39 common frames omitted
      Caused by: javax.naming.CommunicationException: <DNS.NAME>:636
      	at com.sun.jndi.ldap.Connection.<init>(Unknown Source) ~[na:1.8.0_51]
      	at com.sun.jndi.ldap.LdapClient.<init>(Unknown Source) ~[na:1.8.0_51]
      ...
      	... 43 common frames omitted
      Caused by: javax.net.ssl.SSLHandshakeException: java.security.cert.CertificateException: No subject alternative names matching IP address XXXXX found
      	at sun.security.ssl.Alerts.getSSLException(Unknown Source) ~[na:1.8.0_51]
      	at sun.security.ssl.SSLSocketImpl.fatal(Unknown Source) ~[na:1.8.0_51]
      ...
      	... 58 common frames omitted
      Caused by: java.security.cert.CertificateException: No subject alternative names matching IP address XXXXX found
      	at sun.security.util.HostnameChecker.matchIP(Unknown Source) ~[na:1.8.0_51]
      ...
      	... 67 common frames omitted
      

      Fix: install a Java version equal to or newer than 1.8u65

      Java version 1.8u65 contains a fix JDK-8133196 which addresses the problem described in this issue. Please see How to Use System JRE Instead of Embedded JRE for instructions to change the version of Java that JIRA runs with.

      Workaround

      Only necessary if you do not upgrade your Java JDK.

      Add JVM startup parameter -Djdk.tls.trustNameService=true

      This reverts back to the older Java behavior while preserving all the other security related changes in the newer Java release. Your LDAP data will still be encrypted. The risk is described by Oracle in their explanation of the change at http://www.oracle.com/technetwork/java/javase/8u51-relnotes-2587590.html.

      Disable SSL check

      Under the LDAP connection "Advanced Settings" set "Secure SSL" to disabled*. As stated in the UI this will disable all verification of the SSL certificate. Your LDAP data will still be encrypted. This workaround may represent a greater vulnerability as any SSL cert could be substituted with no need to compromise a DNS server.

            [CWD-4444] Secure LDAP connections are broken when using Java 1.8u51, 1.8u60, 1.7.0_85+ and 1.6.0_101+

            Halo Haerul Fuad

            Pimpinan Ketua added a comment - Halo Haerul Fuad

            44444

            Pimpinan Ketua added a comment - 44444

            The release of Java JDK version 8u65 contains a fix (https://bugs.openjdk.java.net/browse/JDK-8135194) which avoids the kind of problem described in this issue. Since a JDK is available which fixes the underlying issue directly, we will not implement a custom fix in Crowd itself. Therefore, we recommend to update to to a version of the JDK equal or newer to 8u65.

            Oliver Senn added a comment - The release of Java JDK version 8u65 contains a fix ( https://bugs.openjdk.java.net/browse/JDK-8135194 ) which avoids the kind of problem described in this issue. Since a JDK is available which fixes the underlying issue directly, we will not implement a custom fix in Crowd itself. Therefore, we recommend to update to to a version of the JDK equal or newer to 8u65.

            For anyone interested in the nature of the issue, the tests in https://bitbucket.org/atlassian/cwd-4444-java-bug-reproducer demonstrate the issue with the JDK SSL implementation we found. There is a fix listed at https://bugs.openjdk.java.net/browse/JDK-8133196 that has not been included in a released JDK version yet.

            Avi Knoll (Inactive) added a comment - For anyone interested in the nature of the issue, the tests in https://bitbucket.org/atlassian/cwd-4444-java-bug-reproducer demonstrate the issue with the JDK SSL implementation we found. There is a fix listed at https://bugs.openjdk.java.net/browse/JDK-8133196 that has not been included in a released JDK version yet.

            UPDATE: We have found there is a problem with the way Java creates LDAPS connections. The way Socket.createSocket() is used results in an SSLSocket without the hostname set in it. This results in use of an IP address to make the connection and since this is no longer allowed for security reasons the connection fails. We have reported this problem to Oracle and also forwarded the information to the Java security dev list: http://mail.openjdk.java.net/pipermail/security-dev/2015-September/012845.html .

            Morgan Knicely (Inactive) added a comment - UPDATE: We have found there is a problem with the way Java creates LDAPS connections. The way Socket.createSocket() is used results in an SSLSocket without the hostname set in it. This results in use of an IP address to make the connection and since this is no longer allowed for security reasons the connection fails. We have reported this problem to Oracle and also forwarded the information to the Java security dev list: http://mail.openjdk.java.net/pipermail/security-dev/2015-September/012845.html .

            Sorry, I was logged into the wrong account. The comment above from "Ops Team" is mine.

            Daniel Parks added a comment - Sorry, I was logged into the wrong account. The comment above from "Ops Team" is mine.

            Ops Team added a comment -

            This is a security vulnerability.

            An attacker with ability to spoof DNS can return a malicious A record to a host with a valid SSL cert with an IP as a SAN, and JIRA will connect without error (assuming that the error message is actually correct). This allows a remote attacker to intercept authentication requests and potentially gain write access to the authentication and authorization store (e.g. LDAP).

            It seems quite clear that this is a bug in JIRA, Confluence, and Crowd, since the description of the Java change in the release notes (referencing JDK-8067695) states that it changed reverse lookup.

            An application should never try to validate an SSL cert on anything other than the requested hostname. Any other behavior is a security vulnerability as described above.

            Ops Team added a comment - This is a security vulnerability. An attacker with ability to spoof DNS can return a malicious A record to a host with a valid SSL cert with an IP as a SAN, and JIRA will connect without error (assuming that the error message is actually correct). This allows a remote attacker to intercept authentication requests and potentially gain write access to the authentication and authorization store (e.g. LDAP). It seems quite clear that this is a bug in JIRA, Confluence, and Crowd, since the description of the Java change in the release notes (referencing JDK-8067695) states that it changed reverse lookup. An application should never try to validate an SSL cert on anything other than the requested hostname. Any other behavior is a security vulnerability as described above.

            We are also seeing the problem with Jira 6.4.10 when using a DNS name: ldap.ad.ucsd.edu.
            ldap.ad.ucsd.edu returns two IP's: 132.239.0.130 and 132.239.0.140.
            Each IP returns a cert with its DNS name as the subject and with ldap.ad.ucsd.edu as a
            subject alternative.
            For example, 132.239.0.130 returns a cert for beernuts.ad.ucsd.edu with ldap.ad.ucsd.edu
            as a subject alternative.
            Still Jira is connecting by IP and then failing because the IP isn't a subject alternative.
            Or does jira let java handle connecting, so it's a java problem?

            Ron Stanonik added a comment - We are also seeing the problem with Jira 6.4.10 when using a DNS name: ldap.ad.ucsd.edu. ldap.ad.ucsd.edu returns two IP's: 132.239.0.130 and 132.239.0.140. Each IP returns a cert with its DNS name as the subject and with ldap.ad.ucsd.edu as a subject alternative. For example, 132.239.0.130 returns a cert for beernuts.ad.ucsd.edu with ldap.ad.ucsd.edu as a subject alternative. Still Jira is connecting by IP and then failing because the IP isn't a subject alternative. Or does jira let java handle connecting, so it's a java problem?

            Also encountering this issue with a brand new JIRA installation, most recent CentOS7.

            Graham Leggett added a comment - Also encountering this issue with a brand new JIRA installation, most recent CentOS7.

            Leo Leung added a comment -

            Yes, I added the parameter -Djdk.tls.trustNameService=true and it is working now.

            Leo Leung added a comment - Yes, I added the parameter -Djdk.tls.trustNameService=true and it is working now.

              Unassigned Unassigned
              chparker ♚♛♜♝♞♟♔♕♖♗♘♙ (Inactive)
              Affected customers:
              11 This affects my team
              Watchers:
              48 Start watching this issue

                Created:
                Updated:
                Resolved: