Uploaded image for project: 'Bitbucket Data Center'
  1. Bitbucket Data Center
  2. BSERV-9203

Bitbucket Search fails when connecting to Elasticsearch over HTTPS in certain environments

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Low Low
    • 4.14.4
    • 4.9.1
    • None

      Summary

      Bitbucket Search fails to connect to a SSL enabled Elasticsearch instance with the following error in catalina.out:

      Exception in thread "I/O dispatcher 11" java.lang.NoClassDefFoundError: javax/security/auth/x500/X500Principal
      

      Environment

      • Elasticsearch 2.3.1
      • Buckler 0.2.1
      • Java 1.8.0_101
      • Ubuntu 14.04

      Steps to Reproduce

      1. Enable HTTPS on Elasticsearch with a self-signed certificate and import this certificate into the truststore of the JVM running Bitbucket Server
        1. For example, with Buckler:
          tls.http.enabled: true
          tls.tcp.enabled: true
          tls.keystore.path: /path/to/elasticsearch.jks
          tls.keystore.password: changeit
      2. Configure Bitbucket Server to point to the HTTPS URL for Elasticsearch
      3. The test will fail and searches also fail

      Expected Results

      Connection test and search works without error.

      Actual Results

      The following is thrown in the atlassian-bitbucket.log:

      2016-10-03 23:09:15,539 DEBUG [I/O dispatcher 13]  c.a.b.s.s.t.DefaultElasticsearchConnectionTester Detailed Exception: 
      java.util.concurrent.CompletionException: org.apache.http.ConnectionClosedException: Connection closed unexpectedly
              at java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:292) ~[na:1.8.0_101]
              at java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:308) ~[na:1.8.0_101]
              at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:593) ~[na:1.8.0_101]
              at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577) ~[na:1.8.0_101]
              at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474) ~[na:1.8.0_101]
              at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1977) ~[na:1.8.0_101]
              at com.atlassian.elasticsearch.client.apache.httpclient.ApacheRequestExecutor$1.failed(ApacheRequestExecutor.java:144) ~[na:na]
              at org.apache.http.concurrent.BasicFuture.failed(BasicFuture.java:134) ~[httpcore-4.4.4.jar:4.4.4]
              at org.apache.http.impl.nio.client.AbstractClientExchangeHandler.failed(AbstractClientExchangeHandler.java:414) ~[na:na]
              at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.closed(HttpAsyncRequestExecutor.java:137) ~[na:na]
              at org.apache.http.impl.nio.client.InternalIODispatch.onClosed(InternalIODispatch.java:71) ~[na:na]
              at org.apache.http.impl.nio.client.InternalIODispatch.onClosed(InternalIODispatch.java:39) ~[na:na]
              at org.apache.http.impl.nio.reactor.AbstractIODispatch.disconnected(AbstractIODispatch.java:102) ~[na:na]
              at org.apache.http.impl.nio.reactor.BaseIOReactor.sessionClosed(BaseIOReactor.java:281) ~[na:na]
              at org.apache.http.impl.nio.reactor.AbstractIOReactor.processClosedSessions(AbstractIOReactor.java:442) ~[na:na]
              at org.apache.http.impl.nio.reactor.AbstractIOReactor.hardShutdown(AbstractIOReactor.java:578) ~[na:na]
              at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:307) ~[na:na]
              at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:106) ~[na:na]
              at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:590) ~[na:na]
              at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_101]
      Caused by: org.apache.http.ConnectionClosedException: Connection closed unexpectedly
              ... 11 common frames omitted
      

      And the corresponding entry in catalina.out:

      Exception in thread "I/O dispatcher 13" java.lang.NoClassDefFoundError: javax/security/auth/x500/X500Principal
              at org.apache.http.conn.ssl.DefaultHostnameVerifier.verify(DefaultHostnameVerifier.java:115)
              at org.apache.http.conn.ssl.DefaultHostnameVerifier.verify(DefaultHostnameVerifier.java:87)
              at org.apache.http.nio.conn.ssl.SSLIOSessionStrategy.verifySession(SSLIOSessionStrategy.java:204)
              at org.apache.http.nio.conn.ssl.SSLIOSessionStrategy$1.verify(SSLIOSessionStrategy.java:188)
              at org.apache.http.nio.reactor.ssl.SSLIOSession.doHandshake(SSLIOSession.java:366)
              at org.apache.http.nio.reactor.ssl.SSLIOSession.isAppInputReady(SSLIOSession.java:507)
              at org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:122)
              at org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:164)
              at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:339)
              at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:317)
              at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:278)
              at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:106)
              at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:590)
              at java.lang.Thread.run(Thread.java:745)

      Workaround

      Use HTTP to connect to Elasticsearch.

          Form Name

            [BSERV-9203] Bitbucket Search fails when connecting to Elasticsearch over HTTPS in certain environments

              pathompson Paul Thompson (Inactive)
              jethomas Jeff Thomas
              Affected customers:
              4 This affects my team
              Watchers:
              10 Start watching this issue

                Created:
                Updated:
                Resolved: