Uploaded image for project: 'Bitbucket Data Center'
  1. Bitbucket Data Center
  2. BSERV-2662

c.a.stash.ssh.server.SshServer - Server is started twice, throws java.net.BindException: Address already in use

    XMLWordPrintable

Details

    • Bug
    • Resolution: Not a bug
    • Low
    • None
    • 1.2.0
    • SSH
    • None

    Description

      Stash 1.2.0 attempts to start the SSH server twice during bootstrap:

      Excerpt from /opt/stash/data/log/atlassian-stash.log
      2012-08-07 23:39:38,464 INFO  [main]  c.a.s.internal.home.HomeLockAcquirer Successfully acquired lock on home directory /opt/stash/data for 7457@lem0
2012-08-07 23:39:42,670 INFO  [main]  c.a.s.i.h.ExtendedAnnotationSessionFactoryBean Building new Hibernate SessionFactory
2012-08-07 23:39:50,274 INFO  [main]  c.a.s.i.server.ApplicationInfoLogger Starting Stash 1.2.0 (cf5bc63d65 built on Tue Aug 07 14:02:36 CDT 2012)
2012-08-07 23:39:50,274 INFO  [main]  c.a.s.i.server.ApplicationInfoLogger JVM: Sun Microsystems Inc. Java HotSpot(TM) 64-Bit Server VM 1.6.0_32-b05
2012-08-07 23:39:56,225 INFO  [Spring executor 1]  c.a.stash.ssh.server.SshServer Starting SSH server on port 7999...
2012-08-07 23:39:56,343 INFO  [Spring executor 1]  c.a.stash.ssh.server.SshServer Started SSH server successfully.
2012-08-07 23:39:58,477 INFO  [main]  c.a.s.i.scm.PluginScmClientProvider Stash is using git version 1.7.10.4
2012-08-07 23:40:03,355 INFO  [main]  c.a.s.internal.home.HomeLockAcquirer Successfully acquired lock on home directory /opt/stash/data for 7457@lem0
2012-08-07 23:40:06,940 INFO  [main]  c.a.s.i.h.ExtendedAnnotationSessionFactoryBean Building new Hibernate SessionFactory
2012-08-07 23:40:13,148 INFO  [main]  c.a.s.i.server.ApplicationInfoLogger Starting Stash 1.2.0 (cf5bc63d65 built on Tue Aug 07 14:02:36 CDT 2012)
2012-08-07 23:40:13,149 INFO  [main]  c.a.s.i.server.ApplicationInfoLogger JVM: Sun Microsystems Inc. Java HotSpot(TM) 64-Bit Server VM 1.6.0_32-b05
2012-08-07 23:40:18,347 INFO  [Spring executor 5]  c.a.stash.ssh.server.SshServer Starting SSH server on port 7999...
2012-08-07 23:40:18,408 ERROR [Spring executor 5]  c.a.stash.ssh.server.SshServer SSH server failed to start
java.net.BindException: Address already in use
        at sun.nio.ch.Net.bind(Native Method) ~[na:1.6.0_32]
        at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:126) ~[na:1.6.0_32]
        at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:59) ~[na:1.6.0_32]
        at org.apache.mina.transport.socket.nio.NioSocketAcceptor.open(NioSocketAcceptor.java:201) ~[stash-ssh-1.2.0_1344366454000.jar:na]
        at org.apache.mina.transport.socket.nio.NioSocketAcceptor.open(NioSocketAcceptor.java:51) ~[stash-ssh-1.2.0_1344366454000.jar:na]
        at org.apache.mina.core.polling.AbstractPollingIoAcceptor.registerHandles(AbstractPollingIoAcceptor.java:543) ~[stash-ssh-1.2.0_1344366454000.jar:na]
        at org.apache.mina.core.polling.AbstractPollingIoAcceptor.access$300(AbstractPollingIoAcceptor.java:67) ~[stash-ssh-1.2.0_1344366454000.jar:na]
        at org.apache.mina.core.polling.AbstractPollingIoAcceptor$Acceptor.run(AbstractPollingIoAcceptor.java:418) ~[stash-ssh-1.2.0_1344366454000.jar:na]
        at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:64) ~[stash-ssh-1.2.0_1344366454000.jar:na]
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) [na:1.6.0_32]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) [na:1.6.0_32]
        at java.lang.Thread.run(Thread.java:662) [na:1.6.0_32]
2012-08-07 23:40:19,846 INFO  [main]  c.a.s.i.scm.PluginScmClientProvider Stash is using git version 1.7.10.4

      Further, Administration - Settings - Server Settings - SSH shows the following:

      The SSH server failed to start on port 7999 because it is already being used.

      Repositories do not show the git+ssh URI.

      lsof shows the default SSH port of Stash bound by the JVM:

      stash@lem0:~$ lsof -n -i -a -p `ps ax |grep [D]atlassian.standalone=STASH|awk '{print $1}'`
COMMAND  PID  USER   FD   TYPE    DEVICE SIZE/OFF NODE NAME
java    7457 stash   37u  IPv6 358446742      0t0  TCP *:7990 (LISTEN)
java    7457 stash   40u  IPv6 358452544      0t0  TCP 127.0.0.1:8006 (LISTEN)
java    7457 stash  193u  IPv6 358435190      0t0  TCP 127.0.0.1:47415->127.0.0.1:postgresql (ESTABLISHED)
java    7457 stash  196u  IPv6 358408106      0t0  TCP 127.0.0.1:47416->127.0.0.1:postgresql (ESTABLISHED)
java    7457 stash  197u  IPv6 358455417      0t0  TCP 127.0.0.1:47417->127.0.0.1:postgresql (ESTABLISHED)
java    7457 stash  198u  IPv6 358444191      0t0  TCP 127.0.0.1:47418->127.0.0.1:postgresql (ESTABLISHED)
java    7457 stash  199u  IPv6 358455418      0t0  TCP 127.0.0.1:47419->127.0.0.1:postgresql (ESTABLISHED)
java    7457 stash  200u  IPv6 358444192      0t0  TCP 127.0.0.1:47420->127.0.0.1:postgresql (ESTABLISHED)
java    7457 stash  201u  IPv6 358455419      0t0  TCP 127.0.0.1:47421->127.0.0.1:postgresql (ESTABLISHED)
java    7457 stash  202u  IPv6 358435193      0t0  TCP 127.0.0.1:47422->127.0.0.1:postgresql (ESTABLISHED)
java    7457 stash  203u  IPv6 357369797      0t0  TCP 127.0.0.1:47423->127.0.0.1:postgresql (ESTABLISHED)
java    7457 stash  205u  IPv6 358452541      0t0  TCP 127.0.0.1:47424->127.0.0.1:postgresql (ESTABLISHED)
java    7457 stash  206u  IPv6 357516120      0t0  TCP 127.0.0.1:47425->127.0.0.1:postgresql (ESTABLISHED)
java    7457 stash  207u  IPv6 358444193      0t0  TCP 127.0.0.1:47426->127.0.0.1:postgresql (ESTABLISHED)
java    7457 stash  208u  IPv6 358455421      0t0  TCP 127.0.0.1:47427->127.0.0.1:postgresql (ESTABLISHED)
java    7457 stash  209u  IPv6 358444194      0t0  TCP 127.0.0.1:47428->127.0.0.1:postgresql (ESTABLISHED)
java    7457 stash  210u  IPv6 358455422      0t0  TCP 127.0.0.1:47429->127.0.0.1:postgresql (ESTABLISHED)
java    7457 stash  211u  IPv6 358444195      0t0  TCP 127.0.0.1:47430->127.0.0.1:postgresql (ESTABLISHED)
java    7457 stash  457u  IPv6 358458343      0t0  TCP *:7999 (LISTEN)
java    7457 stash  459u  IPv6 357516631      0t0  TCP 172.30.10.9:51467->63.246.22.212:https (CLOSE_WAIT)
java    7457 stash  617u  IPv6 358431003      0t0  TCP 127.0.0.1:39697->127.0.0.1:http-alt (CLOSE_WAIT)
java    7457 stash  621u  IPv6 358455574      0t0  TCP 127.0.0.1:47434->127.0.0.1:postgresql (ESTABLISHED)
java    7457 stash  622u  IPv6 358444196      0t0  TCP 127.0.0.1:47435->127.0.0.1:postgresql (ESTABLISHED)
java    7457 stash  624u  IPv6 358455575      0t0  TCP 127.0.0.1:47436->127.0.0.1:postgresql (ESTABLISHED)
java    7457 stash  625u  IPv6 358444197      0t0  TCP 127.0.0.1:47437->127.0.0.1:postgresql (ESTABLISHED)
java    7457 stash  626u  IPv6 358455576      0t0  TCP 127.0.0.1:47438->127.0.0.1:postgresql (ESTABLISHED)
java    7457 stash  627u  IPv6 358444198      0t0  TCP 127.0.0.1:47439->127.0.0.1:postgresql (ESTABLISHED)
java    7457 stash  628u  IPv6 358455577      0t0  TCP 127.0.0.1:47440->127.0.0.1:postgresql (ESTABLISHED)
java    7457 stash  629u  IPv6 358444199      0t0  TCP 127.0.0.1:47441->127.0.0.1:postgresql (ESTABLISHED)
java    7457 stash  630u  IPv6 357369804      0t0  TCP 127.0.0.1:47442->127.0.0.1:postgresql (ESTABLISHED)
java    7457 stash  631u  IPv6 358444200      0t0  TCP 127.0.0.1:47443->127.0.0.1:postgresql (ESTABLISHED)
java    7457 stash  632u  IPv6 358455578      0t0  TCP 127.0.0.1:47444->127.0.0.1:postgresql (ESTABLISHED)
java    7457 stash  633u  IPv6 358444201      0t0  TCP 127.0.0.1:47445->127.0.0.1:postgresql (ESTABLISHED)
java    7457 stash  634u  IPv6 357369806      0t0  TCP 127.0.0.1:47446->127.0.0.1:postgresql (ESTABLISHED)
java    7457 stash  635u  IPv6 358444202      0t0  TCP 127.0.0.1:47447->127.0.0.1:postgresql (ESTABLISHED)
java    7457 stash  636u  IPv6 358455579      0t0  TCP 127.0.0.1:47448->127.0.0.1:postgresql (ESTABLISHED)
java    7457 stash  637u  IPv6 358444203      0t0  TCP 127.0.0.1:47449->127.0.0.1:postgresql (ESTABLISHED)
java    7457 stash  642u  IPv6 358408109      0t0  TCP 127.0.0.1:47988->127.0.0.1:http-alt (CLOSE_WAIT)
java    7457 stash  859u  IPv6 358435216      0t0  TCP 172.30.10.9:53162->63.246.22.212:https (CLOSE_WAIT)
stash@lem0:~$

      Git operations using the git+ssh port do work:

      pdzwart@merkurij:~/Downloads[15:03:51](0,1)$ echo '0000' | ssh -p 7999 git@lem0.perfeng.private.atlassian.com "git-upload-pack '/PROJ/repo.git'"
0000pdzwart@merkurij:~/Downloads[15:04:03](0,0)$

      Environment

      Tuning as default unless otherwise specified.

      Hardware Specifications

      Chassis: Silicon Mechanics Rackform iServ R346.v3
      CPUs: 2x Intel Xeon CPU E5-2650 @ 2 GHz
      Memory: 128GB DDR3 ECC SDRAM, 1.6 GHz
      Storage:
      • 4x Intel 520 SATA SSD, RAID0 (write through cache) via LSI 9266-8i, mounted as:
        • /var/lib/postgresql, ext3, block size=4 KB, stride=2 blocks, stripe width=8 blocks, noatime
        • /opt, ext3, block size=4 KB, stride=2 blocks, stripe width=8 blocks, noatime
      • 1x Intel 520 SATA SSD, mounted as:
        • /, ext3, noatime
      • 1x Western Digital RE4, mounted as:
        • /archive, ext3
      NICs:
      • 1x Intel I350-T2
      • 1x Solarflare SFN5121T

      Operating System

      Name: Debian GNU/Linux
      Vendor: Debian Project
      Version: Wheezy/Sid
      Architecture: x86-64

      Java

      Name: Java SE Development Kit
      Vendor: Oracle Corporation
      Version: 1.6.0_32
      Architecture: x86-64

      Database

      Name: PostgreSQL
      Vendor: PostgreSQL Global Development Group
      Version: 9.1.4
      Architecture: x86-64

      JIRA

      Version: 5.1
      JDBC: Direct
      JVM Options: -XX:+PrintGC -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -Xmx24g -XX:+UseParallelOldGC
      Application home: /opt/jira/data
      Mode: Private
      Tomcat context path: /jira
      jpm.xml
      • Set jira.autoexport to false
      Services
      • Delete com.atlassian.jira.service.services.export.ExportService
      PAM limits, nofile:
      • Soft: 8192
      • Hard: 8192

      Stash

      Version: 1.2.0
      JDBC: Direct
      JVM Options: -XX:+PrintGC -XX:+PrintGCDetails -XX:+PrintGCTimeStamps
      Application home: /opt/stash/data
      Tomcat context path: /stash
      Applinks: Integrated with JIRA.
      Embedded crowd: Backed off JIRA, via 'Integrate with JIRA' set-up option.

      Attachments

        Activity

          People

            mwatson@atlassian.com mwatson
            pdzwart PdZ (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: