-
Bug
-
Resolution: Fixed
-
Low
-
None
-
None
-
Severity 2 - Major
-
Summary
Under some conditions, the Apache Mina SSH server bundled with Bitbucket Server may leak SSH session objects. Under some extreme conditions, this may cause Out Of Memory Errors.
If you open and close a connection really fast it seems to hold onto lots of them in IoServiceListenerSupport.
In the wild this can be caused by haproxy health checks.
Steps to Reproduce
The problem can be reproduced in environments where you're Setting up SSH port forwarding but you left a check port statement on your HAProxy. I.e:
backend your-backend--ssh
mode tcp
server your-server-hostname-stash <YOUR_BTIBUCKET_IP>:7999 check port 7999
Expected Results
The session object number in the JVM Heap is small.
Actual Results
If you set up a Heap Dump on an instance that runs out of memory, you would be able to see huge number of objects:
The data below was extracted from an actual customer's heap dump:
Class Objects Shallow Size Retained Size org.apache.sshd.server.session.ServerSession 328234 97157264 359743528 org.apache.sshd.common.future.DefaultCloseFuture 984456 31502592 41998144 org.apache.sshd.common.util.Buffer 328234 7877616 97156992
This condition could lead to an OutOfMemoryError 24-72 hours after your instance has started up.
This problem has been raised against Apache. See DIRMINA-1021 for further information.
Workaround
Remove the statement check port 7999 from your HAProxy configuration.
- is related to
-
-
- Closed
-
- mentioned in
-
![[Extranet] Page](/images/icons/generic_link_16.png "[Extranet] Page")
Page
Failed to load
| Form Name | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|