Running Bamboo with Oracle Database Native Network Encryption Degrades Performance or Causes Outages

Issue Summary

Oracle provides a feature called Native Network Encryption: ORACLE-BASE - Native Network Encryption for Database Connections This feature was previously part of the Advanced Security Option license, and provides connection encryption without requiring client side configuration.

When this feature is enabled, it adds 350ms+ to the time required to establish a database connection. This alone will cause noteworthy performance degradation, but when combined with the default database connection pool manager in Bamboo, c3p0, it can cause intermittent outages and extreme performance degradation.

Oracle have stated that this latency is working as intended: Slow Connection Using 12c Client When Network Encryption Is Enabled

It can affect any instance/DBMS (not just Oracle) where checking out a database connection takes longer than usual (e.g. just high latency) due to how c3p0 behaves. The performance degradation and outages are not aligned with whats expected given the latency on the checkout

Steps to Reproduce

1. Install any version of Bamboo
2. Install Oracle DB 11g or later with Native Network Encryption enabled
3. Introduce load to the system
4. Monitor Bamboo for delayed or timeout responses

Expected Results

Bamboo should work as per normal.

Actual Results

There is a prolonged delay in establishing database connections that causes c3p0 to get stuck in a loop of attempting to obtain additional database connections. As obtaining these database connections is slow, this will take longer than normal.

Bamboo will remain unresponsive until it reaches the c3p0 maximum pool size for the node.

This issue will not be visible in the logs by default, but the following KB provides additional details on how to diagnose this issue: Confluence Unresponsive Due to High Database Connection Latency

Workaround

A workaround is detailed on this KB: Confluence Unresponsive Due to High Database Connection Latency. While Confluence specific, the same applies to Bamboo.

However, it may be preferably to implement SSL to the database with proper certificate exchange, or disable Native Network Encryption entirely.