Support for rotation of internal application credentials (those of external value)

As discussed extensively on JRASERVER-73857, we have identified that there are some internal credentials in the apps we use (currently Jira, Bitbucket, and Confluence) that are not at all acknowledged in the application UI nor do they have a path for rotation of these credentials. 

This ticket is the "Bitbucket" version of JRASERVER-73857.

We are looking for:

1. A complete list of credentials that are stored internally in Jira, Confluence, and Bitbucket that should be rotated periodically per standard industry security practices – that is – internal credentials that potentially have ‘value’ to an external attacker.  Specifically out-of-scope are:
   • End-user credentials (since the product already has a way to enumerate users and change their passwords and API keys or similar). 
   • Credentials stored by 3rd party plugins.
2. Some mechanism by which system administrators can accomplish a rotation of each internal credential. This could be an in-app feature (either one that exists now, or is added in the future), set of scripts, or some configuration process.  Mainly my focus is internal credentials that potentially have value to an external attacker.

-------------------------

Additional clarifications:

 
We are not talking about user credentials, oauth tokens, or API keys here.  Instead, we are talking about secrets internal to the application.  For example, it was pointed out that https://confluence.atlassian.com/jirasoftwareserver/allowing-oauth-access-939938965.html#AllowingOAuthaccess-revokingtokensRevokingOAuthaccesstokens documents how to revoke OAUTH tokens for existing app integrations.  It was also pointed out that this ticket exists: JRASERVER-72198: As a Jira administrator I would like that OAuth tokens are removed when a user is disabled (inactive).

But, this is not what I'm getting at here. The OAuth signing keys that I’ve been harping on so far are internally stored in the database and it’s not at all obvious that they even exist (unless you happen to know something about OAUTH which I do).

<BEGIN QUOTING>

Here are some more details about our set up:

• We are using “OAuth (Impersonation)” auth type for all links.
• We have links from Jira <> Confluence, Jira <> Bitbucket, Bitbucket <-> Confluence
• We are NOT using application tunnels.

Since OAuth (impersonation) is in play, I believe each app has its own cryptographic private key which is signing the OAuth messages between the two apps. Unfortunately, none of the links provided above supply any method of rotating each app’s OAuth private key. I also tried searching online and didn’t find anything.

Doing a quick look at Jira’s mysql database, I see:

mysql> describe oauthconsumer;
+------------------+---------------+------+-----+---------+-------+
| Field            | Type          | Null | Key | Default | Extra |
+------------------+---------------+------+-----+---------+-------+
| ID               | decimal(18,0) | NO   | PRI | NULL    |       |
| CREATED          | datetime      | YES  |     | NULL    |       |
| consumername     | varchar(255)  | YES  |     | NULL    |       |
| CONSUMER_KEY     | varchar(255)  | YES  | UNI | NULL    |       |
| consumerservice  | varchar(255)  | YES  | UNI | NULL    |       |
| PUBLIC_KEY       | text          | YES  |     | NULL    |       |
| PRIVATE_KEY      | text          | YES  |     | NULL    |       |
| DESCRIPTION      | text          | YES  |     | NULL    |       |
| CALLBACK         | text          | YES  |     | NULL    |       |
| SIGNATURE_METHOD | varchar(60)   | YES  |     | NULL    |       |
| SHARED_SECRET    | text          | YES  |     | NULL    |       |
+------------------+---------------+------+-----+---------+-------+
11 rows in set (0.00 sec)

(note PRIVATE_KEY column)

Running

select * from oauthconsumer;

I get:

| 10100 | 2022-04-18 21:54:32 | Jira-Test    | jira:<uuid> | __HOST_SERVICE__ | <redacted> | <redacted> | Atlassian Jira at <url> | NULL     | RSA_SHA1         | NULL          |

Sure enough there is a public/private keypair being retained by this Jira instance. I’d expect the other apps are doing similar.

<end of quote>

The existence of these OAuth signing private keys are entirely hidden from the JIRA, Confluence and Bitbucket UIs. They are also completely undocumented. The request of this ticket is to:

1. Identify any other said ‘undocumented’ secrets or credentials maintained in the application.
2. Provide a process by which these can be rotated.

 
I of course could be wrong, but my guess is that there are not too many more credentials that are currently in this state (namely, those not having a rotation story). When the some Atlassian folks previously dug into this for us, what they heard from the team is that there are many many similar internal credentials; however, I think a lot of the examples cited were credentials that - yes - do exist internally, but I think MANY of those are credentials that don’t have external value to an attacker.

For example, if Bitbucket is connecting to elasticsearch on localhost (and elasticsearch is bound to 127.0.0.1) then even if I somehow steal the elasticsearch password, it’s of no value, right?

But, say - in the previous examples of OAUTH token signing keys - those do have value to an attacker because the tokens are processed by external-facing APIs (and having the keys would let one forge fake tokens).

I’m of course only really interested in a list of (and rotation story for) credentials that have external value to an attacker.  Hopefully this differentiation makes sense.