Jira and JSM Mail Handlers fail to connect to Google Mail Servers with IMAP 30 days after they were configured with Oauth 2.0

Issue

When the Jira application is configured with an Incoming Mail Server using the combination of parameters below, any Mail Handler associated to this Mail Server will eventually fail to access the Mailbox 30 days after the Mail Server was configured:

• a Google Mail Server (Gmail)
• the SECURE IMAP protocol
• the Oauth 2.0 authorization method

Note

• Even though this bug was raised for the Jira Incoming Mail Handler (configured in ⚙ > System > Incoming Mail), this bug also applies to the Jira Service Management (JSM) Mail handler (configured in JSM projects in Project Settings > Email Requests). This is because both types of Mail Handlers use the same logic to fetch/refresh Oauth 2.0 tokens
• The issue might not occur exactly 30 days later. It might take a few more days than 30 days
• The issue is more likely to occur if the Jira Mail Handlers are configured to run less frequently than every 1 min (which is the default frequency)
• When using SECURE POP instead of SECURE IMAP, the connection will fail 1h later (instead of 30 days later) as per the other bug JRASERVER-74728

Steps to replicate

1. Configure an Oauth 2.0 integration using Google as the provider as per Configure an outgoing link in ⚙ > Applications > Application Links
2. Configure a Mail Server in ⚙ > System > Incoming Mail using a Gmail Server, the SECURE IMAP protocol and the Oauth 2.0 integration configured earlier as the authentication method
   • Authorize the mail box
   • Test the connection
   • Save the Mail Server
3. Configure a Mail Handler in ⚙ > System > Incoming Mail and associated it to the Mail Server configured earlier
4. Verify that new incoming mails are converted into new Jira tickets (or new comments)
5. Wait for 30 days

Expected results

30 days later, the Mail Handler should still work and convert new emails into new Jira tickets (or new comments).

Actual results

After 30 days, the Jira Mail Handler will eventually fail to connect to the Google Mail Server and fetch new emails.

The following symptoms will be observed:

• When going to the Mail Server configuration in ⚙ > System > Incoming Mail and trying to edit the Mail Server, we can see that the authentication method field is no longer set to anything while originally it was pointing to an Oauth 2.0 integration:
  
• The following errors are found in the Jira logs:
  • This error is found in the file atlassian-jira-outgoing-mail.log:

    2023-01-03 16:27:00,008+0000 ERROR [] Caesium-1-2 anonymous    GMAIL Mail Handler [c.a.m.s.managers.jira.OFBizMailServerManager] OAuth2 token not found for id=<TOKEN_ID>.
    

  • This error is found in the file atlassian-jira-incoming-mail.log:

    2023-01-03 16:27:00,048+0000 WARN [GMAIL Server] Caesium-1-2 anonymous    GMAIL Mail Handler GMAIL Mail Handler[10100]: javax.mail.AuthenticationFailedException: failed to connect, no password specified? while connecting to host "imap.gmail.com" as user "null" via protocol "SECURE_IMAP"
    

• When running the following SQL query to check if the token with ID <TOKEN_ID> (as shown in the atlassian-jira-outgoing-mail.log file) exists in the DB table AO_723324_CLIENT_TOKEN, the query does not return any result (the row is missing):

  select * from "AO_723324_CLIENT_TOKEN" where "ID" = <TOKEN_ID>;
  

Workaround

Workaround 1 - Short Term workaround

After you observe that the Mail Handlers associated to the problematic Mail Server fail to fetch emails, do the following:

• Edit the Mail Server configuration in ⚙ > System > Incoming Mail
• Re-select the Oauth 2.0 integration in the authentication method
• Re-authorize the mail server, Re-test the connection, and save the configuration

Workaround 2 - Long Term workaround

A long term workaround is to force update the expiration date of the Refresh token to many years in the future, for example the year 2099 by using the Epoch Timestamp 4101054453000.

The steps are listed below, and need to be repeated for each Jira Mail Server that is already configured, and also to any Jira Mail Server that will be configured in the future:

• Run the following SQL query after replacing <NAME> with the name of the Mail Server:

  select * from mailserver where name = <NAME>;
  

• Check the auth_conf column and take the value from the tokenId field
• Run the following SQL query after replacing <TOKEN_ID> with the Token ID obtained from the previous step:

  update "AO_723324_CLIENT_TOKEN" set "REFRESH_TOKEN_EXPIRATION" = 4101054453000 where "ID" = <TOKEN_ID>;