-
Bug
-
Resolution: Fixed
-
High (View bug fix roadmap)
-
9.1.0, 8.20.11, 9.2.0, 8.22.6, 9.4.6, 9.4.14
-
8.2
-
15
-
Severity 2 - Major
-
111
-
Description
This ticket is a continuation of JSWSERVER-21389. Our team thought the problem was resolved, but later it appeared we ran in troubles with concurrency, that's why that one (JSWSERVER-21417) was created. We have decided to create second ticket because root cause is slightly different and it affects next released versions than it's mentioned in JSWSERVER-21389.
Issue Summary
This is reproducible on Data Center: (yes)
Steps to Reproduce
- Install GitLab 15.0.
- Connect Jira to GitLab
- Wait 2 hours
- Try to perform full sync, this will fail intermittently
Expected Results
The token refresh should take place in the background and sync should work
Actual Results
The token expires and cannot be refreshed. Looks to be issue with threads in order to overwrite the new token.Below are the error logs
2022-07-22 16:50:25,612+0000 ajp-nio-127.0.0.1-8009-exec-397 url: /rest/bitbucket/1.0/repository/705/fullsync; user: xxxxxxxxx TRACE xxxxxxxxx 1010x56004x1 1c71cnj 10.60.128.173 /rest/bitbucket/1.0/repository/705/fullsync [c.a.o.c.lib.token.DefaultTokenService] Got OAuth Provider response: [400], [{"error":"invalid_grant","error_description":"The provided authorization grant is invalid, expired, revoked, does not match the redirection URI used in the authorization request, or was issued to another client."} ] 2022-07-22 16:50:25,614+0000 ajp-nio-127.0.0.1-8009-exec-397 url: /rest/bitbucket/1.0/repository/705/fullsync; user: xxxxxxxxx DEBUG xxxxxxxxx 1010x56004x1 1c71cnj 10.60.128.173 /rest/bitbucket/1.0/repository/705/fullsync [c.a.j.p.d.gitlab.impl.OAuthTokenProviderImpl] Failed to get access token ... 2022-07-22 16:51:46,322+0000 Caesium-1-2 DEBUG ServiceRunner [c.a.j.p.d.gitlab.impl.AbstractGitLabRestClientImpl] Received response [GET https://gitlab.assist.zone/api/v4/users?search=xxxxxxx&per_page=100 returned a response status of 200 OK] 2022-07-22 16:51:46,322+0000 Caesium-1-2 INFO ServiceRunner [c.a.j.p.dvcs.service.RepositoryServiceImpl] Could not load user [xxxxxx, xxxxxxx] com.atlassian.jira.plugins.dvcs.exception.SourceControlException: java.lang.NullPointerException
2022-07-25 13:08:06,585+0000 DVCSConnector.MessageExecutor:thread-48 DEBUG xxxxxxxxx 768x98182x1 10c6dyi 10.60.128.173 /rest/bitbucket/1.0/repository/846/fullsync [c.a.j.p.dvcs.event.ThreadEvents] There is no active ThreadEventsCaptor. Dropping event: DevSummaryChangedEvent{date=Mon Jul 25 13:08:06 UTC 2022, issueKeys=[DEP-512], repositoryId=846, dvcsType='gitlabselfmanaged'} 2022-07-25 13:08:06,590+0000 DVCSConnector.MessageExecutor:thread-48 DEBUG xxxxxxxxxx 768x98182x1 10c6dyi 10.60.128.173 /rest/bitbucket/1.0/repository/846/fullsync [c.a.j.p.d.service.message.MessageConsumer] Could not acquire token for consumer: 'com.atlassian.jira.plugins.dvcs.sync.GitLabSynchronizeChangesetMessageConsumer@23319f60' with id 'com.atlassian.jira.plugins.dvcs.spi.gitlab.message.GitLabSynchronizeChangesetMessage'. There are already '2' threads processing the messages.
mysql> select * from AO_723324_CLIENT_TOKEN \G
*************************** 1. row ***************************
ACCESS_TOKEN: 3bb0d420b0f586f9f7b43aec2263572b46d70c1c7233451c329f1d250c75d049
ACCESS_TOKEN_EXPIRATION: 1658509102379
CONFIG_ID: aef8a204-aa84-4085-babe-42cc67d5b37a
ID: 216a24ff-d0e9-4f7e-87d4-619e12d233dd
LAST_REFRESHED: NULL
LAST_STATUS_UPDATED: 1658753719903
REFRESH_COUNT: 0
REFRESH_TOKEN: acca34b4145b496b99c362715a2331f08a58fa4957d8601a7b1600570f8322f3
REFRESH_TOKEN_EXPIRATION: 1661094301379
STATUS: UNRECOVERABLE
Workaround
Workaround is not very helpful because it consist in recreating GitLab integration after every 2 hours.
| Form Name | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
[JSWSERVER-21417] Jira access token getting expired after 2 hours while connecting to Gitlab
| Remote Link | Original: This issue links to "Page (Confluence)" [ 944264 ] |
| Remote Link | New: This issue links to "Page (Confluence)" [ 944264 ] |
| Affects Version/s | New: 8.22.6 [ 102204 ] |
| Affects Version/s | New: 9.4.14 [ 106446 ] |
| Affects Version/s | New: 9.4.6 [ 104662 ] |
| Remote Link | New: This issue links to "Page (Confluence)" [ 723511 ] |
| Remote Link | New: This issue links to "Page (Confluence)" [ 706261 ] |
We have JIRA v9.1.0 (Upgraded to get the DVCS connector working with Gitlab) as the gitlab was forcing token expiration and JIRA DVCS had to provide a fix to use the refresh token. Without this the repo sync was stopping after two hours for the repositories which were populated.
Once we upgraded to Gitlab ee 14.2.2 (latest) its broken again now and this time its even worse and it cannot even populate the list of repos to sync.
Jarrod, thank you for your response!
I am new to using Jira and this was the first issue I had encountered, so when I found this ticket, I was unaware how separate the development for Cloud and Server/Data Centre are.
I will log a ticket for this issue with Jira Cloud.
Thanks you!
The problem is analyzed with JIRA 8.20.11 and GitLab 15.4.3 ee in https://jira.atlassian.com/browse/JSWSERVER-21243?focusedCommentId=3189438&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-3189438