-
Bug
-
Resolution: Unresolved
-
Low (View bug fix roadmap)
-
None
-
9.12.15
-
9.12
-
2
-
Severity 3 - Minor
-
40
-
Issue Summary
DVCS Connector performing a Full/soft/hourly sync for a GitLab project (repository) may fail if any of the following parameters of a Merge Request is longer than 255 characters:
- AUTHOR
- SOURCE_BRANCH
- DESTINATION_BRANCH
- EXECUTED_BY
- LAST_STATUS
- SOURCE_REPO
- URL
jira=# \d+ "AO_E8B6CC_PULL_REQUEST";
Table "public.AO_E8B6CC_PULL_REQUEST"
Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
--------------------+-----------------------------+-----------+----------+------------------------------------------------------+----------+--------------+-------------
AUTHOR | character varying(255) | | | | extended | |
COMMENT_COUNT | integer | | | 0 | plain | |
CREATED_ON | timestamp without time zone | | | | plain | |
DESTINATION_BRANCH | character varying(255) | | | | extended | |
DOMAIN_ID | integer | | not null | 0 | plain | |
EXECUTED_BY | character varying(255) | | | | extended | |
ID | integer | | not null | nextval('"AO_E8B6CC_PULL_REQUEST_ID_seq"'::regclass) | plain | |
LAST_STATUS | character varying(255) | | | | extended | |
NAME | character varying(255) | | | | extended | |
REMOTE_ID | bigint | | | | plain | |
SOURCE_BRANCH | character varying(255) | | | | extended | |
SOURCE_REPO | character varying(255) | | | | extended | |
TO_REPOSITORY_ID | integer | | | 0 | plain | |
UPDATED_ON | timestamp without time zone | | | | plain | |
URL | character varying(255) | | | | extended | |
Even though the "NAME" column is also limited to 255 chars on Jira DB table, our code currently protects against longer MR title(s) by stripping it to 255 chars to fit into the column restrictions of AO_E8B6CC_PULL_REQUEST table.
Even though GitLab does limit the MR fields viz., source_branch, target_branch etc. to 255 characters it does so only in the UI. It does not impose the 255 char limit on its database table merge_requests.
# \d+ merge_requests;
Table "public.merge_requests"
Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
------------------------------+-----------------------------+-----------+----------+--------------------------------------------+----------+-------------+--------------+-------------
id | bigint | | not null | nextval('merge_requests_id_seq'::regclass) | plain | | |
target_branch | character varying | | not null | | extended | | |
source_branch | character varying | | not null | | extended | | |
source_project_id | bigint | | | | plain | | |
author_id | bigint | | | | plain | | |
assignee_id | bigint | | | | plain | | |
title | character varying | | | | extended | | |
created_at | timestamp without time zone | | | | plain | | |
updated_at | timestamp without time zone | | | | plain | | |
milestone_id | bigint | | | | plain | | |
merge_status | character varying | | not null | 'unchecked'::character varying | extended | | |
target_project_id | bigint | | not null | | plain | | |
iid | integer | | | | plain | | |
description | text | | | | extended | | |
updated_by_id | bigint | | | | plain | | |
merge_error | text | | | | extended | | |
merge_params | text | | | | extended | | |
merge_when_pipeline_succeeds | boolean | | not null | false | plain | | |
merge_user_id | bigint | | | | plain | | |
merge_commit_sha | character varying | | | | extended | | |
approvals_before_merge | integer | | | | plain | | |
rebase_commit_sha | character varying | | | | extended | | |
in_progress_merge_commit_sha | character varying | | | | extended | | |
lock_version | integer | | | 0 | plain | | |
title_html | text | | | | extended | | |
description_html | text | | | | extended | | |
time_estimate | integer | | | 0 | plain | | |
squash | boolean | | not null | false | plain | | |
cached_markdown_version | integer | | | | plain | | |
last_edited_at | timestamp without time zone | | | | plain | | |
last_edited_by_id | bigint | | | | plain | | |
merge_jid | character varying | | | | extended | | |
discussion_locked | boolean | | | | plain | | |
latest_merge_request_diff_id | bigint | | | | plain | | |
allow_maintainer_to_push | boolean | | | true | plain | | |
state_id | smallint | | not null | 1 | plain | | |
rebase_jid | character varying | | | | extended | | |
squash_commit_sha | bytea | | | | extended | | |
sprint_id | bigint | | | | plain | | |
merge_ref_sha | bytea | | | | extended | | |
draft | boolean | | not null | false | plain | | |
prepared_at | timestamp with time zone | | | | plain | | |
merged_commit_sha | bytea | | | | extended | | |
override_requested_changes | boolean | | not null | false | plain | | |
head_pipeline_id | bigint | | | | plain | | |
imported_from | smallint | | not null | 0 | plain | | |
retargeted | boolean | | not null | false | plain | | |
This is reproducible on Data Center: yes
Steps to Reproduce
The Sync Failure can be reproduced in both soft_sync/hourly syncs and Full Sync. The exception stack trace is little bit different.
To reproduce this during soft_sync/hourly sync:
- Create a Merge Request on Gitlab normally
- Do a normal full sync of a repo
- Update the GitLab DB and make the source_branch of the existing MR longer than 255 characters in the merge_requests table:
- Ex.
gitlabhq_production=# UPDATE merge_requests SET source_branch = 'feat_SCRUM-21_KANBAN-3_KANBAN-16_KANBAN-23_KANBAN-2_feat_SCRUM-21_KANBAN-3_KANBAN-16_KANBAN-23_KANBAN-2_feat_SCRUM-21_KANBAN-3_KANBAN-16_KANBAN-23_KANBAN-2_feat_SCRUM-21_KANBAN-3_KANBAN-16_KANBAN-23_KANBAN-2_feat_SCRUM-21_KANBAN-3_KANBAN-16_KANBAN-23_KANBAN-2_feat_SCRUM-21_KANBAN-3_KANBAN-16_KANBAN-23_KANBAN-2_feat_SCRUM-21_KANBAN-3_KANBAN-16_KANBAN-23_KANBAN-2' WHERE id = 5;
- Why manually? To mimic a cloned repository from a different VCS provider where no such limit applies
- Ex.
- Make an MR comment on Gitlab to refresh the GitLab cache (this would also change GitLab's merge_requests table's updated_at value)
- Do a soft sync again or wait for the next hourly sync to occur
To reproduce this during Full sync:
- Same steps as above, except don't do a Full sync until the DB manipulation is done
- Or, just do a Full sync on an existing repo after following the steps above
Expected Results
DVCS Connector App code protects against non-standard development practices where branch names are longer than 255 characters like it does protect against MR titles being longer than 255 chars.
Actual Results
- Sync of the repository fails
. - For hourly/soft syncs, the following exception is seen in atlassian-jira.log:
2025-01-13 14:28:21,216+0100 DVCSConnector.MessageExecutor:thread-78 ERROR [c.a.j.p.d.service.message.MessageConsumer] Synchronization failed: null java.lang.reflect.UndeclaredThrowableException at jdk.proxy208/jdk.proxy208.$Proxy5725.save(Unknown Source) at com.atlassian.jira.plugins.dvcs.dao.impl.RepositoryPullRequestDaoImpl.lambda$updatePullRequestInfo$3(RepositoryPullRequestDaoImpl.java:180) ... at com.atlassian.jira.plugins.dvcs.dao.impl.RepositoryPullRequestDaoImpl.updatePullRequestInfo(RepositoryPullRequestDaoImpl.java:179) at com.atlassian.jira.plugins.dvcs.service.PullRequestServiceImpl.updatePullRequest(PullRequestServiceImpl.java:148) at com.atlassian.jira.plugins.dvcs.sync.GitLabMergeRequestHandler.processMergeRequest(GitLabMergeRequestHandler.java:92) at com.atlassian.jira.plugins.dvcs.sync.GitLabMergeRequestHandler.processMergeRequest(GitLabMergeRequestHandler.java:74) at com.atlassian.jira.plugins.dvcs.sync.GitLabSynchronizePullRequestMessageConsumer.onReceive(GitLabSynchronizePullRequestMessageConsumer.java:64) at com.atlassian.jira.plugins.dvcs.sync.GitLabSynchronizePullRequestMessageConsumer.onReceive(GitLabSynchronizePullRequestMessageConsumer.java:20) at com.atlassian.jira.plugins.dvcs.service.MessageExecutor$MessageRunnable.doRun(MessageExecutor.java:364) at com.atlassian.jira.plugins.dvcs.service.MessageExecutor$ReleaseTokenAndEnqueueNextMessage.run(MessageExecutor.java:274) ... Caused by: org.postgresql.util.PSQLException: ERROR: value too long for type character varying(255) ...
- For Full syncs, the following exception is seen in atlassian-jira.log:
2025-01-30 15:24:30,093+0000 DVCSConnector.MessageExecutor:thread-5 ERROR [c.a.j.p.d.service.message.MessageConsumer] Synchronization failed: There was a SQL exception thrown by the Active Objects library: Database: - name:PostgreSQL - version:12.22 (Debian 12.22-1.pgdg120+1) - minor version:22 - major version:12 Driver: - name:PostgreSQL JDBC Driver - version:42.7.2 org.postgresql.util.PSQLException: ERROR: value too long for type character varying(255) com.atlassian.activeobjects.internal.ActiveObjectsSqlException: There was a SQL exception thrown by the Active Objects library: Database: - name:PostgreSQL - version:12.22 (Debian 12.22-1.pgdg120+1) - minor version:22 - major version:12 Driver: - name:PostgreSQL JDBC Driver - version:42.7.2 org.postgresql.util.PSQLException: ERROR: value too long for type character varying(255) at com.atlassian.activeobjects.internal.EntityManagedActiveObjects.create(EntityManagedActiveObjects.java:103) at com.atlassian.activeobjects.osgi.TenantAwareActiveObjects.create(TenantAwareActiveObjects.java:275) ... at com.atlassian.jira.plugins.dvcs.dao.impl.RepositoryPullRequestDaoImpl.lambda$savePullRequest$0(RepositoryPullRequestDaoImpl.java:112) ... at com.atlassian.jira.plugins.dvcs.dao.impl.RepositoryPullRequestDaoImpl.savePullRequest(RepositoryPullRequestDaoImpl.java:109) at com.atlassian.jira.plugins.dvcs.service.PullRequestServiceImpl.createPullRequest(PullRequestServiceImpl.java:119) at com.atlassian.jira.plugins.dvcs.sync.GitLabMergeRequestHandler.processMergeRequest(GitLabMergeRequestHandler.java:89) at com.atlassian.jira.plugins.dvcs.sync.GitLabMergeRequestHandler.processMergeRequest(GitLabMergeRequestHandler.java:74) at com.atlassian.jira.plugins.dvcs.sync.GitLabSynchronizePullRequestMessageConsumer.onReceive(GitLabSynchronizePullRequestMessageConsumer.java:64) at com.atlassian.jira.plugins.dvcs.sync.GitLabSynchronizePullRequestMessageConsumer.onReceive(GitLabSynchronizePullRequestMessageConsumer.java:20) at com.atlassian.jira.plugins.dvcs.service.MessageExecutor$MessageRunnable.doRun(MessageExecutor.java:364) at com.atlassian.jira.plugins.dvcs.service.MessageExecutor$ReleaseTokenAndEnqueueNextMessage.run(MessageExecutor.java:274) ... Caused by: org.postgresql.util.PSQLException: ERROR: value too long for type character varying(255) ... ... 90 more
Workaround
- (Prefered:) Identify the Merge Request(s) in breach of the DB column limit of 255 characters in any of the parameters mentioned in Issue Summary
- Review all the MR pages for the failing repo with the GitLab REST API: https://<GitLab_URL>/api/v4/projects/<proj_id>/merge_requests?page=1&per_page=100
- Identify the parameter breaching 255 characters and modify it either from the UI or from the GitLab database (and refresh the cache by making a dummy comment on the MR)
- (Not recommended:) We can also modify the column restriction of the AO_E8B6CC_PULL_REQUEST table -
However, this change will not carry forward through any Jira upgrade and may cause temporary dataloss .
- links to
