After Bitbucket restart or forced termination of process the Data Center migration repository import is stuck at status "canceling"

Issue Summary

When using the Data Center Migration to start a repository import, the messages job status shows as "Canceling" if the application was restarted in a non-graceful manner.

Steps to Reproduce

1. Import a repository (I used the git open source project so that it would take time to import)
2. Make note of the JOB ID
3. Shutdown the application while the import progresses
   1. I tested in Docker and stopped the container.
4. Start the application

Expected Results

The job status will migrate to "Canceled" and be reflected in the messages API endpoint:

curl -s -n -X GET http://localhost:7990/rest/api/1.0/migration/imports/<Job ID>/messages | jq .

{
"id": 12,
"initiator": {
"name": "test",
"emailAddress": "test@example.com",
"id": 52,
"displayName": "Test",
"active": true,
"slug": "test",
"type": "NORMAL",
"links": {
"self": [
{
"href": "https://example.com/users/test"
}
]
}
},
"nodeId": "8c9c8db-b604-4160-8ce2-a7db5ef6b2d0",
"progress": {
"message": "The import job was canceled",
"percentage": 99
},
"startDate": 1620831320634,
"state": "CANCELED",
"type": "com.atlassian.bitbucket.migration.import",
"updateDate": 1620884430635
}

Actual Results

The job status remains "Canceling" as reflected in the messages API endpoint:

{
"id": 12,
"initiator": {
"name": "test",
"emailAddress": "test@example.com",
"id": 52,
"displayName": "Test",
"active": true,
"slug": "test",
"type": "NORMAL",
"links": {
"self": [
{
"href": "https://example.com/users/test"
}
]
}
},
"nodeId": "8c9c8db-b604-4160-8ce2-a7db5ef6b2d0",
"progress": {
"message": "The import job was canceled",
"percentage": 99
},
"startDate": 1620831320634,
"state": "CANCELING",
"type": "com.atlassian.bitbucket.migration.import",
"updateDate": 1620884430635
}

Workaround

If possible, perform a graceful shutdown of the Bitbucket service and allow time for it to complete.

When the instance is shutdown, the job is canceled; therefore, the import is no longer running. The problem occurs when the service isn't given enough time to shutdown. During shutdown, the import job is sent a cancel request. The import can only stop at certain points in its process. (see canceling and cleanup) The Bitbucket service has a 30s wait before it forcibly kills the process. If the repository import cannot transition to canceled before that time, the job messages will remain as "canceling."

Follow our canceling and cleanup to remove the impacted repository and resume the import process if needed. Reach out to Atlassian Support if you have any questions.