Issue Summary

The statuses of builds that were running while Bamboo was restarted remain in progress (indefinitely) in Bitbucket Server or Data Center even after Bamboo comes back online. This is happening because Bamboo is using an old REST endpoint to update Bitbucket Server or Data Center about the statuses of builds when it comes back online rather than the new REST endpoint.

Currently when a build is triggered Bamboo uses the new REST endpoint to inform Bitbucket Server or Data Center about it:

/rest/api/latest/projects/{projectKey}/repos/{repositorySlug}/commits/{commitID}/builds

However immediately after a restart Bamboo sends an update to Bitbucket Server or Data Center to inform about those builds using the following REST endpoint:

/rest/build-status/latest/commits/{commitID}

In order for Bamboo to use the new REST endpoint, it must first reach out to Bitbucket Server or Data Center and ask for some data. It then caches the response it gets from Bitbucket. Only after that information is cached Bamboo will use the new REST endpoint. The problem here is that Bamboo is attempting to send build notifications when it comes back online "too soon" before it is capable to cache the data it needs to be able to use the new REST endpoint – so it falls back to the old REST endpoint.

Steps to Reproduce

1. Set up Bamboo and Bitbucket Server or Data Center to communicate to each other using Application Links.
   • Add the following class in DEBUG mode to your <bamboo-install>/atlassian-bamboo/WEB-INF/classes/log4.properties file:

     log4j.logger.com.atlassian.bamboo.plugins.stash=DEBUG
     

     This way you can easily see which one of the REST endpoints Bamboo is using from the <bamboo-home>/logs/atlassian-bamboo.log files.

2. Create a repository configuration in Bamboo pointing to your Bitbucket Server or Data Center repository.
3. Create a plan that checks out code from your Bitbucket Server or Data Center repository.
4. Add a Script task to your plan with something like "sleep 180".
5. Trigger your plan and wait for it to start building.
   • You can tail your <bamboo-home>/logs/atlassian-bamboo.log file to see the endpoint Bamboo uses to information Bitbucket Server or Data Center about the build status. It should look something like this:

     2022-08-11 13:24:38,839 INFO [AtlassianEvent::0-BAM::EVENTS:pool-1-thread-25] [BitbucketServerBuildStatusUpdater] Sending build notification to Stash (build key: PROJ-PLAN-4, repository: MMM)
     2022-08-11 13:24:38,840 DEBUG [AtlassianEvent::0-BAM::EVENTS:pool-1-thread-25] [RestCallHelper] Executing Bitbucket Server request. POST https://bitbucket.net/rest/api/latest/projects/myproject/repos/MMM/commits/b13d5a18e95b539d0cd59ee7058c4272cd603deb/builds
     

6. Access the Builds page in Bitbucket Server or Data Center for the repository Bamboo is checking out code from and confirm that the build is in progress.
7. Restart Bamboo.

Expected Results

After Bamboo comes back online it should send a message using the new REST endpoint to inform the statuses of those builds that were still running when it shut down.

• You can tail your <bamboo-home>/logs/atlassian-bamboo.log file to see the endpoint Bamboo uses to Bitbucket Server or Data Center about the build status.

Actual Results

Bamboo sends a message to Bitbucket Server or Data Center using the old REST endpoint

2022-08-11 13:26:59,949 INFO [AtlassianEvent::0-BAM::EVENTS:pool-1-thread-8] [BitbucketServerBuildStatusUpdater] Sending build notification to Stash (build key: PROJ-PLAN-4, repository: MMM)
2022-08-11 13:26:59,972 DEBUG [AtlassianEvent::0-BAM::EVENTS:pool-1-thread-8] [RestCallHelper] Executing Bitbucket Server request. POST https://bitbucket.net/rest/build-status/latest/commits/b13d5a18e95b539d0cd59ee7058c4272cd603deb

At this stage you should see two entries inside the Builds page in Bitbucket Server or Data Center for the same build. At least one of them will be reporting In progress and will remain this way indefinitely.

Workaround

As a workaround you can stop the In progress build in Bitbucket Server or Data Center using the REST API.

1. First you need to get some details from that build by issuing a GET request to the following Bitbucket Server or Data Center endpoint:

   • /rest/api/1.0/projects/{projectKey}/repos/{repositorySlug}/commits/{commitId}/builds

2. Here's an example:
   Example

   curl -u username -H 'Content-Type: application/json' 'https://{bitbucketBaseURL}/rest/api/1.0/projects/{projectKey}/repos/{repositorySlug}/commits/{commitId}/builds?key={PROJ-PLAN}'
   

   You must replace the following attributes in the cURL command:

   • username
   • bitbucketBaseURL
   • projectKey
   • repositorySlug
   • commitId
   • PROJ-PLAN
3. The response will look something like this:

   {
     createdDate: 1660297090016,
     dateAdded: 1660297681833,
     description: "#4 in progress",
     key: "PROJ-PLAN",
     state: "INPROGRESS",
     name: "Project - Plan",
     updatedDate: 1660297681833,
     url: "https://bamboo.net/browse/PROJ-PLAN-4",
     buildNumber: "4",
     commit: {
       id: "2c3bdaf17c609aa0e17571656943bcc35bec6ce0",
       displayId: "2c3bdaf17c6",
       author: {
         name: "Bruno Rosa",
         emailAddress: "emailaddress@atlassian.com"
       },
       authorTimestamp: 1660290357000,
       committer: {
         name: "Bruno Rosa",
         emailAddress: "emailaddress@atlassian.com"
       },
       committerTimestamp: 1660290357000,
       message: "Test commit",
       arents: [ ]
     },
     parent: "PROJ-PLAN",
     ref: "refs/heads/main"
   }
   

4. In order to update the status of the build to either Failed or Successful you will need to use the following attributes:
   • description
   • key
   • state
   • name
   • url
   • buildNumber
   • parent
   • ref
5. Copy the values of all attributes above from the GET request made on step 2 except state and description.
6. Create a JSON file containing all of the attributes above with their values. For example:
   build.json

   {
       "description": "Build status updated via REST API",
       "key": "PROJ-PLAN",
       "state": "FAILED",
       "name": "Project - Plan",
       "url": "https://bamboo.net/browse/PROJ-PLAN-4",
       "buildNumber": "4",
       "parent": "PROJ-PLAN",
       "ref": "refs/heads/main"
   }
   

   Replace the state attribute with either SUCCESSFUL or FAILED and description with something like "Build status updated via REST API".

7. Update the build status in Bitbucket Server or Data Center by issuing a POST request to the following endpoint:

   • /rest/api/latest/projects/{projecetKey}/repos/{repositorySlug}/commits/{commitId}/builds

8. Here's an example:
   Example

   curl -u username -H 'Content-Type: application/json' -X POST 'https://{bitbucketBaseURL}/rest/api/latest/projects/{projectKey/repos/{repositorySlug}/commits/{commitId}/builds' -d @build.json
   

   You must run the cURL command from the same directory where the build.json file was saved otherwise it will fail. Replace the following attributes in the cURL command:

   • username
   • bitbucketBaseURL
   • projectKey
   • repositorySlug
   • commitId
   • PROJ-PLAN
9. Refresh the Builds page in Bitbucket Server or Data Center. Your build should now show up as successful or failed (depending on what status you chose).

Notes

You might observe the same symptom when Bitbucket Server or Data Center cannot be reached by Bamboo (even if just temporarily – whether due to a planned/ unplanned restart of Bitbucket or a network issue) and your builds finished running while Bitbucket was unavailable. This is an expected behaviour. Bamboo will attempt to send a message to inform Bitbucket Server or Data Center as soon as the build finishes running and it has processed its results. Bamboo does not attempt to re-send the build notification status when Bitbucket is available again.