Uploaded image for project: 'Jira Software Data Center'
  1. Jira Software Data Center
  2. JSWSERVER-14299

Receiving "Sync Failed: API Rate Limit Exceeded for <user-id> (403)" when attempting to add a new project

      Just upgraded to 0.30 due to DCON-210.

      This fixes the problem with that issue, but now I am seeing a new problem. If I hit "force sync" for one of our larger projects it always fails with "Sync Failed: API Rate Limit Exceeded for <user-id> (403)".

      See here: http://developer.github.com/v3/#rate-limiting

      I'm going to contact them to request that my instance be white listed, but I think you need to look at the X-RateLimit-Limit and X-RateLimit-Limit-Remaining headers and throttle your requests appropriately.

          Form Name

            [JSWSERVER-14299] Receiving "Sync Failed: API Rate Limit Exceeded for <user-id> (403)" when attempting to add a new project

            nino The rate limit is set by Github, once the rate limit is reached DVCS connector will cease syncing and retry again later to see if the throttling has passed.

            You can also log a support ticket at https://support.atlassian.com and our support team can investigate for you but if you have reached the Github rate limit then DVCS connector will not be able to issue further requests to Github until the throttling expires.

            Benjamin Morgan (Inactive) added a comment - nino The rate limit is set by Github, once the rate limit is reached DVCS connector will cease syncing and retry again later to see if the throttling has passed. You can also log a support ticket at https://support.atlassian.com and our support team can investigate for you but if you have reached the Github rate limit then DVCS connector will not be able to issue further requests to Github until the throttling expires.

            I'm still having issues with this, I'm using JIRA Cloud, what should I be doing?

            Nino D'Aversa added a comment - I'm still having issues with this, I'm using JIRA Cloud, what should I be doing?

            The second option is the right one, if that user has read access to the organisation repositories then they may be listed as well.

            I have just re-tested this and it will not work, I will remove the work around steps as they are incorrect.

            Benjamin Morgan (Inactive) added a comment - The second option is the right one, if that user has read access to the organisation repositories then they may be listed as well. I have just re-tested this and it will not work, I will remove the work around steps as they are incorrect.

            When I add the second account I have two options: either I specify the organisation name as the account name and then it tell's me that account is already configured or I write the user's name as the account name, and in that case I only see the repo's of the user. What am I doing wrong ?

            Christian Teijon added a comment - When I add the second account I have two options: either I specify the organisation name as the account name and then it tell's me that account is already configured or I write the user's name as the account name, and in that case I only see the repo's of the user. What am I doing wrong ?

            Only one link per account can be configured, but if you have a second account with access to those repositories you can enable a second account and sync the same repositories with that account. A setup something like this:

            Account1 - Repo1, Repo2
            Account2 - Access to Repo1

            Link both accounts, then disable Repo1 for account1 and enable it for Account2

            Benjamin Morgan (Inactive) added a comment - Only one link per account can be configured, but if you have a second account with access to those repositories you can enable a second account and sync the same repositories with that account. A setup something like this: Account1 - Repo1, Repo2 Account2 - Access to Repo1 Link both accounts, then disable Repo1 for account1 and enable it for Account2

            I'm also having this issue.
            Here it is suggested to use 2 or more accounts and split the workload between them. When I try to link a second account, since the team is the same, I get the error "Account is already integrated with JIRA", what I'm doing wrong to implement this workaround ?

            Christian Teijon added a comment - I'm also having this issue. Here it is suggested to use 2 or more accounts and split the workload between them. When I try to link a second account, since the team is the same, I get the error "Account is already integrated with JIRA", what I'm doing wrong to implement this workaround ?

            This will be resolved in an upcoming release of DVCS connector.

            In the current behaviour you will see an error but the sync will resume again at a regular interval so it will eventually complete.

            The fix that we are shipping will show better errors when the rate limit is reached and will handle the case where you cannot link the account because the rate limit for validation has been reached.

            You will still be limited to 5000 requests an hour as that is the rate limit imposed by Github

            Benjamin Morgan (Inactive) added a comment - This will be resolved in an upcoming release of DVCS connector. In the current behaviour you will see an error but the sync will resume again at a regular interval so it will eventually complete. The fix that we are shipping will show better errors when the rate limit is reached and will handle the case where you cannot link the account because the rate limit for validation has been reached. You will still be limited to 5000 requests an hour as that is the rate limit imposed by Github

            I've added some details in https://jira.atlassian.com/browse/DCON-489?focusedCommentId=724290&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-724290 but from some testing it seems that the webhooks are still being installed as the 'repo' scope still allows hook creation.

            We are looking into an improvement around better reporting of rate limit errors which should alleviate this problem.

            Benjamin Morgan (Inactive) added a comment - I've added some details in https://jira.atlassian.com/browse/DCON-489?focusedCommentId=724290&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-724290 but from some testing it seems that the webhooks are still being installed as the 'repo' scope still allows hook creation. We are looking into an improvement around better reporting of rate limit errors which should alleviate this problem.

            Geoff Low added a comment -

            See also DCON-489 for comments (and prospective code changes). I think much of the problem stems from a change to GitHub Authorisation Scopes so that Jira can no longer install webhooks on repositories so we have to rely on hourly or twice-hourly sync's to work and then we blow the limit. Also, the 403 error is special; "After detecting several requests with invalid credentials within a short period, the API will temporarily reject all authentication attempts for that user (including ones with valid credentials) with 403 Forbidden:" (https://developer.github.com/v3/#authentication)

            Geoff Low added a comment - See also DCON-489 for comments (and prospective code changes). I think much of the problem stems from a change to GitHub Authorisation Scopes so that Jira can no longer install webhooks on repositories so we have to rely on hourly or twice-hourly sync's to work and then we blow the limit. Also, the 403 error is special; "After detecting several requests with invalid credentials within a short period, the API will temporarily reject all authentication attempts for that user (including ones with valid credentials) with 403 Forbidden:" ( https://developer.github.com/v3/#authentication )

            rsomma It seems unlikely that the performance of Agile will be impacted on by the sync from dvcs connector. Once the rate limit is reached it will halt the sync for that repository and attempt to sync again on the next schedule sync or webhook. If you would like us to investigate further please log a ticket at https://support.atlassian.com and the support team can help diagnose what is happening.

            Benjamin Morgan (Inactive) added a comment - rsomma It seems unlikely that the performance of Agile will be impacted on by the sync from dvcs connector. Once the rate limit is reached it will halt the sync for that repository and attempt to sync again on the next schedule sync or webhook. If you would like us to investigate further please log a ticket at https://support.atlassian.com and the support team can help diagnose what is happening.

              Unassigned Unassigned
              cb4c63d3183a Rich Seddon
              Affected customers:
              16 This affects my team
              Watchers:
              41 Start watching this issue

                Created:
                Updated:
                Resolved: