Uploaded image for project: 'Jira Software Data Center'
  1. Jira Software Data Center
  2. JSWSERVER-14268

List of repositories cannot be synchronized when parent of forked repositories is deleted

      Summary

      If the parent repository of forked repository was deleted and thus the forked repository was marked as disabled in GitHub, it is still returned in the list of repositories but fetching the detailed information (to find out the fork repository information) causes 404 NOT FOUND error. This also occurs if the repository was marked as deleted in Github but never removed from the sync list in DVCS.

      You'll see the following stack trace in the logs:

       2016-10-05 18:22:57,501 Caesium-1-2 DEBUG ServiceRunner [c.a.j.p.dvcs.service.RepositoryServiceImpl] Synchronizing list of repositories
       2016-10-05 18:22:57,511 Caesium-1-2 DEBUG ServiceRunner [c.a.j.p.d.d.impl.transform.RepositoryTransformer] Repository transformation: [AO_E8B6CC_REPOSITORY_MAPPING \{ID = 15}] 
       2016-10-05 18:22:57,535 Caesium-1-2 ERROR ServiceRunner [c.a.scheduler.core.JobLauncher] Scheduled job with ID 'com.atlassian.jira.plugins.dvcs.scheduler.DvcsScheduler:job' failed
       com.atlassian.jira.plugins.dvcs.exception.SourceControlException: Error retrieving list of repositories
       at com.atlassian.jira.plugins.dvcs.spi.github.GithubCommunicator.getRepositories(GithubCommunicator.java:252)
       at com.atlassian.jira.plugins.dvcs.service.remote.CachingCommunicator.getRepositories(CachingCommunicator.java:232)
       at com.atlassian.jira.plugins.dvcs.service.RepositoryServiceImpl.syncRepositoryList(RepositoryServiceImpl.java:219)
       at com.atlassian.jira.plugins.dvcs.service.RepositoryServiceImpl.syncRepositoryList(RepositoryServiceImpl.java:264)
       at com.atlassian.jira.plugins.dvcs.scheduler.DvcsSchedulerJobRunner.syncOrganizations(DvcsSchedulerJobRunner.java:67)
       at com.atlassian.jira.plugins.dvcs.scheduler.DvcsSchedulerJobRunner.runJob(DvcsSchedulerJobRunner.java:55)
       at com.atlassian.scheduler.core.JobLauncher.runJob(JobLauncher.java:153)
       at com.atlassian.scheduler.core.JobLauncher.launchAndBuildResponse(JobLauncher.java:118)
       at com.atlassian.scheduler.core.JobLauncher.launch(JobLauncher.java:97)
       at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService.launchJob(CaesiumSchedulerService.java:401)
       at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService.executeClusteredJob(CaesiumSchedulerService.java:396)
       at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService.executeQueuedJob(CaesiumSchedulerService.java:349)
       at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService$1.consume(CaesiumSchedulerService.java:255)
       at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService$1.consume(CaesiumSchedulerService.java:252)
       at com.atlassian.scheduler.caesium.impl.SchedulerQueueWorker.executeJob(SchedulerQueueWorker.java:65)
       at com.atlassian.scheduler.caesium.impl.SchedulerQueueWorker.executeNextJob(SchedulerQueueWorker.java:59)
       at com.atlassian.scheduler.caesium.impl.SchedulerQueueWorker.run(SchedulerQueueWorker.java:34)
       at java.lang.Thread.run(Thread.java:745)
       Caused by: org.eclipse.egit.github.core.client.RequestException: Not Found (404)
       at org.eclipse.egit.github.core.client.GitHubClient.createException(GitHubClient.java:552)
       at org.eclipse.egit.github.core.client.GitHubClient.get(GitHubClient.java:740)
       at org.eclipse.egit.github.core.client.PageIterator.next(PageIterator.java:173)
       at org.eclipse.egit.github.core.service.GitHubService.getAll(GitHubService.java:151)
       at org.eclipse.egit.github.core.service.RepositoryService.getRepositories(RepositoryService.java:242)
       ... 17 more
      
      

      and this causes the scheduler for DVCS to break so any subsequent syncs of any repos fail

      Steps to Diagnose

      1. Ensure debug logging is enabled for the following package com.atlassian.jira.plugins.dvcs
      2. Delete the OAuth keys and recreate them using an owner
      3. Mark the logs
      4. Note the start time
      5. Re-add the repository with a user that owns/admins the repository
      6. Attempt to add a new repository
      7. Grab both Github Enterprise / JIRA Server Logs
      8. Compare 404s from web-logs/github.log to DVCS Error thrown in logs to find offending repo

      Environment

      • DVCS 4.0.11

      Workaround

      Delete forked repository on GitHub, the parent repository doesn't exist anymore, there is no use of such repository, even GitHub treats the repository as disabled. If the repository has already been removed, ensure that DVCS does not make any calls to deleted repository.

          Form Name

            [JSWSERVER-14268] List of repositories cannot be synchronized when parent of forked repositories is deleted

            David Newhall II added a comment - - edited

            Can we fix this now? Twitch will thank you.

            David Newhall II added a comment - - edited Can we fix this now? Twitch will thank you.

            BrianL added a comment -

            Is there any update on this issue. I got a similar error here. The dvcs scheduler is not working. The new repositories can't be added automatically. But when manually refresh the account, the new repo can be added.

            BrianL added a comment - Is there any update on this issue. I got a similar error here. The dvcs scheduler is not working. The new repositories can't be added automatically. But when manually refresh the account, the new repo can be added.

            Many thanks for raising this bug. Since there has not been any customer activity on this issue in over a year, we’re closing it as Timed out.
            If you feel that the bug is still relevant to your needs, please leave a comment and we'll reassess it.

            Kind regards,
            Dong Hoang
            JIRA Software

            Hoang Dong (Inactive) added a comment - Many thanks for raising this bug. Since there has not been any customer activity on this issue in over a year, we’re closing it as Timed out. If you feel that the bug is still relevant to your needs, please leave a comment and we'll reassess it. Kind regards, Dong Hoang JIRA Software

            VikasBasra added a comment -

            Any plan on when this issue will be fixed?

            VikasBasra added a comment - Any plan on when this issue will be fixed?

            While getting the list of repositories, one of them returns 404 and the code loop exits without returning any repository.
            So there is one repository that is having a problem. Unfortunately at the moment we don't have enough log in the code to point out which one it is.

            We know that the URL is formed: https://api.github.com/repos/_repo_
            One of them returns 404 so navigating to each one of them on the browser, the problematic repo can be found.

            Mauro Badii (Inactive) added a comment - While getting the list of repositories, one of them returns 404 and the code loop exits without returning any repository. So there is one repository that is having a problem. Unfortunately at the moment we don't have enough log in the code to point out which one it is. We know that the URL is formed: https://api.github.com/repos/_repo_ One of them returns 404 so navigating to each one of them on the browser, the problematic repo can be found.

              cvanes Chris van Es (Inactive)
              mstencel MiroslavA
              Affected customers:
              25 This affects my team
              Watchers:
              38 Start watching this issue

                Created:
                Updated:
                Resolved: