Uploaded image for project: 'FishEye'
  1. FishEye
  2. FE-3461

Git repository indexing slowly

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.5.1, 2.5.5, 3.2.0
    • Fix Version/s: 3.4.0
    • Component/s: None
    • Labels:
      None

      Description

      In a huge Git repository with more than 1100 tags and 1100 branches, the indexing is very slow and there's not much we can do to improve it.

      We tried to follow this document as much as we can:
      http://confluence.atlassian.com/display/FISHEYE/Tuning+FishEye

      • Increased memory heap to -Xmx2048m, without success;
      • We're using ssh:// protocol.
      • We can't exclude paths on Git repositories.
      • We tried to use the flag --Xdisable-head-detection so it will not determine branch heads in DVCS.
        We also tried to use the flag --Xdisable-tags so it will not record tags on commits in DVCS.
        These flags would make no Git tag information available in FE.
        These flags helped a little: from 30 mins dealy to 7 or 12 mins after using the startup params, but still not there yet.
      • We tried increasing the polling interval, but it didn't help.

      The development team suggested to open this improvement request, as we cannot do much here at the moment beyond what has already been tried and suggested.

      This is a core git indexing issue in Fisheye which will need more investigation.

        Attachments

          Issue Links

            Activity

            Hide
            alan.qian Alan Qian added a comment -

            @Nick Pellow, the --Xenable-git-content-hash-resolving-heuristic flag used to help the performance for us, but at a certain point it seems to no longer do the trick. Today it is taking us half hour or more just for commits to show up in Fisheye.

            Show
            alan.qian Alan Qian added a comment - @Nick Pellow, the --Xenable-git-content-hash-resolving-heuristic flag used to help the performance for us, but at a certain point it seems to no longer do the trick. Today it is taking us half hour or more just for commits to show up in Fisheye.
            Hide
            npellow Nick Pellow [Atlassian] added a comment -

            Alan Qian - we are actively working on the solution for this.
            Would you be interested in setting up a Dev Build of FishEye at some stage in the early new year, and pointing it to your troublesome repository, to provide us with some early feedback as to how we are tracking?

            Show
            npellow Nick Pellow [Atlassian] added a comment - Alan Qian - we are actively working on the solution for this. Would you be interested in setting up a Dev Build of FishEye at some stage in the early new year, and pointing it to your troublesome repository, to provide us with some early feedback as to how we are tracking?
            Hide
            rangerer1 Ronnie added a comment -

            Nick Pellow [Atlassian] - We'd be interested in trying that out when it's ready. We have tried all of these same optimizations and have one repo where people routinely wait 30-120+ minutes for commits to be indexed and ready for review in Crucible.

            Show
            rangerer1 Ronnie added a comment - Nick Pellow [Atlassian] - We'd be interested in trying that out when it's ready. We have tried all of these same optimizations and have one repo where people routinely wait 30-120+ minutes for commits to be indexed and ready for review in Crucible.
            Hide
            jochen2 Jochen Billen added a comment -

            Hi,

            I would like try out the early release when available.

            Best Regards,
            Jochen Billen

            Show
            jochen2 Jochen Billen added a comment - Hi, I would like try out the early release when available. Best Regards, Jochen Billen
            Hide
            kschoenh Kelly Schoenhofen added a comment -

            I would as well; we have a forked copy of the Linux kernel (among other larger repos) and it is extremely slow to stay current no matter how many resources we throw at it.

            Show
            kschoenh Kelly Schoenhofen added a comment - I would as well; we have a forked copy of the Linux kernel (among other larger repos) and it is extremely slow to stay current no matter how many resources we throw at it.

              People

              • Votes:
                40 Vote for this issue
                Watchers:
                44 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:
                  Last commented:
                  2 years, 11 weeks, 3 days ago