• Icon: Suggestion Suggestion
    • Resolution: Unresolved
    • None
    • Search
    • None
    • 25
    • 9
    • We collect Bitbucket feedback from various sources, and we evaluate what we've collected when planning our product roadmap. To understand how this piece of feedback will be reviewed, see our Implementation of New Features Policy.

      If there is a way to indicate the current status of the index.. or the queue for documents/commits waiting to be indexed.

      I know it should be magically and instantaneous , but with 2000+ developers and 5000+ repos i need a little more to make sure i know what to say to developers when they say search results are not reflecting their commits.

      Bonus points if there is a way (admin or project admin) to 'touch' a repo, to force it to be reindexed

          Form Name

            [BSERV-8860] Show search index status

            With the new awesome Repository Management page available it would be fairly easy to add an option on the Repository action button for triggering an ES reindex for the repo. 

            Even though there is an end-point available as pointed out above, collecting these admin tasks into one view is a very good practice.

            I would also second the idea to have a panel showing status in real time, especially valuable when you have to do a complete reindexing. Preferably showing some time to completion estimates as well. In our case a complete reindex takes three hours so it would be great to see some sort of progress

            Cheers,

            // Svante

            Svante Gustafsson added a comment - With the new awesome Repository Management page available it would be fairly easy to add an option on the Repository action button for triggering an ES reindex for the repo.  Even though there is an end-point available as pointed out above, collecting these admin tasks into one view is a very good practice. I would also second the idea to have a panel showing status in real time, especially valuable when you have to do a complete reindexing. Preferably showing some time to completion estimates as well. In our case a complete reindex takes three hours so it would be great to see some sort of progress Cheers, // Svante

            Hi cleveland.dana,

            Just wanted to check in, have you utilised the indexing rest endpoint? We have the current indexing queues and status available at /rest/indexing/latest/status. From what you see, does this fulfil your requirements.

            Edit: To refer to the last part about "touching" a repository. You can force a sync of a repository via

            /rest/indexing/latest/projects/<project>/repos/<repo>/sync

            Cheers

            Paul

            Paul Thompson (Inactive) added a comment - - edited Hi cleveland.dana , Just wanted to check in, have you utilised the indexing rest endpoint? We have the current indexing queues and status available at /rest/indexing/latest/status . From what you see, does this fulfil your requirements. Edit: To refer to the last part about "touching" a repository. You can force a sync of a repository via /rest/indexing/latest/projects/<project>/repos/<repo>/sync Cheers Paul

            Brent P added a comment - - edited

            Due to the way events are processed, we're leaning toward a JMX-based metric showing two things:

            • Creation timestamp of last processed index event
            • Length of the indexing queue

            With those two pieces of data, an admin should be able to see if the server is keeping on top of indexing (queue size should be 0) and if event processing is working (if the index event time is changing).

            Brent P added a comment - - edited Due to the way events are processed, we're leaning toward a JMX-based metric showing two things: Creation timestamp of last processed index event Length of the indexing queue With those two pieces of data, an admin should be able to see if the server is keeping on top of indexing (queue size should be 0) and if event processing is working (if the index event time is changing).

            DanaC added a comment -

            Sorry I was out for few days..

            So i have 2 things..

            1. Developers have asked, or questioned why they can't see something they have commit say on the previous day in the past, and i think it had had to do with our server not having enough resources to process all the daily commits/pushes we get.. So I had gotten that on a number of occasions

            2. The index which i can see is elastic/lucene based had gotten corrupt in the past and had forced me to delete the whole directory to force it to rebuild it.. That full rebuild has been really a performance hit on the server, so i have wanted to help explain or give estimates to developers where we are on the rebuild.

            In general i had seen a real performance hit, mostly on indexing, and occasionally on searching.. Again this was based on the original 3rd party plugin that as far as i can tell your implementation is based around.. I had local changes to the plugin in the past so i got to know the layout, architecture of how it work, and i had build similar indexers for source code in the past for openGrok.

            DanaC added a comment - Sorry I was out for few days.. So i have 2 things.. 1. Developers have asked, or questioned why they can't see something they have commit say on the previous day in the past, and i think it had had to do with our server not having enough resources to process all the daily commits/pushes we get.. So I had gotten that on a number of occasions 2. The index which i can see is elastic/lucene based had gotten corrupt in the past and had forced me to delete the whole directory to force it to rebuild it.. That full rebuild has been really a performance hit on the server, so i have wanted to help explain or give estimates to developers where we are on the rebuild. In general i had seen a real performance hit, mostly on indexing, and occasionally on searching.. Again this was based on the original 3rd party plugin that as far as i can tell your implementation is based around.. I had local changes to the plugin in the past so i got to know the layout, architecture of how it work, and i had build similar indexers for source code in the past for openGrok.

            Any thoughts on my previous comment, ultimav?

            Roger Barnes (Inactive) added a comment - Any thoughts on my previous comment, ultimav ?

            Hi Dana,

            Thanks for the suggestion!

            Are you hearing questions from developers relating to the index status? Our aim is to make everything fast and automatic enough for the vast majority of use cases that active monitoring or intervention would be unnecessary.

            We're quite open to addressing the problem if there is one, and are keen to understand the dynamics at play.

            Roger Barnes (Inactive) added a comment - Hi Dana, Thanks for the suggestion! Are you hearing questions from developers relating to the index status? Our aim is to make everything fast and automatic enough for the vast majority of use cases that active monitoring or intervention would be unnecessary. We're quite open to addressing the problem if there is one, and are keen to understand the dynamics at play.

              Unassigned Unassigned
              49f552c5bd11 Dana Cleveland
              Votes:
              30 Vote for this issue
              Watchers:
              26 Start watching this issue

                Created:
                Updated: