• 43
    • We collect Bitbucket feedback from various sources, and we evaluate what we've collected when planning our product roadmap. To understand how this piece of feedback will be reviewed, see our Implementation of New Features Policy.

      Summary

      A public code search API for companies or startups that host Bitbucket Servers (as their own version of Bitbucket).

      Similar to Github's Search functionality and API, you can search through all the code and files using either keyterms or regexes.

      You can also use this code search to scan through all public repos (that are not your own).

      Background

      Vendor is currently hosting a bitbucket instance, and are using the Bitbucket Server APIs. They are currently looking into using Bitbucket to do code searches with regex or key terms such as the phrase "AWS Key." Looking at current SERVER REST APIs, nothing seems suitable for a code search.

      They wanted something like the Code Search API from Github - https://developer.github.com/v3/search/#search-code)

      Use Cases:
      1. A user could quickly find code sections from all of their projects/repos with one SERVER REST API call.
      2. A user could do a code search to check if any of their buckets/repos have passwords/AWS keys in them.

      Benefits:
      1. Reduce the time it is needed to do quick lookups of code in repositories.
      2. Potentially reduce the security risk of individuals (within the company) publishing passwords or AWS keys to public repos.

      Notes

      Feature request raised in DEVHELP

            [BSERV-11632] Public Code Search API for Server REST API

            As part of our automation, we would like to automate our pipeline to verify whether Bitbucket search functionality is working as expected or not using REST Api call or any other way.

            Back ground of the automation:

            We can check the search function is working or not using Bitbucket GUI and also we can verify whether Elastic search endpoint connected successfully or not from the option https://<our domain>/admin/server-settings.
            Now we are trying to verify above 2 things automatically using REST Api or any specific endpoints. SO that we can integrate those steps in our pipeline. 

             

            FYI @amol.kadam@kpn.com @hema.singh@kpn.com 

            Siladitya Bhaumik added a comment - As part of our automation, we would like to automate our pipeline to verify whether Bitbucket search functionality is working as expected or not using REST Api call or any other way. Back ground of the automation: We can check the search function is working or not using Bitbucket GUI and also we can verify whether Elastic search endpoint connected successfully or not from the option https://<our domain>/admin/server-settings. Now we are trying to verify above 2 things automatically using REST Api or any specific endpoints. SO that we can integrate those steps in our pipeline.    FYI @ amol.kadam@kpn.com @ hema.singh@kpn.com  

            niuniu_Qiu added a comment - - edited

            Any updates on providing code search API for Bitbucket server, Bitbucket cloud has this REST API already.

             

            Or public rest/search/latest/search? API

            niuniu_Qiu added a comment - - edited Any updates on providing code search API for Bitbucket server, Bitbucket cloud has this REST API already.   Or public rest/search/latest/search? API

            Cole Orton added a comment -

            Any updates on this request? 

            Does Bitbucket Server have a Search API now? Thanks! 

            Cole Orton added a comment - Any updates on this request?  Does Bitbucket Server have a Search API now? Thanks! 

            This rest api should be based on project and repo access level for e.g 

            • Repo read and higher permissions  should be able to get code search from their repos
            • Project read and higher permissions  should be able to get code search from their projects/project
            • System admin access should be able to search from complete Bitbucket instances .

            DTS Infra Admin added a comment - This rest api should be based on project and repo access level for e.g  Repo read and higher permissions  should be able to get code search from their repos Project read and higher permissions  should be able to get code search from their projects/project System admin access should be able to search from complete Bitbucket instances .

            Can you please let me know how to construct the payload for this private search end-point /rest/search/1.0/search ?

            Mandeep Kaur added a comment - Can you please let me know how to construct the payload for this private search end-point /rest/search/1.0/search ?

            Svante Gustafsson added a comment - - edited

            Hey,

            In a very large instance (20k repos) this is really an important feature to limit potential load on the cluster itself.

            Use-case: As a user I need to find all repos containing a specific metadata-file to compile a report on the code base. 

            It is easy to identify all repos containing this file with the following search query in the UI:

            system-metadata.json path:system-metadata.json 

            But if this returns 2000 locations you cannot easily work with this result.

            An alternative to programmatically do this is to scrape all repos for this file but that will spawn 20k git process which can be devastating for the service.

            There is a private search end-point: /rest/search/1.0/search that takes the search query as one parameter in its payload and returns all occurrences in json. This end-point is what we need! Please make this end-point public so that we can use it without risk that it suddenly is changed or removed.

            This will relieve a lot of performance pressure from our Bitbucket cluster when users need to find files programmatically.

            Cheers,

            // Svante

             

            Svante Gustafsson added a comment - - edited Hey, In a very large instance (20k repos) this is really an important feature to limit potential load on the cluster itself. Use-case: As a user I need to find all repos containing a specific metadata-file to compile a report on the code base.  It is easy to identify all repos containing this file with the following search query in the UI: system-metadata.json path:system-metadata.json But if this returns 2000 locations you cannot easily work with this result. An alternative to programmatically do this is to scrape all repos for this file but that will spawn 20k git process which can be devastating for the service. There is a private search end-point: /rest/search/1.0/search  that takes the search query as one parameter in its payload and returns all occurrences in json. This end-point is what we need! Please make this end-point public so that we can use it without risk that it suddenly is changed or removed. This will relieve a lot of performance pressure from our Bitbucket cluster when users need to find files programmatically. Cheers, // Svante  

            Kristy added a comment -

            Hi acalantog,

            Can I please ask you to elaborate on the request? Would Java API or REST API be more useful? What are some of the use cases?

            These details help us develop the best solution.

             

            Kristy added a comment - Hi acalantog , Can I please ask you to elaborate on the request? Would Java API or REST API be more useful? What are some of the use cases? These details help us develop the best solution.  

              Unassigned Unassigned
              acalantog Anne Calantog
              Votes:
              21 Vote for this issue
              Watchers:
              26 Start watching this issue

                Created:
                Updated: