Uploaded image for project: 'Confluence Data Center'
  1. Confluence Data Center
  2. CONFSERVER-14910

Provide ability to override Lucene tokenisation and stemming and search for exact text (literal search)

XMLWordPrintable

    • 126
    • 26
    • We collect Confluence feedback from various sources, and we evaluate what we've collected when planning our product roadmap. To understand how this piece of feedback will be reviewed, see our Implementation of New Features Policy.

      NOTE: This suggestion is for Confluence Server. Using Confluence Cloud? See the corresponding suggestion.

      We would like to able to search for exact text, and not have Lucene tokenise, remove stop words, or do any stemming.

      We partly use Confluence as a knowledge base for our own software product, which includes lots of system codes, acrnoyms etc. The specific use case that prompted this was a user wanting to search for the text "S/D". Lucene strips off the 'S', and the '/', leaving the search to just look for 'd'. Not particularly useful.

      It would be brilliant if we could enter a query somehow that overrode this.

      Raised following support issue CSP-29584

      Workaround

      We've implemented the functionality requested, but are awaiting a point where we can have a reindex performed as part of upgrading Confluence in the future. Until that point, we cannot turn the feature on by default. In the interim you can follow the below steps to enable the functionality in your instance.

      You will need the following versions of Confluence

      • Confluence 8.0.0+ to do exact searching on the titles of pages and blogs
      • Confluence 8.6.0+ (not yet released at time of writing) to do exact searching on the content of pages and blogs

      To enable the functionality, you will need to

      1. Be running the above listed versions of Confluence
      2. Enable the following dark feature in Confluence confluence.search.improvements.exact
      3. Rebuild the Search Index Note: This will need to be done again when crossing either the Confluence 8.0.0 or 8.6.0 line
      4. Run a search using "quotation marks" to indicate an exact search is desired

      Be aware that this change will have an impact on the size of the index in your instance, and the time taken to index. From our testing, we saw a 10%-15% increase in both index size and time to index. This time will depend heavily on the shape of the data in your instance.

        1. i_forgot.jpg
          i_forgot.jpg
          23 kB
        2. image-2016-12-26-10-54-27-948.png
          148 kB
        3. image-2016-12-26-10-56-53-084.png
          148 kB

            [CONFSERVER-14910] Provide ability to override Lucene tokenisation and stemming and search for exact text (literal search)

            Lawrence Rubanka added a comment -

            Has this "suggestion" really been open for more than 15 years?

            Yikes!  At least I'm not paying the bill.

            Lawrence Rubanka added a comment - Has this "suggestion" really been open for more than 15 years? Yikes!  At least I'm not paying the bill.

            Adam Bartlett added a comment -

            Typical.

            It takes so long for Atlassian to implement this (15 years!) that by the time they do, my employer has jumped ship to an alternative product. 

            Adam Bartlett added a comment - Typical. It takes so long for Atlassian to implement this (15 years!) that by the time they do, my employer has jumped ship to an alternative product. 

            Carla Ann K. Rowland added a comment - - edited

            This is a major problem for our Confluence Data center 8.7.x  I cannot separate kb-how-to-articles from kb-troubleshooting-articles at ALL. Also if the page has no label then it shows up in the search results on the portal–regardless of the permission of the page. NOT good for my gov't client.

            Carla Ann K. Rowland added a comment - - edited This is a major problem for our Confluence Data center 8.7.x  I cannot separate kb-how-to-articles from kb-troubleshooting-articles at ALL. Also if the page has no label then it shows up in the search results on the portal–regardless of the permission of the page. NOT good for my gov't client.

            James Ponting added a comment -

            Hey c09381d4440b,

            It does indeed. Let us know how you get on

            Thanks,
            James Ponting
            Engineering Manager - Confluence Data Center

            James Ponting added a comment - Hey c09381d4440b , It does indeed. Let us know how you get on Thanks, James Ponting Engineering Manager - Confluence Data Center

            Angela L. added a comment -

            Does this also affect Data Center? 

            Angela L. added a comment - Does this also affect Data Center? 

            James Ponting added a comment - - edited

            Hi All,

            A quick update on this one.

            We've been working on introducing this functionality for a while, and have introduced it in stages with the intent of enabling it all in one piece. The work requires reindexing Confluence, and so we were hoping to release all of the feature alongside some planned work that also required a reindex to complete. Unfortunately that work has fallen off the roadmap, leaving us in a bit of an uncomfortable situation where we have an improvement to ship, but it requires manual intervention to enable. So here's how you do it manually as we work out how we'll roll this out for everyone.

            You will need the following versions of Confluence

            • Confluence 8.0.0+ to do exact searching on the titles of pages and blogs
            • Confluence 8.6.0+ (not yet released at time of writing) to do exact searching on the content of pages and blogs

            To enable the functionality, you will need to

            1. Be running the above listed versions of Confluence
            2. Enable the following dark feature in Confluence confluence.search.improvements.exact
            3. Rebuild the Search Index Note: This will need to be done again when crossing either the Confluence 8.0.0 or 8.6.0 line
            4. Run a search using "quotation marks" to indicate an exact search is desired

            Be aware that this change will have an impact on the size of the index in your instance, and the time taken to index. From our testing, we saw a 10%-15% increase in both index size and time to index. This time will depend heavily on the shape of the data in your instance.

            Hopefully this will be of some help to you all. I'll update the ticket when we properly release the feature to the broader public (i.e. without the need for a dark feature), but this just depends on when we can organise an upgrade reindex.

            Thanks,
            James Ponting
            Engineering Manager - Confluence Data Center

            James Ponting added a comment - - edited Hi All, A quick update on this one. We've been working on introducing this functionality for a while, and have introduced it in stages with the intent of enabling it all in one piece. The work requires reindexing Confluence, and so we were hoping to release all of the feature alongside some planned work that also required a reindex to complete. Unfortunately that work has fallen off the roadmap, leaving us in a bit of an uncomfortable situation where we have an improvement to ship, but it requires manual intervention to enable. So here's how you do it manually as we work out how we'll roll this out for everyone. You will need the following versions of Confluence Confluence 8.0.0+ to do exact searching on the titles of pages and blogs Confluence 8.6.0+ (not yet released at time of writing) to do exact searching on the content of pages and blogs To enable the functionality, you will need to Be running the above listed versions of Confluence Enable the following dark feature in Confluence confluence.search.improvements.exact Rebuild the Search Index Note: This will need to be done again when crossing either the Confluence 8.0.0 or 8.6.0 line Run a search using "quotation marks" to indicate an exact search is desired Be aware that this change will have an impact on the size of the index in your instance, and the time taken to index. From our testing, we saw a 10%-15% increase in both index size and time to index. This time will depend heavily on the shape of the data in your instance. Hopefully this will be of some help to you all. I'll update the ticket when we properly release the feature to the broader public (i.e. without the need for a dark feature), but this just depends on when we can organise an upgrade reindex. Thanks, James Ponting Engineering Manager - Confluence Data Center

            Mark Curtis added a comment -

            This impoverished searching ability makes the tool less desirable by teams that use more "primitive" tools that support it.  Harder to sell.  Seems like this should be slam-dunk.

            Mark Curtis added a comment - This impoverished searching ability makes the tool less desirable by teams that use more "primitive" tools that support it.  Harder to sell.  Seems like this should be slam-dunk.

            Colin Green added a comment - - edited

            Of course. Please send me an email at colingreenlabovi@gmail.com

            Colin Green added a comment - - edited Of course. Please send me an email at colingreenlabovi@gmail.com

            Chari, Ambarish added a comment -

            Hey Colin, would love to hear more about this ChatGPT feature plugin. My organization is looking for an AI Chatbot like ChatGpt to be added to the confluence search so that it gives the user specific results pertaining to the search and not irrelevant stuff. So, I don't know whether Atlassian is working on that or not but I need to know. Can we connect if it is not too much trouble?

            Chari, Ambarish added a comment - Hey Colin, would love to hear more about this ChatGPT feature plugin. My organization is looking for an AI Chatbot like ChatGpt to be added to the confluence search so that it gives the user specific results pertaining to the search and not irrelevant stuff. So, I don't know whether Atlassian is working on that or not but I need to know. Can we connect if it is not too much trouble?

            Colin Green added a comment -

            We really look for a solution for this. Seems like there is a new plugin using openAI chatGPT like, is anyone used it? I was not sure if that's might help I have installed it on different account for testing, please let me know what you think..

            Colin Green added a comment - We really look for a solution for this. Seems like there is a new plugin using openAI chatGPT like, is anyone used it? I was not sure if that's might help I have installed it on different account for testing, please let me know what you think..

              rlau@atlassian.com Richard Lau
              f29e75fd9379 Neil Arrowsmith
              Votes:
              899 Vote for this issue
              Watchers:
              417 Start watching this issue

                Created:
                Updated: