Uploaded image for project: 'Atlassian Intelligence'
  1. Atlassian Intelligence
  2. AI-648

Make one big searchableText term in Lucene containing all searchable text for that entity

    • Our product teams collect and evaluate feedback from a number of different sources. To learn more about how we use customer feedback in the planning process, check out our new feature policy.

      NOTE: This suggestion is for Confluence Cloud. Using Confluence Server? See the corresponding suggestion.

      Right now, our searches are complicated by the fact that each field on the object goes into a different term in the index.

      The advantages of doing this are that we can weight titles higher, for example.

      The disadvantage is that if someone searches for "foo AND bar", you'll only get a match if foo and bar are in the same field: an email addressed to Charles, with beer in the message body won't match "charles AND beer"

      I've fixed this for the simplest case - title of a CEO for CONF-3426, but we need a more general solution where everything we want to be searched on gets put in a single field.

            [AI-648] Make one big searchableText term in Lucene containing all searchable text for that entity

            Given that this issue has been open for several years with no votes and no traction, marking as "Won't Fix".

            Sherif Mansour added a comment - Given that this issue has been open for several years with no votes and no traction, marking as "Won't Fix".

            Alexey Alexey added a comment - - edited

            Compose all searchable fields in one "total" is generally a good idea, but anyway it makes sense to build more complex boolean Lucene query with weights for different fields, for example

            BooleanQuery resultQuery = new BooleanQuery();
            resultQuery.add(new MatchAllDocsQuery(), Occur.MUST);
            
            fieldsToSearch = new String[]{"title", "total"};
            boostOptions = new float[]{1.0000f, 0.0000f};
            fieldOptions = new BooleanClause.Occur[]{Occur.SHOULD, Occur.MUST};
            
            for (int i = 0; i < fieldsToSearch.length; i++) {
                Query fieldQuery;
                try {
                    SolrQueryParser queryParser = new SolrQueryParser(schema, fieldsToSearch[i]);
                    queryParser.setDefaultOperator(SolrQueryParser.AND_OPERATOR);
                    fieldQuery = queryParser.parse(keywords);
                    fieldQuery.setBoost(boostOptions[i]);
                    resultQuery.add(fieldQuery, fieldOptions[i]);
                }
                catch (Exception e) {
                    log.debug("Error during query parsing\n" + e.getStackTrace());
                }
            }
            

            Probably you can also check standard MultiFieldQueryParser
            http://lucene.apache.org/java/2_3_2/api/org/apache/lucene/queryParser/MultiFieldQueryParser.html

            Alexey Alexey added a comment - - edited Compose all searchable fields in one "total" is generally a good idea, but anyway it makes sense to build more complex boolean Lucene query with weights for different fields, for example BooleanQuery resultQuery = new BooleanQuery(); resultQuery.add( new MatchAllDocsQuery(), Occur.MUST); fieldsToSearch = new String []{ "title" , "total" }; boostOptions = new float []{1.0000f, 0.0000f}; fieldOptions = new BooleanClause.Occur[]{Occur.SHOULD, Occur.MUST}; for ( int i = 0; i < fieldsToSearch.length; i++) { Query fieldQuery; try { SolrQueryParser queryParser = new SolrQueryParser(schema, fieldsToSearch[i]); queryParser.setDefaultOperator(SolrQueryParser.AND_OPERATOR); fieldQuery = queryParser.parse(keywords); fieldQuery.setBoost(boostOptions[i]); resultQuery.add(fieldQuery, fieldOptions[i]); } catch (Exception e) { log.debug( "Error during query parsing\n" + e.getStackTrace()); } } Probably you can also check standard MultiFieldQueryParser http://lucene.apache.org/java/2_3_2/api/org/apache/lucene/queryParser/MultiFieldQueryParser.html

              Unassigned Unassigned
              cmiller@atlassian.com Charles Miller (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated:
                Resolved: