-
Bug
-
Resolution: Fixed
-
Medium
-
4.1.2
-
jira enterprise standalone 4.1.2
mysql 5.1.150
Apache tomcat 6.0.20
-
4.01
-
Please resurrect IssueIdFilter rather than using a BooleanQuery of individual TermQuery's of issue id's inside CommentClauseQueryFactory.
Searching for Comment contains 'bug' on Jira 3.13 returns 53000 issues and works fine.
Searching for comment ~ "bug" on our Jira 4.1.2 test server that has the same data as the Jira 3.13 server returns no results and causes
The 'comment ~ "bug"' clause in your search matches too many results. If a clause returns too many results, the entire search will fail. Please refine or remove the clause and try the search again.
This is not acceptable.
Diving into the code with remote debug running, I see that the problem is that a search is being made on the comment index and then each hit from this is converted to an issue_id TermQuery. Each of these TermQuery's are collected in one massive BooleanQuery. That query with over 65000 term queries is then run against the issue index. Since the word bug appears in more than 65000 comments in our Jira, this causes the TooManyClauses exception.
In Jira 3 the same query would search the comment index and then the hits were converted to an IssueIdFilter which was wrapped with a FilteredQuery.
This is faster and doesn't cause the TooManyClauses exception.
Please resurrect IssueIdFilter rather than using a BooleanQuery of individual TermQuery's of issue id's inside CommentClauseQueryFactory.
Another improvement would be to collect the issue id's in a set since issues can have more than one comment.
With the improvement that I suggested in JRA-22448 to speed up comment searching by using a HitCollector rather than looping over the hits returned from the comment index, our CommentClauseQueryFactory.generateIssueIdQueryFromCommentQuery now looks like this
QueryFactoryResult generateIssueIdQueryFromCommentQuery(final Query commentIndexQuery, final QueryCreationContext creationContext) { try { // Now we need to execute the search and build up a new query that enumerates the issue id's found by the comment search final Searcher commentSearcher = searchProviderFactory.getSearcher(SearchProviderFactory.COMMENT_INDEX); final Filter filter = getPermissionsFilter(creationContext.isSecurityOverriden(), creationContext.getUser()); final Collection<String> ids = new HashSet<String>(10000); final HitCollector hitCollector = new DocumentHitCollector(commentSearcher) { @Override public void collect(Document doc) { ids.add(doc.get(DocumentConstants.ISSUE_ID)); } }; // JRA-19306 - lets restrict this search by the permissions filter so we can try to avoid the MAX_CLAUSES limit // Perform the search against the comment index if(filter == null) { commentSearcher.search(commentIndexQuery, hitCollector); } else { commentSearcher.search(commentIndexQuery, filter, hitCollector); } // Create a query that will be run against the issue index with the issue ids from the search hits return new QueryFactoryResult(new ConstantScoreQuery(new IssueIdFilter(ids))); } catch(final IOException e) { log.error("Unable to search the comment index - returning false result.", e); return QueryFactoryResult.createFalseResult(); } }
- relates to
-
JRASERVER-22448 Comment Query Performance: Please Change CommentClauseQueryFactory.generateIssueIdFromCommentQuery from for loop to HitCollector
- Closed
-
JRASERVER-64385 Issue search with worklog clause throws an error "An unknown error occurred while trying to perform a search."
- Gathering Impact
-
JRASERVER-22256 Performance Issue with JQL functions
- Closed