When we do a search that includes a comment paramater, we do a search on the comment index, generate a list of keys and then pass that in to the normal search. When we do the search on the comment index we should use a FieldableDocumentHitCollector that only sucks in the issue id, this should improve things as we wont need to pull the body of the comment into the document.
Though, it may not be that simple as we don;t actually use a HitCollector for this. We get the hits and iterate over them manually - might want to use a hit collector....
See - LuceneQueryCreator.getCommentsFilter(Hits hits)