Thank for the good news!!

Could you please let us know which files we need change in Jira 5.1 to work out this issue.

This is HUGE!!! Thanks for the good news!

This is fixed and released to our onDemand releases. It will also be included in our upcoming 5.2 release

Mairead
PM JIRA

Yes, and while people like Luke and us are now content with our modified instances and thus cease prodding Atlassian to finally fix this silly problem, everyone else still has to suffer the bad search behavior and the resulting performance/stability problems - not fair

That comment was from a user taking matters into their own hands and modifying the source code for their own JIRA instance. This issue has still not been addressed by Atlassian.

I'm confused. In 4.4.3, it appears the search (as launched from the "quick search" box) still uses OR by default, despite this comment that seems to indicate otherwise..

Searching for "A B C" results in JQL:

summary ~ "A B C" OR description ~ "A B C" OR comment ~ "A B C"

and appears to return results that don't contain C at all, in any field.

In my opinion, the resulting JQL should be:

(summary ~ "A" OR description ~ "A" OR comment ~ "A") 
AND (summary ~ "B" OR description ~ "B" OR comment ~ "B") 
AND (summary ~ "C" OR description ~ "C" OR comment ~ "C")

This would mean all terms have to appear somewhere.

Forget the admin configuration... "AND" is correct and "OR" is wrong.

Volker, can you copy the information from the support ticket to this ticket, or at least list the file names where these changes were made?

In JIRA 4, all we needed to do was to change com.atlassian.jira.jql.query.LikeQueryFactory a little.

In JIRA 4, JQL uses the LIKE search on all text fields and only on text fields which allows us to easily limit the AND operation to selected fields. We added the following lines of code to the getQueries method which parses the query input and sets the operand:

if(fieldName.equalsIgnoreCase("summary") || fieldName.equalsIgnoreCase("description") ||
                fieldName.equalsIgnoreCase("body") || fieldName.equalsIgnoreCase("environment") )
        {
            parser.setDefaultOperator(QueryParser.AND_OPERATOR);
        }

So we limited the AND operand as the default to only the following fields:

1. Summary
2. Description
3. Environment
4. Comments

Any other text fields will still be using the default operand, which is OR, and the combination between different search conditions is always AND unless you use the advanced search to change that.

We now have much more precise and useful search results. We no longer have people overloading the server with their imprecise searches. And if someone happens to enter a very generic search term, he no longer gets the message that the number of search results exceeded the configured limit, but the other not-so-generic search terms limit his search results as they well should.

We have also provided the details to Atlassian in our support ticket with them, in the hope that they will implement the change or even better make this an admin-configurable option.

Volker, you bring up a very good point. Two keywords searched with an "OR" combination typically take much longer to execute than the same two words with an "AND". Resolving this bug would actually have two really important benefits. Search performance would be vastly improved and search results would be much more accurate.

Atlassian, please up the priority on this. It is a trivial change for a huge win.

So there are thousands of other things that are keeping you busy - I understand that. But in the meantime every one of your users is wasting time due to bad searching, everyone is suffering from too-slow search performance in JIRA (which also impacts its performance elsewhere), and the amount of time lost as consequence is quite likely the biggest drawback JIRA has. So perhaps prioritizing this ticket should have been among those thousand other things? Most likely the first ten of them?

Ok, thanks Anton. I certainly wasn't looking for a guarantee. Just a sanity check on whether I'm proposing something very stupid.

I'll probably do some more testing at our end and try deploying it if it looks ok. I guess we can always roll the change back if it introduces a problem.

Quite honestly we did not do this this yet as there are thousands of other things that are keeping us busy.

From on top of my head the changes you are making should work. However, I cannot guarantee anything.

Ok, so I went and tried out my idea on a test server. I took all instances of QueryParser and called setDefaultOperator on them before parsing. It didn't work immediately because Jira also uses MultiFieldQueryParser in one place. This has a static parse method which doesn't seem to allow the default operator to be set. However I got the source to this function from Lucene, it's a tiny function. It instantiates some QueryParser instances itself, so I pasted a copy of the code inside the Jira source and modified these instances to call setDefaultOperator.

After that, it appears that searches work with AND as the default conjunction.

Part of me's tempted to make these changes to the source in our installation. But the sensible part of me is (a) concerned that I haven't tested thoroughly enough, and (b) suspicious that if it was really this easy, Atlassian could have made this a configurable option a long time ago.

I'd love to get an opinion from an Atlassian developer that understands the search code better. Is anyone listening? Are there any obvious pitfalls of this?

One big aspect that has been totally overlooked here so far is the load produced by these searches. When I search for an issue with three or four keywords, I expect no more than a handful of results, with a search executed in at most a few seconds, and a page that displays quickly, displays all results, and puts no unreasonable load on the server that will make others suffer. And with AND as default search operator I know that I will improve all of the above by adding additional keywords, i.e. I will get more precise results, and the loading time and server load will go down. This is what people do all the time, and what they expect from the system - that adding search terms will make their search experience better.

Enter JIRA. Adding ONE bad search term in JIRA easily makes your number of results skyrocket, with search execution time, page display time, server load, and resulting garbage collection time all going through the roof. Sorry, but this is, ahem, not very smart. It is also dangerous, and leads to a bad user experience. I don't know a single person at our organization that likes the default OR behavior of JIRA's search. No, it is definitely not true that this provides users with the best search results - even entirely disregarding the ordering issues.

Now with the ordering issues taken into account, the net result is that the user that just executed this incredibly heavy search gives up after looking at the first few summaries and realizing that all those results are irrelevant to him. He won't even try looking at more than a few of those issues, he will just go ahead and issue yet another search, or he'll give up. So he just swamped the box for no purpose at all - but it's really JIRA's fault, not his!

I can't help noticing that the description of the default operator in the Jira search documentation is copied verbatim from the Lucene documentation, and that the Lucene documentation says you just have to call "setDefaultOperator" on a QueryParser to change Lucene's behaviour. There seem to be just a handful of places in the Jira source where a QueryParser is instantiated.

Clearly I'd be insane to change these myself (presumably if it was that simple this could have been fixed already?), I was wondering whether anyone who knows more has an opinion on this

The same argumentation used here led to success for Confluence users in CONF-1881. Confluence is now much easier to use - whereas our users still tear their hair out over how cumbersome searching is in JIRA. They just don't find stuff, and the fact that they have to type the term AND in uppercase doesn't help. If they don't find the page they're looking for in the first 10 results, most people won't be looking further and will assume it does not exist.

I also strongly disagree with the notion that Confluence returns the results in a useful order. For one thing, it would be sensible to return more recent results first among those with equal matching weight. But no, you always get the old junk from ten versions ago listed first. And clearly a match in the title should weight much more heavily than one in the description, but a phrase match in the description should weigh more heavily than a single-word match in the title.

Maurizio,

I have reopened this issue. I think having the same behaviour between JIRA and Confluence certainly makes sense.

We will definitely keep this issue in mind, however, at the moment, I cannot give the exact implementation date for this issue.

Cheers,
Anton

Hi,

I want to add my 2 cents to this issue. My original issue was https://support.atlassian.com/browse/JSP-17318. This is an extract from my comment on that issue:

***************
You have to keep in mind that people are now used to using a "google like" ability and while the arguments made on issue JRA-5930 are okay I don't think they are solid enough to justify ignoring the issue.

Since confluence is using the AND by default, then why not be consistent. I believe that is what Atlasssian is now pushing with your new JIRA studio. The fact that it is a uniform platform. If you want your platform to be uniform then what could be more important than having a uniform way of searching for stuff regardless of which product I am using?

If you guys are really worried about affecting existing users that are used to the OR default, then give me a system parameter that I can set based on my preference. Then this would make everyone happy.
****************

I think that this issue should be reopened and reconsidered for a number of reasons. The prime reason is that I strongly disagree with the notion that "JIRA's existing search behaviour returns the best results for most people".

Concretely, when I type the following search phrase into our JIRA installation:

"Error Log" topmenuen

I expect issues containing both the phrase "Error Log" and the word "topmenuen" to be listed at the top of the list, but instead the list is swamped with issues containing only one of the search words, and the two issues containing both words aren't listed first.

My theory is that the problem stems from the fact that one of the words is located in the Summary, whereas the other word is located in the Description. Trying to boost the weight of one of the search words using, for example, ^4, doesn't help either. (This just shuffles the results, still without putting results with both words in the top).

I've previously filed a support case regarding the issue, but I was told to comment on this issue instead.

Another reason for reopening and reconsidering this issue is that AND searches are now the default behaviour in Confluence as well (see the linked issue).

I stand corrected - thanks.

Also, a search for 'big AND teeth" on yahoo produces the same results and number of results as "big teeth." Also, on the results page it removes the AND from the search.

If you use the MSN Search Builder, and you type words in the Search Terms box and select "All of these terms," then click "Add to search," it inserts the words verbatim without any + or AND, which shows that MSN search is also AND by default.

From http://www.google.com/help/basics.html#and :

"By default, Google only returns pages that include all of your search terms. There is no need to include "and" between terms. Keep in mind that the order in which the terms are typed will affect the search results. To restrict a search further, just include more terms. For example, to plan a vacation to Hawaii, simply type vacation hawaii."

Keith,

I am not an expert on the matter but I am not sure if google returns only AND results. I believe the results are returned in order of relevance, and hence documents with more words in them are returned first.

Thanks,
Anton

If this is how you feel then I accept it but I don't think it's best for users. I wonder how you justify the behavior when Google, Yahoo, MSN all use AND search, and they are considered to produce very good results.

Yes, I see what you mean.

I would submit that JIRA's existing search behaviour returns the best results for most people. If you search for "Default search operator", you get this issue as the first hit (keeping exact searchers happy) and get other issues listed, which are often also of interest to searchers. For instance, searching for "Default search operator" also returns the analogous Confluence issue CONF-1881.

This isn't to say that AND is never valid; just that OR returns more useful results for most people. If you want AND, add && between terms, eg. "Default && search && operator".

Jeff, this is not a duplicate of that bug. I don't want to see any results which don't contain my words in the summary, without having to type + before every word. This is how every popular search engine works.

It sounds like the problem here is that JIRA 3.0 returned results effectively unordered. In JIRA 3.2, searching for "xml doctype exception" will return issues with those exact words in the summary first, and subsequent results in order of decreasing 'relevance'. See JRA-5151

I don't want to have to weed through bad results. I want to search the issues for "xml doctype exception", see that there are no matches, and file my issue. I don't want to have to browse through every single request which contains "xml" or "exception."

Keith,

I am not exactly sure what you mean. Are your refering to free tet searching in issues' summary/description/comments?

If so, JIRA sorts the output by "relevance" so the issues that closer match the input will appear first. Is this not what you are seeing? If you absolutely need to limit the search with an AND operator you can separate the string by &&. For example:

search && for && all && these && strings

Please let us know if I misunderstood your query.

Thanks,
Anton