-
Suggestion
-
Resolution: Fixed
-
None
-
All with internationalization settings set correctly and using CJK indexing
Hello,
We noticed incorrect behavior in how Confluence searches for single-bye and double-byte Japanese strings. Search for roman alphabets yields results incorporating both single and double-byte matches, as it should be. Similar behavior is required when searching for Japanese characters, for example, search for single byte katakana or numeric characters should return results matching both single and double byte occurrences, but now only double-byte matches are being retrieved. Please see attached Excel sheets summarizing the current behavior and the required behavior for single-byte
and double-byte Japanese strings. Could you please investigate this and incorporate this improvement in a future release?
Thanks,
Neeraj
- is incorporated by
-
- Closed
- is related to
-
CONFSERVER-9833 Search not working for Powerpoint or PDF files containing Japanese text
-
- Closed
-
-
-
- Closed
-
- relates to
-
-
- Closed
-
-
-
- Closed
-
-
-
- Closed
-
[CONFSERVER-9258] Incorrect search results for single and double-byte Japanese strings
Removed a test result file, which had been attached to this issue, as requested by the customer.
Neeraj Jhanji
added a comment - Yes, would be great if you can provide a patch for 2.6.0 that address CONF-9258, CONF-9833 & CONF-9834, which are all major issues from Japanese QA perspective.
Neeraj Jhanji
added a comment - Yes, would be great if you can provide a patch for 2.6.0 that address CONF-9258 , CONF-9833 & CONF-9834 , which are all major issues from Japanese QA perspective. Hi Neeraj,
The changes were not too numerous, so I should be able to provide a patch for 2.6.0 if essential.
The search tab has not yet been fixed, I have created another issue (CONF-9834) which should be watched for updates.
We are focused on releasing Confluence 2.6 to Japanese customers. This is a major upgrade from the previous JP release 2.4.3 since it fixes a major issue surrounding PDF export. If possible, we'd like to include the search fix in this release as well since the next Japanese release will be further out (possibly next year). Is it possible to get a patch for 2.6?
Also, to confirm, will search for half width katakana and numeric characters work properly from the top search bar as well as from the search tab?
Neeraj Jhanji
added a comment - - edited We are focused on releasing Confluence 2.6 to Japanese customers. This is a major upgrade from the previous JP release 2.4.3 since it fixes a major issue surrounding PDF export. If possible, we'd like to include the search fix in this release as well since the next Japanese release will be further out (possibly next year). Is it possible to get a patch for 2.6?
Also, to confirm, will search for half width katakana and numeric characters work properly from the top search bar as well as from the search tab? Hi Neeraj,
My apologies, I provide the wrong issue number. It should be LUCENE-1032.
The custom Japanese analyzer will be available as an option in the Content Indexing tab from 2.6.2 onwards.
Regards,
Andrew Lynch
Neeraj Jhanji
added a comment - Hi Andrew,
1. Where can I get the Custom Japanese Analyzer and what are the installation instructions for it?
2. Regarding the Lucene support issue you mention above, I did not see any mention of the problems with half-width Japanese characters.
Please clarify.
regards,
Neeraj
Neeraj Jhanji
added a comment - Hi Andrew,
1. Where can I get the Custom Japanese Analyzer and what are the installation instructions for it?
2. Regarding the Lucene support issue you mention above, I did not see any mention of the problems with half-width Japanese characters.
Please clarify.
regards,
Neeraj
A quick update Neeraj,
Lucene's CJKAnalyzer is definitely not indexing half width characters correctly. I have raised an issue (http://issues.apache.org/jira/browse/LUCENE-1032) to address this.
We were considering creating a patch ourselves, but the simplest implementation would require usage of Java 6's Normalizer class. In order to solve this, I have created our Analyzer, Custom Japanese Analyzer. Unfortunately this only works on Sun JDKs and so it will not be incorporated into Lucene and may not work for all customers.
Customers who are experiencing problems such as the ones you outlined should use this Analyzer in place of CJKAnalyzer until the issue with Lucene is resolved, assuming they have a Sun JDK.
systeminfo.png was removed as the customer's request.