-
Bug
-
Resolution: Fixed
-
High
-
8.1.0, 8.0.1, 8.0.2, 8.1.1, 8.2.1, 8.2.2, 8.3.1
-
8
-
1
-
Severity 2 - Major
-
2
-
Issue Summary
Both background and foreground reindex requests fail due to a field that has more than 32766 bytes in the field.
Public KB: https://confluence.atlassian.com/jirakb/indexing-fails-due-to-immense-field-974366114.html
Environment
This affects JIRA 8.0 and above.
In JIRA 8.0, Lucene was upgraded from 3.3 to 7.3. Lucene will now throw an exception instead of silently failing.
See Lucene upgrade for more information.
Steps to Reproduce
- Step 1 Insert more than 32766 characters into an indexed field.
- Step 2 Attempt to reindex
Expected Results
The Lucene Issue index completes without error.
Actual Results
The Issue reindex fails and the following error appears in the atlassian-jira.log:
2019-07-02 16:32:24,247 IssueIndexer:thread-9 WARN 10.x.x.x /secure/admin/jira/IndexReIndex!reindex.jspa [c.a.jira.index.AccumulatingResultBuilder] Document contains at least one immense term in field="<fieldname>" (whose UTF8 encoding is longer than the max length 32766), all of which were skipped. Please correct the analyzer to not produce such terms. The prefix of the first immense term is: '[list of entries in the field]...', original message: bytes can be at most 32766 in length; got 32767 2019-07-02 16:32:24,248 JiraTaskExectionThread-2 WARN 10.x.x.x /secure/admin/jira/IndexReIndex!reindex.jspa [c.a.jira.index.AccumulatingResultBuilder] Indexing failed for Issue - '<#####>'
Notes
Relevant links:
https://stackoverflow.com/questions/37070593/how-to-deal-with-document-contains-at-least-one-immense-term-in-solr
https://issues.apache.org/jira/browse/LUCENE-5472
https://discuss.elastic.co/t/error-document-contains-at-least-one-immense-term-in-field/66486
Notes on fix:
Starting with "Fix-Version", fields that exceed this limit will be removed before they are committed to Jira’s Lucene index in order to prevent entire indexing operations from failing. Each such event will emit an ERROR level message to the logs, allowing plugin developers to pinpoint the offending fields. The log entry looks like this:
2019-09-04 20:06:24,091 IssueIndexer:thread-6 ERROR admin 1206x3196x1 ujgapq 0:0:0:0:0:0:0:1 /secure/admin/IndexReIndex!reindex.jspa [c.a.j.issue.index.DocumentScrubber] A document contained a potential immense term in field customfield_10220_timeline. The field has been removed from the document.
Workaround
Currently there is no known workaround for this behavior. A workaround will be added here when available
- causes
-
JRASERVER-76392 Attempts to index a Lucene field with immense term still fails the issue index
- Gathering Impact
- is related to
-
JSDSERVER-6417 Indexing fails when SLA timeline becomes too big
- Closed
-
AI-198 An immense permission field causes index unavailability
- In Review
- mentioned in
-
Page Loading...