Index optimize breaks: java.io.IOException: read past EOF

A user reported getting errors when transitioning an issue:

Errors

• com.atlassian.bonnie.LuceneException: java.io.IOException: read past EOF

In the logs, this "read past EOF" error first occurs when the nightly reindex runs:

2007-01-28 00:00:00,019 JiraQuartzScheduler_Worker-0 INFO [issue.index.job.OptimizeIndexJob] Optimize Index Job running...
2007-01-28 00:00:00,341 JiraQuartzScheduler_Worker-0 ERROR [issue.index.job.OptimizeIndexJob] Error occurred while optimizing indexes.
com.atlassian.jira.issue.index.IndexException: com.atlassian.bonnie.LuceneException: java.io.IOException: read past EOF
at com.atlassian.jira.issue.index.DefaultIndexManager.optimize(DefaultIndexManager.java:374)
at com.atlassian.jira.issue.index.job.OptimizeIndexJob.execute(OptimizeIndexJob.java:19)
at org.quartz.core.JobRunShell.run(JobRunShell.java:191)
at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:516)
Caused by: com.atlassian.bonnie.LuceneException: java.io.IOException: read past EOF
at com.atlassian.bonnie.ConcurrentLuceneConnection.withWriter(ConcurrentLuceneConnection.java:303)
at com.atlassian.bonnie.ConcurrentLuceneConnection.optimize(ConcurrentLuceneConnection.java:116)
at com.atlassian.jira.issue.index.SingleThreadedIssueIndexer.optimize(SingleThreadedIssueIndexer.java:74)
at com.atlassian.jira.issue.index.DefaultIndexManager.optimize0(DefaultIndexManager.java:393)
at com.atlassian.jira.issue.index.DefaultIndexManager.optimize(DefaultIndexManager.java:370)
... 3 more
Caused by: java.io.IOException: read past EOF
at org.apache.lucene.store.BufferedIndexInput.refill(BufferedIndexInput.java:60)
at org.apache.lucene.store.BufferedIndexInput.readByte(BufferedIndexInput.java:33)
at org.apache.lucene.store.IndexInput.readInt(IndexInput.java:46)
at org.apache.lucene.index.SegmentTermEnum.<init>(SegmentTermEnum.java:47)
at org.apache.lucene.index.TermInfosReader.<init>(TermInfosReader.java:52)
at org.apache.lucene.index.SegmentReader.initialize(SegmentReader.java:147)
at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:129)
at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:110)
at org.apache.lucene.index.IndexWriter.mergeSegments(IndexWriter.java:674)
at org.apache.lucene.index.IndexWriter.mergeSegments(IndexWriter.java:658)
at org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java:517)
at com.atlassian.bonnie.ConcurrentLuceneConnection$2.perform(ConcurrentLuceneConnection.java:120)
at com.atlassian.bonnie.ConcurrentLuceneConnection.withWriter(ConcurrentLuceneConnection.java:296)
... 7 more

It seems that after this, the index is corrupt and any further attempt to modify it results in an error:

2007-01-28 00:29:19,998 TP-Processor294 ERROR [event.listeners.search.IssueIndexListener] Error re-indexing Issue[GenericEntity:Issue][summary,cloneNode forgets IDness of attribute][status,1][votes,0][fixfor,null][assignee,null][security,null][resolution,null][type,1][timespent,null][id,27912][reporter,elharo@metalab.unc.edu][project,10520][environment,java version "1.4.2_05"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.4.2_05-141.3)
Java HotSpot(TM) Client VM (build 1.4.2-38, mixed mode)][component,null][updated,2007-01-28 00:29:19.917][created,2004-10-22 06:24:43.0][workflowId,37907][description,Cloning a node that contains an ID type attribute forgets that the attribute is an ID. Interestingly importing the element into a new document, even one without a DocType, does not forget that the Attr has ID type.

The attached program generates the following output:

Xerces-J 2.6.2
Found element with ID
Did not find element with ID][priority,3][timeestimate,null][duedate,null][timeoriginalestimate,null][key,XERCESJ-1022]
com.atlassian.jira.issue.index.IndexException: com.atlassian.bonnie.LuceneException: java.io.IOException: read past EOF
at com.atlassian.jira.issue.index.DefaultIndexManager.reIndexIssues(DefaultIndexManager.java:292)
at com.atlassian.jira.issue.index.DefaultIndexManager.reIndexIssues(DefaultIndexManager.java:244)
at com.atlassian.jira.issue.index.DefaultIndexManager.reIndex(DefaultIndexManager.java:264)
at com.atlassian.jira.event.listeners.search.IssueIndexListener.reIndex(IssueIndexListener.java:109)
at com.atlassian.jira.event.listeners.search.IssueIndexListener.reIndex(IssueIndexListener.java:119)
at com.atlassian.jira.event.listeners.search.IssueIndexListener.issueCommented(IssueIndexListener.java:75)
at com.atlassian.jira.event.issue.AbstractIssueEventListener.workflowEvent(AbstractIssueEventListener.java:106)
at com.atlassian.jira.event.issue.IssueEventDispatcher.dispatchEvent(IssueEventDispatcher.java:107)
at com.atlassian.jira.event.issue.IssueEventDispatcher.dispatchEvent(IssueEventDispatcher.java:82)
at com.atlassian.jira.web.action.issue.AddComment.alertSystemOfComment(AddComment.java:101)
at com.atlassian.jira.web.action.issue.AddComment.doExecute(AddComment.java:85)
at webwork.action.ActionSupport.execute(ActionSupport.java:153)
at com.atlassian.jira.action.JiraActionSupport.execute(JiraActionSupport.java:54)
......
at java.lang.Thread.run(Thread.java:619)
Caused by: com.atlassian.bonnie.LuceneException: java.io.IOException: read past EOF
at com.atlassian.bonnie.ConcurrentLuceneConnection.constructIndexDeleter(ConcurrentLuceneConnection.java:438)
at com.atlassian.bonnie.ConcurrentLuceneConnection.withReaderAndDeletes(ConcurrentLuceneConnection.java:261)
at com.atlassian.bonnie.ConcurrentLuceneConnection.withDeleteAndWrites(ConcurrentLuceneConnection.java:324)
at com.atlassian.jira.issue.index.SingleThreadedIssueIndexer.reindexIssues(SingleThreadedIssueIndexer.java:332)
at com.atlassian.jira.issue.index.DefaultIndexManager.reIndexIssues(DefaultIndexManager.java:287)
... 68 more
Caused by: java.io.IOException: read past EOF
at org.apache.lucene.store.BufferedIndexInput.refill(BufferedIndexInput.java:60)
at org.apache.lucene.store.BufferedIndexInput.readByte(BufferedIndexInput.java:33)
at org.apache.lucene.store.IndexInput.readInt(IndexInput.java:46)
at org.apache.lucene.index.SegmentTermEnum.<init>(SegmentTermEnum.java:47)
at org.apache.lucene.index.TermInfosReader.<init>(TermInfosReader.java:52)
at org.apache.lucene.index.SegmentReader.initialize(SegmentReader.java:147)
at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:129)
at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:110)
at org.apache.lucene.index.IndexReader$1.doBody(IndexReader.java:154)
at org.apache.lucene.store.Lock$With.run(Lock.java:109)
at org.apache.lucene.index.IndexReader.open(IndexReader.java:143)
at org.apache.lucene.index.IndexReader.open(IndexReader.java:138)
at com.atlassian.bonnie.ConcurrentLuceneConnection.constructIndexDeleter(ConcurrentLuceneConnection.java:434)
... 72 more