We have this hardcoded in the scheduler configuration to run at midnight every day:
That needs to go away.
During normal operation, Lucene deletes documents by simply marking them as such but leaving them in the segment. This is a performance trade-off because it makes the write much faster, but wastes disk space and would eventually slow down searches that have to skip over the deleted documents.
In older versions of Lucene, such as those used prior to JIRA 5.0, this is dealt with by explicitly optimizing the index, which rebuilds all of the segments without the deleted documents in them. To make sure the indexes stay in reasonably good shape, we added the scheduled job shown above.
In JIRA 5.0, we adopted Lucene 3.2 and its new TieredMergePolicy that tries to distribute this work more reasonably over time by noticing when deleting a document puts the segment over some threshold (say 10% deleted documents) and optimizing that single segment immediately. Since this happens automatically, there is much less reason to explicitly request a global optimization of the entire index, which for large customers in particular can be a very expensive operation and lead to request timeouts as write operations stall waiting up to 30 seconds for the index optimization to get out of the way if people happen to be doing work at midnight server time.
In some cases, we have even seen this lead to inconsistent state between the database and the Lucene indexes as the write requests for the index get dropped. This was allegedly fixed in JRA-14220, but a recent support case indicates that it may not adequately address the problem.
In short, the optimize operation should be superfluous at this point, and the long delays it causes may be creating additional problems, so it should just go away.