Confluence currently stores the entire contents of textual attachments (like word, txt and powerpoint documents) in the search index. This is done mainly to produce excerpts that display in search results.

During index optimization the bodies of attachment are read from the disk into memory in bulk (meaning a block of memory equal to the size of all indexed & stored attachments is requested from the heap). With enough large attachments, the JVM will be asked for a large block of heap it cannot allocate and an OutOfMemoryError is produced.