[#CONF-8989] Index rebuild tasks involving an index optimization produce an OutOfMemoryError when there are many large textual attachments

Confluence

Index rebuild tasks involving an index optimization produce an OutOfMemoryError when there are many large textual attachments

Created: 23/Jul/07 07:58 PM Updated: 14/May/08 01:13 AM

Component/s:

Performance, Searching / Indexing

Affects Version/s:

2.5.4

Fix Version/s:

2.5.6

Time Tracking:

Not Specified

File Attachments:

atlassian-bonnie-2.1.0-patched-CONF-8989.jar (122 kB)

Issue Links:

Reference

This issue relates to:

~~CONF-10512~~ Index optimization produce an OutOfMemoryError due to large textual attachments

This issue is related to:

~~CONF-9000~~ OutOfMemoryError's during indexing

Participants:	Dave Loeng [Atlassian] and Don Willis [Atlassian]
Since last comment:	1 year, 1 day ago
Resolution Date:	28/Feb/08 12:26 AM
Labels:

Description

« Hide

Confluence currently stores the entire contents of textual attachments (like word, txt and powerpoint documents) in the search index. This is done mainly to produce excerpts that display in search results.

During index optimization the bodies of attachment are read from the disk into memory in bulk (meaning a block of memory equal to the size of all indexed & stored attachments is requested from the heap). With enough large attachments, the JVM will be asked for a large block of heap it cannot allocate and an OutOfMemoryError is produced.

All

Comments

Work Log

Change History

Sort Order:

[ Permlink | « Hide ]

Don Willis [Atlassian] added a comment - 19/Aug/07 09:56 PM - edited

The attached jar is a patch to the bonnie library in Confluence. It can be installed by removing the existing bonnie jar and adding this one to (usually) the WEB-INF/lib directory, then restarting Confluence. This patch is suitable for Confluence 2.5.4 and 2.5.5.

The patch prevents the content of large attachments being stored in the Confluence index, and thus will:

decrease the size of the index when large attachments are present
decrease the memory used in indexing large attachments
stop excerpts of large attachments being displayed in search results

By default, large attachment is defined as greater than 1MB.
The threshold for attachments that won't get excerpts can be modified using the system property atlassian.indexing.contentbody.maxsize, which takes a size in bytes. Eg to specify 250 kb you would use the JVM parameter:
-Datlassian.indexing.contentbody.maxsize=256000