-
Bug
-
Resolution: Fixed
-
Medium
-
5.8.2, 5.8.4
-
None
-
Java 8
Problem Definition
Default Java metaspace defined in the Confluence system properties is too small. Support has received multiple tickets from user instances that hang or crash after upgrading to 5.8.x, and a similar error is displayed in the logs:
2015-06-10 10:02:05,782 ERROR [scheduler_Worker-10] [org.quartz.core.ErrorLogger] schedulerError Job (DEFAULT.IndexQueueFlusher threw an exception.
org.quartz.SchedulerException: Job threw an unhandled exception. [See nested exception: java.lang.OutOfMemoryError: Metaspace]
at org.quartz.core.JobRunShell.run(JobRunShell.java:234)
at com.atlassian.confluence.schedule.quartz.ConfluenceQuartzThreadPool.lambda$runInThread$46(ConfluenceQuartzThreadPool.java:19)
at com.atlassian.confluence.schedule.quartz.ConfluenceQuartzThreadPool$$Lambda$91/1891777742.run(Unknown Source)
at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:549)
Caused by: java.lang.OutOfMemoryError: Metaspace
Suggested Solution
Set the default Java metaspace as -XX:MaxMetaspaceSize=384m or higher in the CATALINA_OPTS, or do not include a -XX:MaxMetaspaceSize since the JVM will allocate it as needed.
See KB : Confluence crashes due to 'java.lang.OutOfMemoryError Metaspace' error for more information
- mentioned in
-
![[Atlassian Documentation] Page](/images/icons/generic_link_16.png "[Atlassian Documentation] Page")
Page
No Confluence page found with the given URL.
-
Page
No Confluence page found with the given URL.
-
Page
No Confluence page found with the given URL.
-
Page
No Confluence page found with the given URL.
-
Page
No Confluence page found with the given URL.
-
Page Loading...
-
-
-
[CONFSERVER-37925] Increase or remove default -XX:MaxMetaspaceSize argument
I didn't remember the sources as this discussion was hot a long time ago, but if this source is right, then the solution is fine. I had in mind that its not garbage collected until it reaches the MaxMetaspaceSize, which defaults to the available native memory. But I didn't find anything regarding this anymore. Seems I was wrong.
Frank, I'm interested in your sources of information, and specifically why a production environment should have it set. I googled a abit on the new Metaspace memory space, and the most informative article I found with real data (though they did use a pre-release of Java 8 in their tests) was here: https://dzone.com/articles/java-8-permgen-metaspace.
My impression is you are best to not limit metaspace but do monitor memory usage, unless you are running a Java app under very specific memory constraints (where it's more important to fail with a memory allocation error than it is to start swapping), perhaps on an embedded device. That's not how I would run a production Jira instance, I would rather we discover a memory leak by high memory usage and not the jvm grinding to a halt for our 500 users, unable to allocate memory.
I couldn't find any information in agreement where you said the metaspace was not GC'd until the complete main memory was full - I read that the metaspace is GC'd only when the dynamically set metaspace was full and had to be resized larger. If you can point me to contrary information, please do.
Lastly, I haven't seen any performance issues in our production Confluence instance from the metaspace being dynamically resized; if there were (e.g. a 30 second pause in responsiveness), that would be a reason to manually set metaspace size very large ahead of time, though you are guessing at what it should be set to (+1mb higher than your highest usage), and if you guess wrong, your production instance goes down.
In this specific case, the MaxMetaspaceSize value that shipped with Confluence 5.8.5 was set too low for the product to function once under load; setting MaxMetaspaceSize in production was an anti-pattern for us.
-Kelly Schoenhofen
Interesting - I hope they made a notice in the installation doc, that you should add this argument for production environments. Otherwise the metaspace is not garbage collected until the complete main memory is full - which is obviously bad.
Although Atlassian did not explain how they addressed the problem, I can confirm that in v5.8.6 the MaxMetaspaceSize argument has been removed from setenv.sh. ie, followed the suggestion of that ChangJoon Lee made.
It can't be 5.8.5, because 5.8.5 contained the argument. Affects Versions should be updated to include 5.8.5. I'm eagerly awaiting 5.8.6, due to the vast # of javascript/Chrome bugs in 5.8.5.
-Kelly Schoenhofen
Andreas van Rienen (Scandio)
added a comment - Is it right for me to presume that Fix Version should be set to 5.8.5? Would be interesting to know.
Andreas van Rienen (Scandio)
added a comment - Is it right for me to presume that Fix Version should be set to 5.8.5? Would be interesting to know. 
I vote for remove.This brought us down after our upgrade from 5.7 to 5.8. Voting and watching. Ugh.
Jira has a closed assignment which includes increasing and decreasing malathion. Here you can get rushessay some important skill about the study. They are providing multiple tickets from user instances that hang or crash after upgrading to 5.8. You can also get their V version. Join it for more.