Details
-
Bug
-
Resolution: Unresolved
-
Medium
-
None
-
CCMA - 3.9.1
-
None
-
5
-
Severity 3 - Minor
-
3
Description
Issue Summary
The below error occurs while running the migration plan containing more than 30+ spaces.
java.lang.OutOfMemoryError: unable to create native thread: possibly out of memory or process/resource limits reached
The max-user-processes and max-file-descriptors are set to 32k on the machine where this issue occurred. The total allocated heap size is 18GB, which should suffice for space migration. During the error, only 16% of the heap space was utilized. Despite this configuration, the OOM still occurs. After the server restarts, it can successfully migrate 10 spaces. However, subsequent migrations fail due to Out of Memory (OOM) errors.
Environment Details:
Confluence version: 7.13.7
OS: Linux
Database: Postgres 14.10
This is reproducible on Data Center: (yes) / (no)
Steps to Reproduce
- Use the CCMA 3.9.0 or 3.9.1
- Create a migration containing more than 10+ spaces and run the migration
- Again, create a migration plan with 30+ spaces and run the migration.
- The space migration in 2nd migration plan fails with the below error.
Expected Results
All the spaces should be successfully migrated
Actual Results
The below exception is thrown in the atlassion-confluence.log file:
2024-01-02 13:09:33,677 ERROR [http-nio-8090-exec-365] [common.error.jersey.ThrowableExceptionMapper] toResponse Uncaught exception thrown by REST service: unable to create native thread: possibly out of memory or process/resource limits reached -- referer: https://xxxxxx.xxxxxx.xx/admin/migration.action | url: /rest/migration/latest/plan/ea4765bd-0698-4dcb-95e1-4344ca466caa | traceId: c7fbae74bb736c1a | userName: xxxxxxxxxxxxxxxx java.lang.OutOfMemoryError: unable to create native thread: possibly out of memory or process/resource limits reached at java.base/java.lang.Thread.start0(Native Method) at java.base/java.lang.Thread.start(Unknown Source) at java.base/java.util.concurrent.ForkJoinPool.createWorker(Unknown Source) at java.base/java.util.concurrent.ForkJoinPool.tryAddWorker(Unknown Source) at java.base/java.util.concurrent.ForkJoinPool.signalWork(Unknown Source) at java.base/java.util.concurrent.ForkJoinPool.externalPush(Unknown Source) at java.base/java.util.concurrent.ForkJoinTask.fork(Unknown Source) at java.base/java.util.stream.AbstractTask.compute(Unknown Source) at java.base/java.util.concurrent.CountedCompleter.exec(Unknown Source) at java.base/java.util.concurrent.ForkJoinTask.doExec(Unknown Source) at java.base/java.util.concurrent.ForkJoinTask.doInvoke(Unknown Source) at java.base/java.util.concurrent.ForkJoinTask.invoke(Unknown Source) at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateParallel(Unknown Source) at java.base/java.util.stream.AbstractPipeline.evaluate(Unknown Source) at java.base/java.util.stream.ReferencePipeline.collect(Unknown Source) at com.atlassian.migration.agent.service.impl.PlanConverter.resolveTaskDTOsForPlan(PlanConverter.java:172) at com.atlassian.migration.agent.service.impl.PlanConverter.entityToDto(PlanConverter.java:138)
The below exception is thrown in the catalina.log
72730.407s][warning][os,thread] Failed to start thread - pthread_create failed (EAGAIN) for attributes: stacksize: 1024k, guardsize: 0k, detached. [72730.407s][warning][os,thread] Failed to start thread - pthread_create failed (EAGAIN) for attributes: stacksize: 1024k, guardsize: 0k, detached. Exception in thread "OkHttp TaskRunner" java.lang.OutOfMemoryError: unable to create native thread: possibly out of memory or process/resource limits reached [72730.408s][warning][os,thread] Failed to start thread - pthread_create failed (EAGAIN) for attributes: stacksize: 1024k, guardsize: 0k, detached. at java.base/java.lang.Thread.start0(Native Method) at java.base/java.lang.Thread.start(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.addWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.processWorkerExit(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source)
Workaround
- Downgrade to CCMA v3.7.1 following the steps provided in the article: How to install a specific version of the Jira Cloud Migration Assistant