Uploaded image for project: 'Confluence Data Center'
  1. Confluence Data Center
  2. CONFSERVER-80171

Confluence leaking file descriptors on Linux with Java 8

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Highest
    • 7.19.7
    • 7.14.0, 7.15.0, 7.16.4, 7.19.0, 7.19.2, 7.20.0
    • Content - Attachments

    Description

      Note

      The only affected version that is currently supported is Confluence 7.19.x which will receive the fix directly. As such, there are no backports required for this fix.

      Please also see this comment for additional details on the fix as the fix may not be comprehensive.

      Issue Summary

      Confluence is leaking open file descriptors.

      This is reproducible on Data Center: yes

      Steps to Reproduce

      1. Install Confluence with Java 8 on Linux OS
      2. Check the file descriptor count for attachments directory:
        lsof -p `cat /opt/atlassian/confluence/work/catalina.pid` | grep ver003
        
        • Should be 0 rows returned
      1. Create a new page and add an image and apply Image Effects
        • This is just one sample scenario of how this issue can be reproduced
      2. Publish the page
      3. Check the file descriptor count for attachments directory:
        lsof -p `cat /opt/atlassian/confluence/work/catalina.pid` | grep ver003
        

      Expected Results

      The file descriptor count after publishing the page may spike up but eventually should drop back to 0 or remain low.

      Actual Results

      The file descriptor count after publishing the page increased to 1 and will not drop back down.

      $ lsof -p `cat /opt/atlassian/confluence/work/catalina.pid` | grep ver003
      java    5958 confluence 1238r      REG               0,50    898618     96 /mnt/shared-home/conflinux/attachments/ver003/74/131/131074/149/217/13467649/14516226/1
      

      The thread stack trace in the affected Confluence versions go down this code path for getAttachmentData:

      sun.nio.fs.UnixChannelFactory.newFileChannel(UnixChannelFactory.java:-1)
      sun.nio.fs.UnixChannelFactory.newFileChannel(UnixChannelFactory.java:148)
      sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:212)
      java.nio.file.Files.newByteChannel(Files.java:361)
      java.nio.file.Files.newByteChannel(Files.java:407)
      java.nio.file.spi.FileSystemProvider.newInputStream(FileSystemProvider.java:384)
      java.nio.file.Files.newInputStream(Files.java:152)
      com.atlassian.dc.filestore.impl.filesystem.FilesystemPathImpl.openInputStream(FilesystemPathImpl.java:105)
      com.atlassian.confluence.impl.pages.attachments.filesystem.AttachmentDataFile.getInputStream(AttachmentDataFile.java:137)
      com.atlassian.confluence.pages.persistence.dao.AttachmentDataStream$2.getInputStream(AttachmentDataStream.java:70)
      com.atlassian.confluence.pages.persistence.dao.hibernate.AbstractHibernateAttachmentDao.getAttachmentData(AbstractHibernateAttachmentDao.java:409)
      com.atlassian.confluence.pages.attachments.AbstractDelegatingAttachmentDao.getAttachmentData(AbstractDelegatingAttachmentDao.java:141)
      com.atlassian.confluence.pages.DefaultAttachmentManager.getAttachmentData(DefaultAttachmentManager.java:406)
      com.atlassian.confluence.pages.DefaultAttachmentManager.getAttachmentData(DefaultAttachmentManager.java:400)
      com.atlassian.confluence.pages.DelegatorAttachmentManager.getAttachmentData(DelegatorAttachmentManager.java:135)
      com.atlassian.confluence.impl.pages.attachments.ReadThroughCachingAttachmentManager.getAttachmentData(ReadThroughCachingAttachmentManager.java:116)
      sun.reflect.GeneratedMethodAccessor1594.invoke (Unknown Source)
      sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      java.lang.reflect.Method.invoke(Method.java:498)
      org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
      org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
      org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
      org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:123)
      org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:388)
      org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:119)
      org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
      org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215)
      com.sun.proxy.$Proxy187.getAttachmentData (Unknown Source)
      sun.reflect.GeneratedMethodAccessor1594.invoke (Unknown Source)
      sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      java.lang.reflect.Method.invoke(Method.java:498)
      com.atlassian.plugin.util.ContextClassLoaderSettingInvocationHandler.invoke(ContextClassLoaderSettingInvocationHandler.java:26)
      com.sun.proxy.$Proxy597.getAttachmentData (Unknown Source)
      sun.reflect.GeneratedMethodAccessor1594.invoke (Unknown Source)
      sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      java.lang.reflect.Method.invoke(Method.java:498)
      org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
      org.eclipse.gemini.blueprint.service.importer.support.internal.aop.ServiceInvoker.doInvoke(ServiceInvoker.java:56)
      org.eclipse.gemini.blueprint.service.importer.support.internal.aop.ServiceInvoker.invoke(ServiceInvoker.java:60)
      org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
      org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:137)
      org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:124)
      org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
      org.eclipse.gemini.blueprint.service.util.internal.aop.ServiceTCCLInterceptor.invokeUnprivileged(ServiceTCCLInterceptor.java:70)
      org.eclipse.gemini.blueprint.service.util.internal.aop.ServiceTCCLInterceptor.invoke(ServiceTCCLInterceptor.java:53)
      org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
      org.eclipse.gemini.blueprint.service.importer.support.LocalBundleContextAdvice.invoke(LocalBundleContextAdvice.java:57)
      org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
      org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:137)
      org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:124)
      org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
      org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215)
      com.sun.proxy.$Proxy1264.getAttachmentData (Unknown Source)
      com.atlassian.confluence.image.effects.ImageFilterServlet.lambda$buildAttachmentContext$4(ImageFilterServlet.java:238)
      com.atlassian.confluence.image.effects.ImageFilterTask.readOriginalImage(ImageFilterTask.java:210)
      com.atlassian.confluence.image.effects.ImageFilterTask.rotateWhenExifExist(ImageFilterTask.java:237)
      com.atlassian.confluence.image.effects.ImageFilterTask.inMemoryTransform(ImageFilterTask.java:262)
      com.atlassian.confluence.image.effects.ImageFilterTask.processInMemory(ImageFilterTask.java:200)
      com.atlassian.confluence.image.effects.ImageFilterTask.call(ImageFilterTask.java:108)
      com.atlassian.confluence.image.effects.ImageFilterTask.call(ImageFilterTask.java:62)
      com.atlassian.sal.core.executor.ThreadLocalDelegateCallable.call(ThreadLocalDelegateCallable.java:38)
      com.atlassian.confluence.impl.vcache.VCacheRequestContextManager.doInRequestContextInternal(VCacheRequestContextManager.java:84)
      com.atlassian.confluence.impl.vcache.VCacheRequestContextManager.doInRequestContext(VCacheRequestContextManager.java:68)
      com.atlassian.confluence.vcache.VCacheRequestContextOperations.lambda$withRequestContext$1(VCacheRequestContextOperations.java:59)
      java.util.concurrent.FutureTask.run(FutureTask.java:266)
      java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
      java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
      java.lang.Thread.run(Thread.java:748)
      

      Earlier unaffected Confluence versions went down this code path for getAttachmentData:

      	at java.io.FileInputStream.open0(Native Method)
      	at java.io.FileInputStream.open(FileInputStream.java:195)
      	at java.io.FileInputStream.<init>(FileInputStream.java:138)
      	at com.atlassian.confluence.pages.persistence.dao.AttachmentDataStream$FileWrapper.getInputStream(AttachmentDataStream.java:76)
      	at com.atlassian.confluence.pages.persistence.dao.hibernate.AbstractHibernateAttachmentDao.getAttachmentData(AbstractHibernateAttachmentDao.java:348)
      	at com.atlassian.confluence.pages.attachments.CachingAttachmentDao.getAttachmentData(CachingAttachmentDao.java:197)
      	at com.atlassian.confluence.pages.DefaultAttachmentManager.getAttachmentData(DefaultAttachmentManager.java:266)
      	at com.atlassian.confluence.pages.DefaultAttachmentManager.getAttachmentData(DefaultAttachmentManager.java:260)
      	at com.atlassian.confluence.pages.DelegatorAttachmentManager.getAttachmentData(DelegatorAttachmentManager.java:129)
      	at com.atlassian.confluence.pages.CachingAttachmentManager.getAttachmentData(CachingAttachmentManager.java:102)
      ...
      

      Workaround

      As both third party apps and internal Atlassian calls to getAttachmentData may result in an increasing file descriptors count in certain situations of Linux/Java 8, the current known workaround is to change Confluence to use Java 11 which is not affected by this issue.

       

       

      Additional notes
      It was identified that Microsoft Windows/Java 8 is also affected by leaking file handles, although unsure the impact is as extensive as on Linux. Switching to Java 11 also stops the file handle leak on Windows.

      Attachments

        Issue Links

          Activity

            People

              a9293dbdc671 Thomas Rogozinski
              hlam@atlassian.com Eric Lam
              Votes:
              9 Vote for this issue
              Watchers:
              39 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: