Confluence
  1. Confluence
  2. CONF-16067

Restore fails with "Too many open files" error

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 3.0
    • Fix Version/s: 3.4
    • Component/s: Backup / Restore
    • Environment:

      Ubuntu 9.04 x86-64
      Tomcat 6.0.18 (WAR version of Confluence)
      Sun JDK 1.6.0_14

      ulimit -n 1024 (i.e. the default max number of open files)

    • Last commented by user?:
      true
    • Internal Complexity:
      2
    • Internal Value:
      7

      Description

      Create a backup that includes a largish number of attachments (1292 attachments from Confluence 2.8.1 in this case). Try restoring in Confluence 3.0 and you get:

      2009-06-09 18:46:25,069 ERROR [Importing data task] [confluence.importexport.impl.BackupImporter] copyPaths There was a problem moving the attachments from the import to a temporary location.
       -- referer: http://10.1.0.30/wiki/admin/backup.action | url: /wiki/admin/restore-local-file.action | userName: aherbert | action: restore-local-file
      com.atlassian.confluence.importexport.ImportExportException: Couldn't restore directory from backup! src: /data/confluence/temp/import_06092009_183931/attachments/39453843/39616583 dest: /data/confluence/temp/import_06092009_183931/attachments-new/39453843/39616583
              at com.atlassian.confluence.importexport.impl.FileBackupImporter.restoreDirectory(FileBackupImporter.java:463)
              at com.atlassian.confluence.importexport.impl.FileBackupImporter.copyPaths(FileBackupImporter.java:245)
              at com.atlassian.confluence.importexport.impl.FileBackupImporter.importAttachments(FileBackupImporter.java:289)
              at com.atlassian.confluence.importexport.impl.FileBackupImporter.importEverything(FileBackupImporter.java:104)
              at com.atlassian.confluence.importexport.impl.BackupImporter.doImport(BackupImporter.java:100)
              at com.atlassian.confluence.importexport.impl.FileBackupImporter.doImport(FileBackupImporter.java:42)
              at com.atlassian.confluence.importexport.DefaultImportExportManager.importAs(DefaultImportExportManager.java:74)
              at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
              at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
              at java.lang.reflect.Method.invoke(Method.java:597)
              at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:304)
              at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:182)
              at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:149)
              at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:106)
              at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
              at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
              at $Proxy73.importAs(Unknown Source)
              at com.atlassian.confluence.importexport.actions.ImportLongRunningTask.runInternal(ImportLongRunningTask.java:54)
              at com.atlassian.confluence.util.longrunning.ConfluenceAbstractLongRunningTask.run(ConfluenceAbstractLongRunningTask.java:21)
              at java.lang.Thread.run(Thread.java:619)
      Caused by: java.io.IOException: Too many open files
              at java.io.UnixFileSystem.createFileExclusively(Native Method)
              at java.io.File.createNewFile(File.java:883)
              at com.atlassian.core.util.FileUtils.copyFile(FileUtils.java:465)
              at com.atlassian.core.util.FileUtils.copyDirectory(FileUtils.java:351)
              at com.atlassian.confluence.importexport.impl.FileBackupImporter.restoreDirectory(FileBackupImporter.java:457)
              ... 20 more
      2009-06-09 18:46:25,990 ERROR [Importing data task] [confluence.importexport.actions.ImportLongRunningTask] runInternal Failure during import
      

        Activity

        Hide
        Alfred Nathaniel added a comment -

        Very annoying regression in 3.x. I just found that to be the cuprit for breaking my instance every few days with a blue screen about not finding dashboard.vm.

        I now have the choice between stopping backups or restarting the instance once per day

        Show
        Alfred Nathaniel added a comment - Very annoying regression in 3.x. I just found that to be the cuprit for breaking my instance every few days with a blue screen about not finding dashboard.vm. I now have the choice between stopping backups or restarting the instance once per day
        Hide
        Matt Ryall [Atlassian] added a comment -

        Please fix this by using the implementation of copyDirectory() in org.apache.commons.io.FileUtils. It is much better tested and likely to have fewer bugs.

        (It has also been fixed in atlassian-core, see CORE-142.)

        I've reduced the complexity value appropriately.

        Show
        Matt Ryall [Atlassian] added a comment - Please fix this by using the implementation of copyDirectory() in org.apache.commons.io.FileUtils . It is much better tested and likely to have fewer bugs. (It has also been fixed in atlassian-core, see CORE-142.) I've reduced the complexity value appropriately.
        Hide
        Mark Berger added a comment -

        Hi

        Is there any plan to get this fixed soon? It would be good so that we can stop using the kludge workaround.

        Cheers
        Mark

        Show
        Mark Berger added a comment - Hi Is there any plan to get this fixed soon? It would be good so that we can stop using the kludge workaround. Cheers Mark
        Hide
        Roy Hartono [Atlassian] added a comment -

        Hi guys,

        If anyone has a sample space backup to share that exhibits this problem please comment on this ticket so we can test the fix for it.

        We will contact you via personal email after you commented.

        Thanks,

        Roy Hartono
        Sydney Confluence Support
        Atlassian

        Show
        Roy Hartono [Atlassian] added a comment - Hi guys, If anyone has a sample space backup to share that exhibits this problem please comment on this ticket so we can test the fix for it. We will contact you via personal email after you commented. Thanks, Roy Hartono Sydney Confluence Support Atlassian
        Hide
        Mark Berger added a comment -

        Hi Roy

        Yep, we have a file - it is the nightly backup of our production confluence system. However, it is unlikely that we can send this off site.

        If you need to call me, my direct landline no is 3 9425 2721

        Cheers
        Mark

        Show
        Mark Berger added a comment - Hi Roy Yep, we have a file - it is the nightly backup of our production confluence system. However, it is unlikely that we can send this off site. If you need to call me, my direct landline no is 3 9425 2721 Cheers Mark

          People

          • Votes:
            4 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:
              Last commented:
              3 years, 33 weeks, 1 day ago