Uploaded image for project: 'Confluence Data Center'
  1. Confluence Data Center
  2. CONFSERVER-15937

OutOfMemoryError while migrating attachments from database to files

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • Medium
    • 4.0.5
    • 2.10.2
    • None

    Description

      I have changed the attachments storage from "database" to "Locally in Confluence home directory".

      During the migration task there were several OutOfMemoryError, so I increased memory with additional vm arguments: "-Xmx1600m -Xms200m -XX:MaxPermSize=256m". Nevertheless there's an OutOfMemoryError (but with a different java stack trace than before!):

      Exception in thread "Attachment data migration task" java.lang.OutOfMemoryError: Java heap space
      	at org.postgresql.util.PGbytea.toBytes(PGbytea.java:29)
      	at org.postgresql.jdbc2.AbstractJdbc2ResultSet.getBytes(AbstractJdbc2ResultSet.java:2048)
      	at org.postgresql.jdbc2.AbstractJdbc2ResultSet.getBinaryStream(AbstractJdbc2ResultSet.java:2168)
      	at com.mchange.v2.c3p0.impl.NewProxyResultSet.getBinaryStream(NewProxyResultSet.java:519)
      	at org.springframework.jdbc.support.lob.DefaultLobHandler.getBlobAsBinaryStream(DefaultLobHandler.java:64)
      	at org.springframework.orm.hibernate.support.BlobInputStreamType.nullSafeGetInternal(BlobInputStreamType.java:24)
      	at org.springframework.orm.hibernate.support.SpoolingBlobInputStreamType.nullSafeGetInternal(SpoolingBlobInputStreamType.java:45)
      	at org.springframework.orm.hibernate.support.AbstractLobType.nullSafeGet(AbstractLobType.java:137)
      	at net.sf.hibernate.type.CustomType.nullSafeGet(CustomType.java:92)
      	at net.sf.hibernate.type.AbstractType.hydrate(AbstractType.java:67)
      	at net.sf.hibernate.loader.Loader.hydrate(Loader.java:690)
      	at net.sf.hibernate.loader.Loader.loadFromResultSet(Loader.java:631)
      	at net.sf.hibernate.loader.Loader.instanceNotYetLoaded(Loader.java:590)
      	at net.sf.hibernate.loader.Loader.getRow(Loader.java:505)
      	at net.sf.hibernate.loader.Loader.getRowFromResultSet(Loader.java:218)
      	at net.sf.hibernate.loader.Loader.doQuery(Loader.java:285)
      	at net.sf.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:138)
      	at net.sf.hibernate.loader.Loader.doList(Loader.java:1063)
      	at net.sf.hibernate.loader.Loader.list(Loader.java:1054)
      	at net.sf.hibernate.hql.QueryTranslator.list(QueryTranslator.java:854)
      	at net.sf.hibernate.impl.SessionImpl.find(SessionImpl.java:1554)
      	at net.sf.hibernate.impl.SessionImpl.find(SessionImpl.java:1531)
      	at net.sf.hibernate.impl.SessionImpl.delete(SessionImpl.java:1678)
      	at net.sf.hibernate.impl.SessionImpl.delete(SessionImpl.java:1664)
      	at com.atlassian.confluence.pages.persistence.dao.hibernate.HibernateAttachmentDataDao.clearAttachmentDataTable(HibernateAttachmentDataDao.java:236)
      	at com.atlassian.confluence.pages.persistence.dao.hibernate.HibernateAttachmentDataDao.afterMigrationFrom(HibernateAttachmentDataDao.java:222)
      	at com.atlassian.confluence.pages.persistence.dao.hibernate.AbstractHibernateAttachmentDao$IntraHibernateAttachmentMigrator.migrate(AbstractHibernateAttachmentDao.java:526)
      	at com.atlassian.confluence.util.longrunning.AttachmentMigrationLongRunningTask$1.doInTransactionWithoutResult(AttachmentMigrationLongRunningTask.java:71)
      	at org.springframework.transaction.support.TransactionCallbackWithoutResult.doInTransaction(TransactionCallbackWithoutResult.java:33)
      	at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:127)
      	at com.atlassian.confluence.util.longrunning.AttachmentMigrationLongRunningTask.runInternal(AttachmentMigrationLongRunningTask.java:66)
      	at com.atlassian.confluence.util.longrunning.ConfluenceAbstractLongRunningTask.run(ConfluenceAbstractLongRunningTask.java:21)
      

      I think the problem isn't in the postgresql database driver but in clearAttachmentDataTable which calls SessionImpl.delete. SessionImpl.delete internally loads all attachments into the Heap, so if there is a few gigabytes of attachment data it must end in an OutOfMemoryError.

      Because of the OutOfMemoryError the only I can do is to rollback through a database restore.

      Attachments

        Issue Links

          Activity

            People

              slancashire Steve Lancashire (Inactive)
              a4a2fd6ab2d3 Andreas Bentele
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: