Uploaded image for project: 'Confluence Data Center'
  1. Confluence Data Center
  2. CONFSERVER-21043

Excel Sheets with large row count causes out of memory errors

      I've been able to reproduce this problem easily in Confluence 3.3.

      Steps:

      1. Create an Excel spreadsheet with data in a very high row number (the attached one has data in rows 1, 2, and 65536). This file can actually be quite small (41kb). One of our users actually did this by accident!
      2. Attach spreadsheet to a page
      3. Use the viewxls macro to display the spreadsheet on your page.

      Result: Most of the time, the page won't load - it times out or returns "Page Not Found". Sometimes it WILL load with a "viewxls: java.lang.OutOfMemoryError: Java heap space" error. Java VM Heap free space rapidly plummets (as reported by refreshing the System Info page of Confluence's Admin panel). The entire Confluence instance gets slower. Sometimes the entire Confluence installation stops responding for a little while (eventually it recovered itself).

      sample stack trace
      2010-11-24 11:18:12,051 ERROR [TP-Processor6] [renderer.v2.components.MacroRendererComponent] processMacro Unexpected error formatting macro: viewfile
       -- url: /display/spacekey/pageTitle | page: 34709164 | userName: admin | referer: https://confluenceURL/display/spacekey/pageTitle | action: viewpage
      java.lang.OutOfMemoryError: Java heap space
      	at java.util.Arrays.copyOf(Arrays.java:2882)
      	at java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:100)
      	at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:390)
      	at java.lang.StringBuffer.append(StringBuffer.java:224)
      	at java.io.StringWriter.write(StringWriter.java:84)
      	at com.benryan.conversion.SpreadsheetConverter.writeHeaderStyles(SpreadsheetConverter.java:204)
      	at com.benryan.conversion.SpreadsheetConverter.write(SpreadsheetConverter.java:65)
      	at com.benryan.conversion.SpreadsheetConverter.convert(SpreadsheetConverter.java:395)
      	at com.benryan.conversion.XlsConverter$1.doConversion(XlsConverter.java:42)
      	at com.benryan.conversion.DocConverter.execute(DocConverter.java:59)
      	at com.benryan.conversion.macro.ConverterMacro.execute(ConverterMacro.java:276)
      	at com.atlassian.renderer.v2.macro.ResourceAwareMacroDecorator.execute(ResourceAwareMacroDecorator.java:51)
      	at com.atlassian.renderer.v2.components.MacroRendererComponent.executeMacro(MacroRendererComponent.java:203)
      	at com.atlassian.renderer.v2.components.MacroRendererComponent.processMacro(MacroRendererComponent.java:169)
      	at com.atlassian.renderer.v2.components.MacroRendererComponent.makeMacro(MacroRendererComponent.java:67)
      	at com.atlassian.renderer.v2.components.WikiContentRendererHandler.handleMacro(WikiContentRendererHandler.java:18)
      	at com.atlassian.renderer.v2.WikiMarkupParser.makeMacro(WikiMarkupParser.java:120)
      	at com.atlassian.renderer.v2.WikiMarkupParser.handlePotentialMacro(WikiMarkupParser.java:105)
      	at com.atlassian.renderer.v2.WikiMarkupParser.parse(WikiMarkupParser.java:60)
      	at com.atlassian.renderer.v2.components.MacroRendererComponent.render(MacroRendererComponent.java:46)
      	at com.atlassian.renderer.v2.V2Renderer.render(V2Renderer.java:53)
      	at com.atlassian.renderer.v2.plugin.PluggableV2Renderer.render(PluggableV2Renderer.java:30)
      	at com.atlassian.renderer.v2.V2RendererFacade.convertWikiToXHtml(V2RendererFacade.java:57)
      	at sun.reflect.GeneratedMethodAccessor393.invoke(Unknown Source)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
      	at java.lang.reflect.Method.invoke(Method.java:597)
      	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:304)
      	at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:182)
      	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:149)
      	at com.atlassian.spring.interceptors.SpringProfilingInterceptor.invoke(SpringProfilingInterceptor.java:16)
      	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
      	at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:106)
      

            [CONFSERVER-21043] Excel Sheets with large row count causes out of memory errors

            Hi Eric,
            This is fixed in the version of the office connector bundled with Confluence 4.0.5 and up, and was mostly due to the sheer amount of html and inline styles generated by the macro when trying to generate 100,000+ cells.

            Steve Lancashire (Inactive) added a comment - Hi Eric, This is fixed in the version of the office connector bundled with Confluence 4.0.5 and up, and was mostly due to the sheer amount of html and inline styles generated by the macro when trying to generate 100,000+ cells.

            We are experiencing this on a 3.4.8 instance. To the above poster, it has nothing to do with content, rather rowcount. Follow the instructions at the top. A 37k file can consume 4G of ram as it goes down a MASSIVE string concatenation loop.

            Eric Koepfle added a comment - We are experiencing this on a 3.4.8 instance. To the above poster, it has nothing to do with content, rather rowcount. Follow the instructions at the top. A 37k file can consume 4G of ram as it goes down a MASSIVE string concatenation loop.

            Daz added a comment -

            Looks good to me!

            Running Confluence @ -Xmx612m, I couldn't make the excel conversion fall over, even after pasting an entire copy of Anna Karenina in to 100 odd table cells. The same spreadsheet still got parsed after throwing a 1MB image in to some cells, too.

            Daz added a comment - Looks good to me! Running Confluence @ -Xmx612m, I couldn't make the excel conversion fall over, even after pasting an entire copy of Anna Karenina in to 100 odd table cells. The same spreadsheet still got parsed after throwing a 1MB image in to some cells, too.

            Awaiting atlas-eye to start technical review

            Steve Lancashire (Inactive) added a comment - Awaiting atlas-eye to start technical review

            We are severely impacted by this issue. We are running Confluence in a clustered setup. We have 4GB of memory configured. We get out of memory and CPU spike, which effectively brings node-by-node down.
            Sometimes users just make a mistake and put text into some high value rows/columns. So there is no way to prevent this from happening in the future.
            The macro should limit amount of rows x columns it can process.

            Dmytro Ivchenko added a comment - We are severely impacted by this issue. We are running Confluence in a clustered setup. We have 4GB of memory configured. We get out of memory and CPU spike, which effectively brings node-by-node down. Sometimes users just make a mistake and put text into some high value rows/columns. So there is no way to prevent this from happening in the future. The macro should limit amount of rows x columns it can process.

            We are experience with this same type error. See: CSP-66718

            Stephen Gramm added a comment - We are experience with this same type error. See: CSP-66718

            We have observed and documented that this same problem is present in every version of Confluence up to version 3.5.x when using the Office Connector plugin.

            Stephen Gramm added a comment - We have observed and documented that this same problem is present in every version of Confluence up to version 3.5.x when using the Office Connector plugin.

            The CONF-21043 is marked to be affected in 3.3 and 3.3.1 version. The problem appears also on 3.4 and 3.4.9 version with the same excel file that crashes Confluence...

            Hope it will be resolve soon.

            Michael

            Michael Regelin added a comment - The CONF-21043 is marked to be affected in 3.3 and 3.3.1 version. The problem appears also on 3.4 and 3.4.9 version with the same excel file that crashes Confluence... Hope it will be resolve soon. Michael

            Hi Brad,

            You can delete it using <BASEURL>/pages/removepage.action?pageId=XXXXX

            Or edit it to remove the spreadsheet using <BASEURL>/pages/editpage.action?pageId=XXXXX

            To help us diagnose the problem, please can you provide the file which causes it. Either attach to this issue, or if you want to keep the contents private then create a new ticket at https://support.atlassian.com

            Cheers,
            Mark

            Mark Hrynczak (Inactive) added a comment - Hi Brad, You can delete it using <BASEURL>/pages/removepage.action?pageId=XXXXX Or edit it to remove the spreadsheet using <BASEURL>/pages/editpage.action?pageId=XXXXX To help us diagnose the problem, please can you provide the file which causes it. Either attach to this issue, or if you want to keep the contents private then create a new ticket at https://support.atlassian.com Cheers, Mark

            Got this on a file with only 500 rows. Stuck with the Page Not Found error; how do I delete the page? Are there other ways to embed spreadsheets in a page?

            Brad Newman added a comment - Got this on a file with only 500 rows. Stuck with the Page Not Found error; how do I delete the page? Are there other ways to embed spreadsheets in a page?

              slancashire Steve Lancashire (Inactive)
              346167d53d55 Rome Reginelli
              Affected customers:
              16 This affects my team
              Watchers:
              24 Start watching this issue

                Created:
                Updated:
                Resolved: