Uploaded image for project: 'Confluence Data Center'
  1. Confluence Data Center
  2. CONFSERVER-17225

Viewxls macro causes out of memory error and then fails to render

      Creating on behalf of customer:

      The problem occurs while rendering the content from the attached Excel Spreadsheet using the viewxls macro.
      When the page the macro is used on is about to display the application is under high load, meaning both memory and cpu usage increase above normal, and one of the following things happens:

      1. The page is displayed but the rendering of the macro fails with an OutOfMemoryError
      2. The page is not displayed and the upstream Apache HTTP server replies with

      Proxy Error

      The proxy server received an invalid response from an upstream server.
      The proxy server could not handle the request GET /confluence/display/prozesse/Kennzahlenprotokoll.

      Reason: Error reading from remote server

      Regardless of which of the both cases occurs, eventually the whole application becomes unavailable and has to be restarted. Also requesting other pages (without the macro) is not possible anymore.

      It seems that the sheet "Leading Diagramm" is causing the problems (though other sheets do also cause high load).

      When more memory is given to the Java process it requires more concurrent users to cause the aforementioned problems. But even with 1,5 GB of memory 4 users are enough to cause the problems.

      Below you can find the Wiki markup of the page embedding the Excel content:

      {toc}
      
      h2. Leading Kennzahlen
      
      {viewxls:page=Kennzahlen|name=4_Kennzahlenprotokoll_Atlassian.xls|sheet=Leading Diagramm|row=80|col=7}
      
      h2. Lagging Kennzahlen
      
      {viewxls:page=Kennzahlen|name=4_Kennzahlenprotokoll_Atlassian.xls|sheet=Lagging Diagramm|row=160|col=7}
      

            [CONFSERVER-17225] Viewxls macro causes out of memory error and then fails to render

            As I have noted elsewhere, if {viewxls} tries to render a new Excel 2007 XML-encoded spreadsheet as an old-style document (which it will do if it was uploaded as a file named with a .xls extension, since Confluence's attachment upload accepts that and gives it a MIME-type of application/vnd.ms-excel), it will cause an out-of-memory error because it apparently reads four bytes of arbitrary XML text as binary numbers specifying the rows and columns of the spreadsheet, and tries to allocate that many cells.

            Specifying a tame number of maximum rows and columns with {viewxls} parameters can avoid the catastrophe, and you will see XML code splattered all over the cells instead of the actual spreadsheet data.

            Brian M. Thomas added a comment - As I have noted elsewhere, if { viewxls } tries to render a new Excel 2007 XML-encoded spreadsheet as an old-style document (which it will do if it was uploaded as a file named with a .xls extension, since Confluence's attachment upload accepts that and gives it a MIME-type of application/vnd.ms-excel ), it will cause an out-of-memory error because it apparently reads four bytes of arbitrary XML text as binary numbers specifying the rows and columns of the spreadsheet, and tries to allocate that many cells. Specifying a tame number of maximum rows and columns with { viewxls } parameters can avoid the catastrophe, and you will see XML code splattered all over the cells instead of the actual spreadsheet data.

            Partha added a comment -

            Dear Rome,

            Thanks for those clear steps and sample attachment.

            I have created a new bug for you at http://jira.atlassian.com/browse/CONF-21043.

            Kind Regards,
            Partha Kamal

            Partha added a comment - Dear Rome, Thanks for those clear steps and sample attachment. I have created a new bug for you at http://jira.atlassian.com/browse/CONF-21043 . Kind Regards, Partha Kamal

            I've been able to reproduce this problem easily in Confluence 3.3.

            Steps:

            1. Create an Excel spreadsheet with data in a very high row number (the attached one has data in rows 1, 2, and 65536). This file can actually be quite small (41kb). One of our users actually did this by accident!
            2. Attach spreadsheet to a page
            3. Use the viewxls macro to display the spreadsheet on your page.

            Result: Most of the time, the page won't load - it times out or returns "Page Not Found". Sometimes it WILL load with a "viewxls: java.lang.OutOfMemoryError: Java heap space" error. Java VM Heap free space rapidly plummets (as reported by refreshing the System Info page of Confluence's Admin panel). The entire Confluence instance gets slower. Sometimes the entire Confluence installation stops responding for a little while (eventually it recovered itself).

            Rome Reginelli added a comment - I've been able to reproduce this problem easily in Confluence 3.3. Steps: Create an Excel spreadsheet with data in a very high row number (the attached one has data in rows 1, 2, and 65536). This file can actually be quite small (41kb). One of our users actually did this by accident! Attach spreadsheet to a page Use the viewxls macro to display the spreadsheet on your page. Result: Most of the time, the page won't load - it times out or returns "Page Not Found". Sometimes it WILL load with a "viewxls: java.lang.OutOfMemoryError: Java heap space" error. Java VM Heap free space rapidly plummets (as reported by refreshing the System Info page of Confluence's Admin panel). The entire Confluence instance gets slower. Sometimes the entire Confluence installation stops responding for a little while (eventually it recovered itself).

            Small spreadsheet with data in a row > 65,000

            Rome Reginelli added a comment - Small spreadsheet with data in a row > 65,000

            ok, CSP-50699

            Jeff Kirby added a comment - ok, CSP-50699

            Hi Jeff,

            You should open a new support ticket at https://support.atlassian.com, and upload the file there.

            Then we can determine whether it is a known issue or a new bug.

            Cheers,
            Mark

            Mark Hrynczak (Inactive) added a comment - Hi Jeff, You should open a new support ticket at https://support.atlassian.com , and upload the file there. Then we can determine whether it is a known issue or a new bug. Cheers, Mark

            Jeff Kirby added a comment - - edited

            We're still having problems with this in Confluence 3.3.1. I have a spreadsheet that a user once attempted to upload. No one has any idea how he created it, but he did, and I still have it and it still causes java.lang.OutOfMemoryError: GC Overhead Limit Exceeded.

            Shall we open a new bug?

            I can't attach it as it's 67 MB and contains some data that we'd rather not publish to everyone. I'll gladly make it available to Atlassian.

            This was working in 3.2.1_01.
            Something regressed.

            Jeff Kirby added a comment - - edited We're still having problems with this in Confluence 3.3.1. I have a spreadsheet that a user once attempted to upload. No one has any idea how he created it, but he did, and I still have it and it still causes java.lang.OutOfMemoryError: GC Overhead Limit Exceeded. Shall we open a new bug? I can't attach it as it's 67 MB and contains some data that we'd rather not publish to everyone. I'll gladly make it available to Atlassian. This was working in 3.2.1_01. Something regressed.

            Jeff Kirby added a comment -

            In our Confluence 3.0.2 instance we're avoiding the problem by using the patch for the OfficeConnector-1.5.jar that I attached to CONF-17783. It first calls an extension of the ConverterMacro that rejects a spreadsheet greater than 2 MB.

            Jeff Kirby added a comment - In our Confluence 3.0.2 instance we're avoiding the problem by using the patch for the OfficeConnector-1.5.jar that I attached to CONF-17783 . It first calls an extension of the ConverterMacro that rejects a spreadsheet greater than 2 MB.

            RyanA added a comment - - edited

            Thanks for the suggestion. We use a third party library for the Excel rendering so it's not as simple as you make it sound to "detect" files that the library can't open. It either works or it doesn't. The best we can do is add better exception handling which I will look at doing in the future.

            I have good news, the issue with this particular file has been fixed for Confluence 3.1. Note that it works locally for us with a server running 1.5 GB of RAM. I have no idea of the minimum memory requirement that this file may have.

            RyanA added a comment - - edited Thanks for the suggestion. We use a third party library for the Excel rendering so it's not as simple as you make it sound to "detect" files that the library can't open. It either works or it doesn't. The best we can do is add better exception handling which I will look at doing in the future. I have good news, the issue with this particular file has been fixed for Confluence 3.1. Note that it works locally for us with a server running 1.5 GB of RAM. I have no idea of the minimum memory requirement that this file may have.

            Hi Anatoli,

            that's exactly how we perceive it. The interesting question is why is this happening? Is it caused by the size of the document (1.3 MB)? Or is there some content within the file the macro chokes on?
            In any case I think it would be great if the macro would be able to detect possible "malicious" files, refuses to render them and writes a warning to the logs.

            I'm stressing this, because this particular document caused one week of repeated application downtime for one of our customers until we were able to identify the cause. As a consequence they were forced to deactivate the OfficeConnector-Plugin. This is a setback to them as the plugin was a crucial part of their intranet.

            Regards,
            Tino

            Tino Winkler [Communardo] added a comment - Hi Anatoli, that's exactly how we perceive it. The interesting question is why is this happening? Is it caused by the size of the document (1.3 MB)? Or is there some content within the file the macro chokes on? In any case I think it would be great if the macro would be able to detect possible "malicious" files, refuses to render them and writes a warning to the logs. I'm stressing this, because this particular document caused one week of repeated application downtime for one of our customers until we were able to identify the cause. As a consequence they were forced to deactivate the OfficeConnector-Plugin. This is a setback to them as the plugin was a crucial part of their intranet. Regards, Tino

              rackley RyanA
              ganand Gurleen Anand [Atlassian]
              Affected customers:
              0 This affects my team
              Watchers:
              9 Start watching this issue

                Created:
                Updated:
                Resolved: