Uploaded image for project: 'Confluence Data Center'
  1. Confluence Data Center
  2. CONFSERVER-40394

Space export fails with error: "Couldn't backup database data"

      Summary

      Exporting selected pages of a relatively large space to XML will failed and the error message: "Couldn't backup database data" is shown in the UI

      Environment

      • Oracle 11g
      • Oracle 12
      • SQL Server
      • Microsoft SQL Server

      Steps to Reproduce

      1. Have Confluence connected to Oracle 11g
      2. Have a relatively large space (with 1000+ deleted pages in Oracle)
      3. Go to Space Tools > Content Tools > Export > XML
      4. Custom Export > Choose few pages
      5. Export

      Expected Results

      Export was successful

      Actual Results

      Export was not successful

      2015-12-08 14:36:48,837 ERROR [Long running task: Export Space] [confluence.importexport.impl.AbstractXmlExporter] backupEntities Couldn't backup database data.
       -- referer: https://www-test.wiki.ed.ac.uk/spaces/exportspacexml.action?key=GenePool | url: /spaces/doexportspace.action | userName: aforres3 | action: doexportspace
      org.springframework.jdbc.BadSqlGrammarException: Hibernate operation: Could not execute query; bad SQL grammar []; nested exception is java.sql.SQLSyntaxErrorException: ORA-01795: maximum number of expressions in a list is 1000
      ...
      Caused by: java.sql.SQLSyntaxErrorException: ORA-01795: maximum number of expressions in a list is 1000
      ...
      

      This has also been observed with SQL Server:

      2018-10-22 23:08:42,036 WARN [Long running task: Export Space] [hibernate.query.internal.QueryParameterBindingsImpl] expandListValuedParameters HHH000443: Dialect [com.atlassian.confluence.impl.hibernate.dialect.SQLServerDialect] limits the number of elements in an IN predicate to 2100 entries.  However, the given parameter list [exclusions] contained 20701 entries, which will likely cause failures to execute the query in the database
      

      For the Confluence running on the MSSQL server, the error message in the logs is this one:

       2020-05-26 17:44:23,744 ERROR [Long running task: Export Space] [engine.jdbc.spi.SqlExceptionHelper] logExceptions The incoming request has too many parameters. The server supports a maximum of 2100 parameters. Reduce the number of parameters and resend the request.
       – url: /spaces/doexportspace.action | referer: [https://xxxxxxxxxxxxxxxx:spaces/exportspacexml.action?key=SPACE] | traceId: 116358ab22940a24 | userName: xxxxxxx | action: doexportspace
       2020-05-26 17:44:23,745 ERROR [Long running task: Export Space] [confluence.importexport.impl.AbstractXmlExporter] backupEntities Couldn't backup database data.
       – url: /spaces/doexportspace.action | referer: [https://xxxxxxxxx/spaces/exportspacexml.action?key=SPAPCE] | traceId: 116358ab22940a24 | userName: XXXXXXXX | action: doexportspace
       org.springframework.dao.InvalidDataAccessResourceUsageException: could not extract ResultSet; SQL [n/a]; nested exception is org.hibernate.exception.SQLGrammarException: could not extract ResultSet
       at org.springframework.orm.hibernate5.SessionFactoryUtils.convertHibernateAccessException(SessionFactoryUtils.java:224)
       at org.springframework.orm.hibernate5.HibernateTemplate.doExecute(HibernateTemplate.java:368)
       at org.springframework.orm.hibernate5.HibernateTemplate.execute(HibernateTemplate.java:315)
       ...
       at java.lang.Thread.run(Thread.java:748)
       Caused by: org.hibernate.exception.SQLGrammarException: could not extract ResultSet
       ....
       Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: The incoming request has too many parameters. The server supports a maximum of 2100 parameters. Reduce the number of parameters and resend the request.

      Notes

      This is caused by some methods in confluence that returns more than 1000 entities/object. Oracle has a limitation of maximum acceptable number of entities in a list to be 1000

      On the MSSQL there is a hard-coded limit (2100) for the number of entries in the IN statement:

      https://github.com/Microsoft/msphpsql/blob/v4.2.0-preview/source/shared/core_sqlsrv.h#L172

      https://github.com/microsoft/msphpsql/issues/410

      Workaround

      Exporting the whole space to XML works alright. As such, by copying the pages to a new (empty) Space, and exporting the entire new Space, the pages should be exported successfully.

            [CONFSERVER-40394] Space export fails with error: "Couldn't backup database data"

            We have 7.4.8. and have this issue (with MSSQL), so it is not quite fixed I assume

            alexander.schultz@init.de added a comment - We have 7.4.8. and have this issue (with MSSQL), so it is not quite fixed I assume

            A fix for this issue is available to Server and Data Center customers in Confluence 7.6.3
            Upgrade now or check out the Release Notes to see what other issues are resolved.

            Hasnae (Inactive) added a comment - A fix for this issue is available to Server and Data Center customers in Confluence 7.6.3 Upgrade now or check out the Release Notes to see what other issues are resolved.

            A fix for this issue is available to Server and Data Center customers in Confluence 7.10.1
            Upgrade now or check out the Release Notes to see what other issues are resolved.

            Hasnae (Inactive) added a comment - A fix for this issue is available to Server and Data Center customers in Confluence 7.10.1 Upgrade now or check out the Release Notes to see what other issues are resolved.

            If you're running the Confluence 7.4 Enterprise release, a fix for this issue is now available in Confluence 7.4.6, which you can find in the Download Archives.

            Oliver Shen added a comment - If you're running the Confluence 7.4 Enterprise release, a fix for this issue is now available in Confluence 7.4.6, which you can find in the Download Archives .

            Peter-I added a comment -

            jmarfell, thank you! Glad this this fix made it in the 7.4 LTS release

            Peter-I added a comment - jmarfell , thank you! Glad this this fix made it in the 7.4 LTS release

            972c8973aa3b It is part of 7.4.6. I just updated this ticket to highlight that.

            Jen Marfell (Inactive) added a comment - 972c8973aa3b It is part of 7.4.6. I just updated this ticket to highlight that.

            Peter-I added a comment -

            jmarfell, I was checking the release notes for 7.4.6, however this fix is not listed there. Can you confirm which version is getting it?

            Peter-I added a comment - jmarfell , I was checking the release notes for 7.4.6, however this fix is not listed there. Can you confirm which version is getting it?

            We are now backporting this change to 7.4.6.

            Jen Marfell (Inactive) added a comment - We are now backporting this change to 7.4.6.

            MegaFon Team added a comment - - edited

            gonchik added a comment  6 days ago  edited
            Hi!
            Any plans to backport onto 7.4.x release?

            Because with that on 7.4.4

            Hi, 

            I join the question.
            7.4.x is stated as "Long Term Support release". This bug is quite critical and should be fixed in the LTS release.
             

            MegaFon Team added a comment - - edited gonchik  added a comment   6 days ago   edited Hi! Any plans to backport onto 7.4.x release? Because with that on 7.4.4 Hi,  I join the question. 7.4.x is stated as "Long Term Support release". This bug is quite critical and should be fixed in the LTS release.  

            Gonchik Tsymzhitov added a comment - - edited

            Hi!
            Any plans to backport onto 7.4.x release?

            Because with that on 7.4.4

            Gonchik Tsymzhitov added a comment - - edited Hi! Any plans to backport onto 7.4.x release? Because with that on 7.4.4

            Tam Tran added a comment -

            A fix for this issue is available to Server and Data Center customers in Confluence 7.8.0
            Upgrade now or check out the Release Notes to see what other issues are resolved.

            Tam Tran added a comment - A fix for this issue is available to Server and Data Center customers in Confluence 7.8.0 Upgrade now or check out the Release Notes to see what other issues are resolved.

            Hi g.beraudo! The deleting of pages is a separate case. The un-selecting of pages during a custom export is covered in the other notes, ie "Custom with only 2/2500 pages selected".

            Bradley Hyde added a comment - Hi g.beraudo ! The deleting of pages is a separate case. The un-selecting of pages during a custom export is covered in the other notes, ie "Custom with only 2/2500 pages selected".

            GBE added a comment -

            @Bradley, regarding your testing notes:

            > * Do we have tests for space export?
            > ** If so, do we delete ~2500 pages beforehand to make sure the "not in" clause is handled correctly?

             

            Please note that the issue is when 'un selecting' more than 1000+pages, not when deleting those pages from the space (at least in our case).

            Thus, if you create a space with 1001 pages and select only one in the export, Confluence is considering that you excluded 1000 and will crash.

            GBE added a comment - @Bradley, regarding your testing notes: > * Do we have tests for space export? > ** If so, do we delete ~2500 pages beforehand to make sure the "not in" clause is handled correctly?   Please note that the issue is when 'un selecting' more than 1000+pages, not when deleting those pages from the space (at least in our case). Thus, if you create a space with 1001 pages and select only one in the export, Confluence is considering that you excluded 1000 and will crash.

            Jeramy S added a comment - - edited

            My company is trying to migrate from our Confluence Server (6.0.6 - hosted by my division) to Confluence Server (7.4 - hosted by our parent company instead of us) and we want to migrate spaces using the export space as XML feature. We have 10 spaces that we want to migrate. I can export all ten of them as an XML without any issues; however, one of these spaces is 4.5 GB.

            For testing purposes, we want to export a few pages from the 4.5 GB space. However, we get this error. While it is a disappointment to read that your teams are de-prioritizing this issue, I understand.

             

            Other than copying the pages to a new space, are there any workarounds that we can do to extract a few pages from that large space in XML format for testing?

            Jeramy S added a comment - - edited My company is trying to migrate from our Confluence Server (6.0.6 - hosted by my division) to Confluence Server (7.4 - hosted by our parent company instead of us) and we want to migrate spaces using the export space as XML feature. We have 10 spaces that we want to migrate. I can export all ten of them as an XML without any issues; however, one of these spaces is 4.5 GB. For testing purposes, we want to export a few pages from the 4.5 GB space. However, we get this error. While it is a disappointment to read that your teams are de-prioritizing this issue, I understand.   Other than copying the pages to a new space, are there any workarounds that we can do to extract a few pages from that large space in XML format for testing?

            We're de-prioritising this as we have more higher priority and higher UIS bugs.

            Jen Marfell (Inactive) added a comment - We're de-prioritising this as we have more higher priority and higher UIS bugs.

            Workaround to export the whole space does not help in our instance.

            The LongRunning export task is aborted by exceeding the threshold of 60000 ms due to attempt to backup the db.

            When and how will this be fixed?

            Harald Mueller added a comment - Workaround to export the whole space does not help in our instance. The LongRunning export task is aborted by exceeding the threshold of 60000 ms due to attempt to backup the db. When and how will this be fixed?

            marc.riddell It will be picked up after we finish what we needed to deliver for our ER release. I'm anticipating sometime late this month or early next month if nothing else pops up.

            Jen Marfell (Inactive) added a comment - marc.riddell It will be picked up after we finish what we needed to deliver for our ER release. I'm anticipating sometime late this month or early next month if nothing else pops up.

            mriddell added a comment -

            Is there any guidance on when this issue will go back into in-progress? 

            mriddell added a comment - Is there any guidance on when this issue will go back into in-progress? 

            Moving this back to Short Term Backlog due to re-prioritisation of work in the team.

            Jen Marfell (Inactive) added a comment - Moving this back to Short Term Backlog due to re-prioritisation of work in the team.

            Sattesh M added a comment -

            Hi all,

            This bug has been moved from the Long term backlog to the Short Term Backlog of our development team.

            We have also recently created a new dashboard that will make it easier for you to see what bug fixes we are working on that will provide you with a view into our short term backlog and includes the following key information:

            • Bugs that have been prioritised and are currently being worked on.
            • Issues or feature suggestions that have been prioritised for development through a project/theme of work.
            • Version the fix will be shipped in so you can track when the update will be available.

            We hope with this dashboard, you can better manage your instance and plan upgrades.

            Cheers,
            Sattesh Maran

            Sattesh M added a comment - Hi all, This bug has been moved from the Long term backlog to the Short Term Backlog of our development team. Atlassian Server Bug Fix Policy We have also recently created a new dashboard that will make it easier for you to see what bug fixes we are working on that will provide you with a view into our short term backlog and includes the following key information: Bugs that have been prioritised and are currently being worked on. Issues or feature suggestions that have been prioritised for development through a project/theme of work. Version the fix will be shipped in so you can track when the update will be available. We hope with this dashboard, you can better manage your instance and plan upgrades. Confluence Server & Data Center Bug Fix Board Cheers, Sattesh Maran

            AndreasZ added a comment -

            That's still an issue with Confluence Server 7.3.2. Our database is Microsoft SQL Server 2017.
            The error message here is "Die eingehende Anforderung verfügt über zu viele Parameter. Der Server unterstützt maximal 2100 Parameter. Reduzieren Sie die Anzahl der Parameter, und senden Sie die Anforderung erneut." (that should be "The incoming tabular data stream (TDS) remote procedure call (RPC) protocol stream is incorrect. Too many parameters were provided in this RPC request. The maximum is 2100." in English).
            Please fix that, because that is a growing problem for us. A full space export is not an alternative because it doesn't respect page restrictions.

            AndreasZ added a comment - That's still an issue with Confluence Server 7.3.2. Our database is Microsoft SQL Server 2017. The error message here is "Die eingehende Anforderung verfügt über zu viele Parameter. Der Server unterstützt maximal 2100 Parameter. Reduzieren Sie die Anzahl der Parameter, und senden Sie die Anforderung erneut." (that should be "The incoming tabular data stream (TDS) remote procedure call (RPC) protocol stream is incorrect. Too many parameters were provided in this RPC request. The maximum is 2100." in English). Please fix that, because that is a growing problem for us. A full space export is not an alternative because it doesn't respect page restrictions.

            Same problem here with Confluence 6.14.1and Oracle 12c. The workaround is not really an alternative .

            Dietmar Pradler added a comment - Same problem here with Confluence 6.14.1and Oracle 12c. The workaround is not really an alternative .

            Same problem here with Confluence 6.6.13 and Oracle 12c. The workaround is not really an alternative .

            Simone Kaiser added a comment - Same problem here with Confluence 6.6.13 and Oracle 12c. The workaround is not really an alternative .

            We are facing same issue in confluence 6.13.4, Oracle 12c

            Patrik Vincej added a comment - We are facing same issue in confluence 6.13.4, Oracle 12c

            ITSO AS added a comment -

            We're facing the same Problem with Confluence 6.15.2 and SQL Server

            ITSO AS added a comment - We're facing the same Problem with Confluence 6.15.2 and SQL Server

            glennrees added a comment -

            I am getting this error with Oracle 12c and Confluence 6.14.2

            glennrees added a comment - I am getting this error with Oracle 12c and Confluence 6.14.2

            Also with Confluence Server 6.11 with Oracle 11g

            Oficina Calidad y Mejora Continua added a comment - Also with Confluence Server 6.11 with Oracle 11g

            GBE added a comment -

            We are also facing the same problem with CONFLUENCE 5.10.8 and SQL Server 

            Seems similar to https://jira.atlassian.com/browse/CONF-42654 

            GBE added a comment - We are also facing the same problem with CONFLUENCE 5.10.8 and SQL Server  Seems similar to https://jira.atlassian.com/browse/CONF-42654  

            AM added a comment -

            We have this issue as well on Oracle 11g and Confluence 5.9.9

            AM added a comment - We have this issue as well on Oracle 11g and Confluence 5.9.9

              bhyde Bradley Hyde
              jwaihou Jonathan Soo
              Affected customers:
              42 This affects my team
              Watchers:
              74 Start watching this issue

                Created:
                Updated:
                Resolved: