Uploaded image for project: 'Confluence Data Center'
  1. Confluence Data Center
  2. CONFSERVER-34751

Importing an XML Export does not create inline tasks and comments correctly

      NOTE: This bug report is for Confluence Server. Using Confluence Cloud? See the corresponding bug report.

      Fix description

      Hello everyone,

      The fix for this issue will not fix the links for XML exports that are already imported, but will prevent the issue from happening on newer import jobs.
      This applies to Space XML exports prior to Confluence 7.0.1 as well, and they will not hit into this issue if imported into Confluence 7.0.1 and later.

      Steps to Reproduce
      1. Create a new Space
      2. Export the Space as XML
      3. Delete the Space you just created
      4. Import the Space Export
      5. On a page, create the Task Report Macro, and point it at the imported space.
      Expected Result

      The Task Report Macro should show the tasks that are incomplete, or complete (depending on the settings). Using this example, the report should show 4 incomplete tasks (from the default home page)

      Actual Result

      The Task Report Macro doesn't show the tasks that are incomplete. Marking a task as complete and then incomplete doesn't show the task either.

      The task does not appear in the database.

      Workaround (not applicable to Confluence Cloud)
      1. Perform a full backup of the Confluence Database
      2. Shutdown Confluence
      3. Execute the following SQL Queries:
        delete from "AO_BAF3AA_AOINLINE_TASK";
        delete from bandana where bandanakey = 'com.atlassian.confluence.plugins.confluence-inline-tasks:build';
        
      4. Start Confluence

      Confluence will re-migrate any inline tasks to the new inline task format, which will correctly re-add the tasks not handled when importing.

      Notes

      It is possible to tweak the migration task thread number to speed up the process. To do so, add the following system property to the instance before starting the process:

      CATALINA_OPTS="-Dtasklist.storagetoaoupgradetask.numthreads=18 ${CATALINA_OPTS}"
      

      To define the ideal number of threads for the process, we can use the following formula:

      ideal_thread_number = availableProcessors() * 2 + 2;
      

            [CONFSERVER-34751] Importing an XML Export does not create inline tasks and comments correctly

            Sattesh M added a comment -

            Hello daniel.hertrich.external, good question!

            I just tested this issue to confirm the behavior, and unfortunately, just upgrading will not re-create the link to the Task Report macro.
            That being said, since the data is in the XML file, we do not need to export another Space export from the source Confluence, and can just import the existing XML file.

            Cheers,
            Sattesh

            Sattesh M added a comment - Hello daniel.hertrich.external , good question! I just tested this issue to confirm the behavior, and unfortunately, just upgrading will not re-create the link to the Task Report macro. That being said, since the data is in the XML file, we do not need to export another Space export from the source Confluence, and can just import the existing XML file. Cheers, Sattesh

            Daniel added a comment - - edited

            We migrated spaces from Confluence Server 6.11.2 to another instance with same version and lost all tasks in the task reports and in User Avatar > Tasks.

            I think this is this issue, correct?

            Question: If we update Confluence on the migration target environment to >= 7.0.1, will the tasks re-appear "automatically"? Or do we have to apply the workaround? Thanks!

            Daniel added a comment - - edited We migrated spaces from Confluence Server 6.11.2 to another instance with same version and lost all tasks in the task reports and in User Avatar > Tasks. I think this is this issue, correct? Question: If we update Confluence on the migration target environment to >= 7.0.1, will the tasks re-appear "automatically"? Or do we have to apply the workaround? Thanks!

            Quan Pham added a comment -

            A fix for this issue is available to Server and Data Center customers in Confluence 7.0.1
            Upgrade now or check out the Release Notes to see what other issues are resolved.

            Quan Pham added a comment - A fix for this issue is available to Server and Data Center customers in Confluence 7.0.1 Upgrade now or check out the Release Notes to see what other issues are resolved.

            lukasz added a comment -

            Fixed in Confluence Inline Task plugin 13.0.0 which will be bundled with Confluence 7.0.1

            lukasz added a comment - Fixed in Confluence Inline Task plugin 13.0.0 which will be bundled with Confluence 7.0.1

            Thank you, Jack! We will have to test this out, but I will report back with results.

            Taniya Choudhury added a comment - Thank you, Jack! We will have to test this out, but I will report back with results.

            Jack Chen added a comment -

            Assume the task report is working OK on the original 5.10 instance, then those task report records should be in 5.10 instance's "AO_BAF3AA_AOINLINE_TASK" table. We can recreate the task report records by two steps:

            1. run a SQL query on the original 5.10 instance like 

            copy (
            select distinct spacekey,title from content c join "AO_BAF3AA_AOINLINE_TASK" a on a."CONTENT_ID"=c.contentid
            join spaces s on s.spaceid=c.spaceid
            where c.prevver is null and content_status = 'current' and spacekey in ( 'key1','key2', ...  ) order by spacekey,title
            ) to '/tmp/taskpages.csv' DELIMITER '^'
            

            We use postgres database, if you are using a different database, the command can be different. The result file should include all migrated pages with task report records, with their spacekey and title data. You can later locate those migrated pages by the spacekey and page title.

            2. after the migration, update all those pages by script:

            #!/bin/bash
            
            BASEDIR=$(dirname $0)
            
            shopt -s expand_aliases
            source  $BASEDIR/.config.sh
            
            while read line; do
                IFS='^'       read  spacekey title<<< "$line"
            
                echo $title
                cli --action getPageSource  --space $spacekey --title "$title"  | grep -v '^Source$'   > .TMPPG
                sed -i -e  's/<ac:task-id>\([0-9]\+\)</<ac:task-id>10\1</g' .TMPPG
                cli --action  storepage --space $spacekey --title "$title"  --noConvert  --file .TMPPG --minor
            
            done < taskpages.csv
            exit
            

            "cli" is a linux alias to call confluence CLI with credential, so it will get the new page's content, then replace anything like

            "<ac:task-id>123</ac:taskid>" to "<ac:task-id>10123</ac:taskid>".

            the task-id is not a globally unique id, it only need to be unique in current task list. When the id is changed, it will trigger Confluence StorageToAoMigrationUpgradeTask to recreate the task record in "AO_BAF3AA_AOINLINE_TASK" table, so we only rebuild those missing records in those migrated pages.

            Jack Chen added a comment - Assume the task report is working OK on the original 5.10 instance, then those task report records should be in 5.10 instance's "AO_BAF3AA_AOINLINE_TASK" table. We can recreate the task report records by two steps: 1. run a SQL query on the original 5.10 instance like  copy ( select distinct spacekey,title from content c join "AO_BAF3AA_AOINLINE_TASK" a on a. "CONTENT_ID" =c.contentid join spaces s on s.spaceid=c.spaceid where c.prevver is null and content_status = 'current' and spacekey in ( 'key1' , 'key2' , ... ) order by spacekey,title ) to '/tmp/taskpages.csv' DELIMITER '^' We use postgres database, if you are using a different database, the command can be different. The result file should include all migrated pages with task report records, with their spacekey and title data. You can later locate those migrated pages by the spacekey and page title. 2. after the migration, update all those pages by script: #!/bin/bash BASEDIR=$(dirname $0) shopt -s expand_aliases source $BASEDIR/.config.sh while read line; do IFS= '^' read spacekey title<<< "$line" echo $title cli --action getPageSource --space $spacekey --title "$title" | grep -v '^Source$' > .TMPPG sed -i -e 's/<ac:task-id>\([0-9]\+\)</<ac:task-id>10\1</g' .TMPPG cli --action storepage --space $spacekey --title "$title" --noConvert --file .TMPPG --minor done < taskpages.csv exit "cli" is a linux alias to call confluence CLI with credential, so it will get the new page's content, then replace anything like "<ac:task-id>123</ac:taskid>" to "<ac:task-id>10123</ac:taskid>". the task-id is not a globally unique id, it only need to be unique in current task list. When the id is changed, it will trigger Confluence StorageToAoMigrationUpgradeTask to recreate the task record in "AO_BAF3AA_AOINLINE_TASK" table, so we only rebuild those missing records in those migrated pages.

            Hey Jack, can you provide the script you used? We're running into the same issue after a 5.10 -> 6.10 migration, and the workaround proposed by Atlassian would be too risky.

            Thanks.

            Taniya Choudhury added a comment - Hey Jack, can you provide the script you used? We're running into the same issue after a 5.10 -> 6.10 migration, and the workaround proposed by Atlassian would be too risky. Thanks.

            Jack Chen added a comment -

            I met the problem when migrating another confluence into our main confluence server.

             

            The workaround doesn't work for our instance. I see what it does is deleting existing data in AO_BAF3AA_AOINLINE table, and recreate them from page bodycontent. 

             

            This is a dangerous operation, specially for big instances with lots of history. First it will take very long time for the rebuilding job crawl all the pages to identify tasks, second in big instance with old data, there could be many data error introduced by system upgrade/old addons, etc. then the rebuilding job will fail on certain page and never able to finish the task report rebuilding.

            The log doesn't tell which content is causing the problem:

            2019-07-11 10:52:49,874 ERROR [StorageToAoMigrationUpgradeTask-TaskExtractor-1] [plugins.tasklist.upgradetask.InterruptingUncaughtExceptionHandler] uncaughtException Uncaught exception in thread StorageToAoMigrationUpgradeTask-TaskExtractor-1. Interrupting all threads.
            [com.ctc.wstx.exc.WstxLazyException] com.ctc.wstx.exc.WstxParsingException: Unexpected close tag </xml>; expected </ac:rich-text-body>.
              

             

            I suggest Atlassian to add more information like the contentid , or make the StorageToAoMigrationUpgradeTask more robust to bypass error pages.

             

            I found a better workaround for space migration: just find all those current pages with tasks from original confluence server, then use confluence CLI ( or script runner ) to update those page's taskid, that will trigger the task report be rebuilt only in those pages.

             

            Jack Chen added a comment - I met the problem when migrating another confluence into our main confluence server.   The workaround doesn't work for our instance. I see what it does is deleting existing data in AO_BAF3AA_AOINLINE table, and recreate them from page bodycontent.    This is a dangerous operation, specially for big instances with lots of history. First it will take very long time for the rebuilding job crawl all the pages to identify tasks, second in big instance with old data, there could be many data error introduced by system upgrade/old addons, etc. then the rebuilding job will fail on certain page and never able to finish the task report rebuilding. The log doesn't tell which content is causing the problem: 2019-07-11 10:52:49,874 ERROR [StorageToAoMigrationUpgradeTask-TaskExtractor-1] [plugins.tasklist.upgradetask.InterruptingUncaughtExceptionHandler] uncaughtException Uncaught exception in thread StorageToAoMigrationUpgradeTask-TaskExtractor-1. Interrupting all threads. [com.ctc.wstx.exc.WstxLazyException] com.ctc.wstx.exc.WstxParsingException: Unexpected close tag </xml>; expected </ac:rich-text-body>.     I suggest Atlassian to add more information like the contentid , or make the StorageToAoMigrationUpgradeTask more robust to bypass error pages.   I found a better workaround for space migration: just find all those current pages with tasks from original confluence server, then use confluence CLI ( or script runner ) to update those page's taskid, that will trigger the task report be rebuilt only in those pages.  

            Patrick Schneider added a comment - - edited

            Problem occurs on 6.12.2 too after importing from Confluence 5.5/5.9.
            Workaround fixes the issue.

            Patrick Schneider added a comment - - edited Problem occurs on 6.12.2 too after importing from Confluence 5.5/5.9. Workaround fixes the issue.

            Using 6.1.2.. 

            Note: When the AO_BAF3AA_AOINLINE_TASK is rebuild, all COMPLETE_DATE and UPDATE_DATE values turn to NULL. This is a problem if you have teams utilizing the task report macro. Unfortunately, there is no unique ID to reference in order to copy the dates over either, except CONTENT_ID and ID (in combination) but since CONTENT_ID changes after re-importing a space, you will not be able to get the dates from before the space restore. 

            So... this is a problem. 

            Cinnober Admins added a comment - Using 6.1.2..  Note: When the AO_BAF3AA_AOINLINE_TASK is rebuild, all COMPLETE_DATE and UPDATE_DATE values turn to NULL. This is a problem if you have teams utilizing the task report macro. Unfortunately, there is no unique ID to reference in order to copy the dates over either, except CONTENT_ID and ID (in combination) but since CONTENT_ID changes after re-importing a space, you will not be able to get the dates from before the space restore.  So... this is a problem. 

            This also did not work for me in 5.10.4

            Leann Adams added a comment - This also did not work for me in 5.10.4

            I can only agree with that

            Robert Grillmair added a comment - I can only agree with that

            @mtran@atlassian.com, @mchaudhry and @jturnquist please stop changing the Symptom Severity and look forward to solve the problem and close the issue. It is now 2,5 years old and still open!

            Thanks in advance.

            Chris Engel added a comment - @ mtran@atlassian.com , @ mchaudhry  and @ jturnquist please stop changing the Symptom Severity and look forward to solve the problem and close the issue. It is now 2,5 years old and still open! Thanks in advance.

            is this "bandana" stuff just some dummy tables which you have to replace - or is it a real name in confluence?

            please clearify....

            Mike Barmettler added a comment - is this "bandana" stuff just some dummy tables which you have to replace - or is it a real name in confluence? please clearify....

            Hello,

            the Workaround does not work for me in 5.10.4

            // postgres=# delete from "AO_BAF3AA_AOINLINE_TASK";
            FEHLER:  Relation „AO_BAF3AA_AOINLINE_TASK“ existiert nicht
            ZEILE 1: delete from "AO_BAF3AA_AOINLINE_TASK";
            postgres=# delete from bandana where bandanakey = 'com.atlassian.confluence.plugins.confluence-inline-tasks:build';
            FEHLER:  Relation „bandana“ existiert nicht
            ZEILE 1: delete from bandana where bandanakey = 'com.atlassian.conflu...
            
            

            I also exported and importes a Space. What am I doing wrong?

             

            regards

            Admin Dohle added a comment - Hello, the Workaround does not work for me in 5.10.4 // postgres=# delete from "AO_BAF3AA_AOINLINE_TASK" ; FEHLER:  Relation „AO_BAF3AA_AOINLINE_TASK“ existiert nicht ZEILE 1: delete from "AO_BAF3AA_AOINLINE_TASK" ; postgres=# delete from bandana where bandanakey = 'com.atlassian.confluence.plugins.confluence-inline-tasks:build' ; FEHLER:  Relation „bandana“ existiert nicht ZEILE 1: delete from bandana where bandanakey = 'com.atlassian.conflu... I also exported and importes a Space. What am I doing wrong?   regards

              lgrzyb lukasz
              dnorton@atlassian.com Dave Norton
              Affected customers:
              32 This affects my team
              Watchers:
              45 Start watching this issue

                Created:
                Updated:
                Resolved: