Uploaded image for project: 'Confluence Data Center'
  1. Confluence Data Center
  2. CONFSERVER-63615

Provide workaround for fixing broken unknown attachment placeholder in storage format

    • 280
    • We collect Confluence feedback from various sources, and we evaluate what we've collected when planning our product roadmap. To understand how this piece of feedback will be reviewed, see our Implementation of New Features Policy.

      Atlassian Update - 10 June 2021

      We've released an app for Confluence that will begin to resolve a number of the missing attachment issues being seen by customers. The app attempts to automatically repair attachments, and provides some "quality of life" improvements when attachments cannot be automatically repaired.

      There are currently a number of limitations with the app and it is important that you are aware of these limitations. We have documented the functionality of the app at Unknown attachment error in Confluence Server and Data Center 7.12 and earlier and strongly encourage reading this page in full prior to installing the app.

      Our focus here was on getting a partial fix out to repair as many pages as possible and avoid pages entering an irreparable state when edited. We are capturing additional analytics to understand the impact of this issue and assess the best path forward to providing a broader fix.

      The app can be downloaded from reconcile-unknown-attachments-1.0.6.jar

      We are working to have a Data Center approved version of the app available in coming days, but this app should work for Data Center customers in the interim.

      Currently, unknown attachments issue stands has high User Impact and it has resulted in pages broken on customer sites. This has caused a lot of support and customer pain leading to admins manually fixing pages.

      As an admin, I would like to restore pages/shared-drafts which has unknown-placeholder in creeped into editor format.

      This ticket tracks the fix to address already broken data caused by https://jira.atlassian.com/browse/CONFSERVER-55928.
      This includes only attachments that are broken in storage format due to being replaced with an unknown-attachment placeholder:

      • Broken format in storage for broken pages by "Unknown Placeholder" (Covered by this ticket):
        <p>
         <ac:image>
          <ri:url ri:value="http://localhost:5990/confluence/plugins/servlet/confluence/placeholder/unknown-attachment?locale=en_GB&amp;version=2" />
         </ac:image>
        </p>
        
      • "<ri content-entity> broken links" issue storage format (Covered by https://jira.atlassian.com/browse/CONFSERVER-63700):
        <p>
          <ac:image>
            <ri:attachment ri:filename="image2020-10-21_12-40-38.png">
              <ri:content-entity ri:content-id="138910055"/>
            </ri:attachment>
          </ac:image>
        </p>
        

          Form Name

            [CONFSERVER-63615] Provide workaround for fixing broken unknown attachment placeholder in storage format

            Tobias added a comment -

            jponting, just a quick question about the solution. 
            I have checked some pages with the label "unrestored-unknown-attachment" and have numerous pages that are displayed correctly, so without error message "unknown attachment". What I don't understand is the step in the solution description where I am supposed to reselect the attachment if there is an error. pages that have this error have no attachments at all, so this solution is only partially correct. 
            also the question on my part. When can we uninstall the app, but which version is no longer expected to have the error?

             

            Tobias added a comment - jponting , just a quick question about the solution.  I have checked some pages with the label "unrestored-unknown-attachment" and have numerous pages that are displayed correctly, so without error message "unknown attachment". What I don't understand is the step in the solution description where I am supposed to reselect the attachment if there is an error. pages that have this error have no attachments at all, so this solution is only partially correct.  also the question on my part. When can we uninstall the app, but which version is no longer expected to have the error?  

            This is somewhat crappy, as the label does not disappear?

            1. A User copies a that has attachment, but selects not to have attachments copies with it.
            2. The User saves the page copy to ensure it - and emidiatly its labeled with "unrestored-unknown-attachment"
            3. The User cleans up the page - but the label does not disappear.

             

            Now the label is incorrectly on the page and all "show related content by label" is a lot of other pages with "unrestored-unknown-attachment".

            I have not seen any normal user aware of this behavior removing the label after page clean up...

            Normann P. Nielsen (Netic) added a comment - - edited This is somewhat crappy, as the label does not disappear? A User copies a that has attachment, but selects not to have attachments copies with it. The User saves the page copy to ensure it - and emidiatly its labeled with "unrestored-unknown-attachment" The User cleans up the page - but the label does not disappear.   Now the label is incorrectly on the page and all "show related content by label" is a lot of other pages with "unrestored-unknown-attachment". I have not seen any normal user aware of this behavior removing the label after page clean up...

            Hi All,

            Just another quick update.

            672f90c65081 - I forgot I'd followed up on this already with the developers; we're already setting the preview of the images back to the default size if there's no sizing information available. Unfortunately when the link to the attachment is lost, we lose the sizing data as well. In some cases we can recover, in some we can't. The previews should be the default size if they have to be restored. It will require manual intervention to change from the default.

            The analytics from the app are indicating we're covering all the use cases we're aware of, and that we're reconciling all of the attachments that can be reconciled. The remainder of cases show that the metadata was lost and we're unable to reconcile the attachment automatically. This is unfortunately expected as a large number of pages were already unreconcilable before the release of the app. To try and address these pages we've implemented an automatic labelling function. Any page that contains an attachment that cannot be automatically repaired will have the unrestored-unknown-attachment label added. This allows users and administrators to find impacted pages and to repair them over time.

            The basic workflow would be

            1. In Confluence, perform a label search for unrestored-unknown-attachment
            2. Open a page tagged with this label
            3. Edit the page, and click any of the Unknown Attachments to open the attachment selector screen
            4. Select the attachment you want to associate with the Unknown Attachment
            5. Save the page
            6. Check the attachment is appearing correctly
            7. Remove the label from the page

            One note, the functionality to repair pages is "on demand" meaning that every page load or edit results in that page being checked to see if there's an attachment that can be repaired. Over time you may find new pages with the associated label that were previously broken, and have only been checked recently resulting in the label being added. We do not expect to see any pages created on versions of Confluence with the fix implemented tagged with this label.

            With this in mind, we're going to mark this ticket as closed. We'll continue to monitor the analytics to ensure no new use cases emerge, and that we don't see a regression in success rates.

            Thanks,
            James Ponting
            Premier Support Engineer

            James Ponting added a comment - Hi All, Just another quick update. 672f90c65081 - I forgot I'd followed up on this already with the developers; we're already setting the preview of the images back to the default size if there's no sizing information available. Unfortunately when the link to the attachment is lost, we lose the sizing data as well. In some cases we can recover, in some we can't. The previews should be the default size if they have to be restored. It will require manual intervention to change from the default. The analytics from the app are indicating we're covering all the use cases we're aware of, and that we're reconciling all of the attachments that can be reconciled. The remainder of cases show that the metadata was lost and we're unable to reconcile the attachment automatically. This is unfortunately expected as a large number of pages were already unreconcilable before the release of the app. To try and address these pages we've implemented an automatic labelling function. Any page that contains an attachment that cannot be automatically repaired will have the unrestored-unknown-attachment label added. This allows users and administrators to find impacted pages and to repair them over time. The basic workflow would be In Confluence, perform a label search for unrestored-unknown-attachment Open a page tagged with this label Edit the page, and click any of the Unknown Attachments to open the attachment selector screen Select the attachment you want to associate with the Unknown Attachment Save the page Check the attachment is appearing correctly Remove the label from the page One note, the functionality to repair pages is "on demand" meaning that every page load or edit results in that page being checked to see if there's an attachment that can be repaired. Over time you may find new pages with the associated label that were previously broken, and have only been checked recently resulting in the label being added. We do not expect to see any pages created on versions of Confluence with the fix implemented tagged with this label. With this in mind, we're going to mark this ticket as closed. We'll continue to monitor the analytics to ensure no new use cases emerge, and that we don't see a regression in success rates. Thanks, James Ponting Premier Support Engineer

            Hi All,

            Just a couple of bits and bobs.

            marco.leist645132732 - I'm a little late reply to all these, but the Confiforms incompatibility is now fixed (for anyone who comes across this comment now )

            672f90c65081 - I've passed the feedback onto the developers to see if there's anything we can do to improve on this. Sorry for the belated reply!

            henning.karl - We're currently investigating this issue together, and our current findings indicate that the issue is unrelated to the App in question. The App is still good to install

            We're also pushing an update to the Reconcile Unknown Attachments plugin today that adds functionality to add a label to a page when an Unknown Attachment is found and can't be auto repaired.

            Using this label, Confluence administrators and users can search by label to find impacted pages that need manual intervention to correct. Once corrected, simply remove the label to complete the repair.

            The app is pre-installed in Confluence 7.13.0, and can be updated in the Manage Apps panel on the Confluence Administration page. To install it for the first time, you can search for Reconcile Unknown Attachments in the Find New Apps panel.

            Thanks,
            James Ponting
            Premier Support Engineer

            James Ponting added a comment - Hi All, Just a couple of bits and bobs. marco.leist645132732 - I'm a little late reply to all these, but the Confiforms incompatibility is now fixed (for anyone who comes across this comment now ) 672f90c65081 - I've passed the feedback onto the developers to see if there's anything we can do to improve on this. Sorry for the belated reply! henning.karl - We're currently investigating this issue together, and our current findings indicate that the issue is unrelated to the App in question. The App is still good to install We're also pushing an update to the Reconcile Unknown Attachments plugin today that adds functionality to add a label to a page when an Unknown Attachment is found and can't be auto repaired. Using this label, Confluence administrators and users can search by label to find impacted pages that need manual intervention to correct. Once corrected, simply remove the label to complete the repair. The app is pre-installed in Confluence 7.13.0, and can be updated in the Manage Apps panel on the Confluence Administration page. To install it for the first time, you can search for Reconcile Unknown Attachments in the Find New Apps panel. Thanks, James Ponting Premier Support Engineer

            Since we updated Confluence DataCenter (single-node) to 7.12.3 and install the Reconcile Unknown Attachments - Atlassian Marketplace we have some other errors/bugs in our system.
            The update and app installation was nescessary because we have more than 500 affected pages from bug the following
            https://jira.atlassian.com/browse/CONFSERVER-55928
            https://jira.atlassian.com/browse/CONFSERVER-63615
            By editing somes page the complete content is doubled or tripled in there automatically.
            We suspect that this is triggered by the Reconcile Unknown Attachment App.

             

            I also created a Support-Ticket PSSRV-18810

            Perhaps others have the same problem with them as well.

            Kind regards,
            Henning

            Henning Karl added a comment - Since we updated Confluence DataCenter (single-node) to 7.12.3 and install the  Reconcile Unknown Attachments - Atlassian Marketplace  we have some other errors/bugs in our system. The update and app installation was nescessary because we have more than 500 affected pages from bug the following https://jira.atlassian.com/browse/CONFSERVER-55928 https://jira.atlassian.com/browse/CONFSERVER-63615 By editing somes page the complete content is doubled or tripled in there automatically. We suspect that this is triggered by the Reconcile Unknown Attachment App.   I also created a Support-Ticket PSSRV-18810 Perhaps others have the same problem with them as well. Kind regards, Henning

            Short feedback: Fix works in principle, but images usually have a wrong size afterwards.

            Marco Leist added a comment - Short feedback: Fix works in principle, but images usually have a wrong size afterwards.

            Marco Leist added a comment - - edited

            Hi All,

            after we updated Confluence to 7.4.9 and installed the reconcile-unknown-attachment repair-app https://marketplace.atlassian.com/apps/1225406/reconcile-unknown-attachments, the installed App Confiforms stopped to work and showed the error:

            "Error rendering macro 'confiform'
              [com.ctc.wstx.exc.WstxLazyException] Undeclared namespace prefix "ac" at [row, col] unknown source: [1,189]"

            Thanks to google, i've found the following comment, where this behavior already had been mentioned:
            https://jira.atlassian.com/browse/CONFSERVER-25507?focusedCommentId=2802070&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-2802070

             

            As an immediate measure, I deactivated the Atlassian repair app again because Confiforms is used massively in our company and disabling it is not an option.

            Is this bug already known by Atlassian?
            A soon fix would be very important for us, as we strongly rely on Confiforms but at the same time have a great need to fix the (hundreds of) broken attachments in our system.

            Thanks in advance, Marco

            Update 21-07-06: Atlassian has now created a new issue for this: https://jira.atlassian.com/browse/CONFSERVER-66518

            Marco Leist added a comment - - edited Hi All, after we updated Confluence to 7.4.9 and installed the reconcile-unknown-attachment repair-app  https://marketplace.atlassian.com/apps/1225406/reconcile-unknown-attachments,  the installed App  Confiforms  stopped to work and showed the error: "Error rendering macro 'confiform'    [com.ctc.wstx.exc.WstxLazyException]  Undeclared namespace prefix "ac" at  [row, col]  unknown source:  [1,189] " Thanks to google, i've found the following comment, where this behavior already had been mentioned: https://jira.atlassian.com/browse/CONFSERVER-25507?focusedCommentId=2802070&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-2802070   As an immediate measure, I deactivated the Atlassian repair app again because Confiforms is used massively in our company and disabling it is not an option. Is this bug already known by Atlassian? A soon fix would be very important for us, as we strongly rely on Confiforms but at the same time have a great need to fix the (hundreds of) broken attachments in our system. Thanks in advance, Marco Update 21-07-06: Atlassian has now created a new issue for this:  https://jira.atlassian.com/browse/CONFSERVER-66518

            James Ponting added a comment - - edited

            Hi All,

            Updates!

            We've released an app for Confluence that will begin to resolve a number of the missing attachment issues being seen by customers. The app attempts to automatically repair attachments, and provides some "quality of life" improvements when attachments cannot be automatically repaired.

            There are currently a number of limitations with the app and it is important that you are aware of these limitations. We have documented the functionality of the app at Unknown attachment error in Confluence Server and Data Center 7.12 and earlier and strongly encourage reading this page in full prior to installing the app.

            Our focus here was on getting a partial fix out to repair as many pages as possible and avoid pages entering an irreparable state when edited. We are capturing additional analytics to understand the impact of this issue and assess the best path forward to providing a broader fix.

            The app can be downloaded from Reconcile Unknown Attachments - Atlassian Marketplace

            We are working to have a Data Center approved version of the app available in coming days, but this app should work for Data Center customers in the interim.

            I'll continue to update the comment in the issue description as we obtain additional info.

            Thanks,
            James Ponting
            Premier Support Engineer

            James Ponting added a comment - - edited Hi All, Updates! We've released an app for Confluence that will begin to resolve a number of the missing attachment issues being seen by customers. The app attempts to automatically repair attachments, and provides some "quality of life" improvements when attachments cannot be automatically repaired. There are currently a number of limitations with the app and it is important that you are aware of these limitations. We have documented the functionality of the app at Unknown attachment error in Confluence Server and Data Center 7.12 and earlier and strongly encourage reading this page in full prior to installing the app. Our focus here was on getting a partial fix out to repair as many pages as possible and avoid pages entering an irreparable state when edited. We are capturing additional analytics to understand the impact of this issue and assess the best path forward to providing a broader fix. The app can be downloaded from Reconcile Unknown Attachments - Atlassian Marketplace We are working to have a Data Center approved version of the app available in coming days, but this app should work for Data Center customers in the interim. I'll continue to update the comment in the issue description as we obtain additional info. Thanks, James Ponting Premier Support Engineer

            Hi All,

            I just wanted to provide an update on this ticket.

            In terms of timeline, I don't have a date to share at this point. As with the other ticket, I'll provide updates and information as soon as it becomes available. To shed a little more light on progress though, we have progressed sufficiently in the fix to begin initial testing, but identified some considerations we need to address. This work is very much in progress, and we'll keep at it.

            gerrit.berberich1398982012 - My apologies for the belated reply here, I simply didn't see your comment previously. We have updated CONFSERVER-55928: Attachments become 'Unknown Attachment' in the page editor with Collaborative Editing turned on (fixed in Confluence Server and Data Center 7.4.9, 7.12.1, 7.11.3) as versions are released with the fix. We've split our focus here to prioritise stopping new occurrences of the issue, and then here to fix existing occurrences. Both problems have proven complex, but we're working on it.

            I hope to have more to share with you soon.

            Thanks,
            James Ponting
            Premier Support Engineer

            James Ponting added a comment - Hi All, I just wanted to provide an update on this ticket. In terms of timeline, I don't have a date to share at this point. As with the other ticket, I'll provide updates and information as soon as it becomes available. To shed a little more light on progress though, we have progressed sufficiently in the fix to begin initial testing, but identified some considerations we need to address. This work is very much in progress, and we'll keep at it. gerrit.berberich1398982012 - My apologies for the belated reply here, I simply didn't see your comment previously. We have updated CONFSERVER-55928: Attachments become 'Unknown Attachment' in the page editor with Collaborative Editing turned on (fixed in Confluence Server and Data Center 7.4.9, 7.12.1, 7.11.3) as versions are released with the fix. We've split our focus here to prioritise stopping new occurrences of the issue, and then here to fix existing occurrences. Both problems have proven complex, but we're working on it. I hope to have more to share with you soon. Thanks, James Ponting Premier Support Engineer

            Please just fix this issue. This has really high impact and it's causing a lot of frustration!! I cannot understand the price increases even though such bugs cause a lot of pain... 

            Kemal Tuncelli added a comment - Please just fix this issue. This has really high impact and it's causing a lot of frustration!! I cannot understand the price increases even though such bugs cause a lot of pain... 

              oshen@atlassian.com Oliver Shen
              ggautam Ganesh Gautam
              Votes:
              117 Vote for this issue
              Watchers:
              140 Start watching this issue

                Created:
                Updated:
                Resolved: