Hi All,

Just a quick update on the work on CONFSERVER-63615: Provide workaround for fixing broken unknown attachment placeholder in storage format

We're pushing an update to the Reconcile Unknown Attachments plugin today that adds functionality to add a label to a page when an Unknown Attachment is found and can't be auto repaired.

Using this label, Confluence administrators and users can search by label to find impacted pages that need manual intervention to correct. Once corrected, simply remove the label to complete the repair.

The app is pre-installed in Confluence 7.13.0, and can be updated in the Manage Apps panel on the Confluence Administration page. To install it for the first time, you can search for Reconcile Unknown Attachments in the Find New Apps panel.

Thanks,
James Ponting
Premier Support Engineer

Hi All,

Just a heads up that Confluence 7.13.0 has released today with the fix for this issue included.

Thanks all for your patience and understanding as we worked through this issue.

Regards,
James Ponting
Premier Support Engineer

Hello together, 

the app also works for our environment. We use Confluence Server Version 7.4.9. Thanks for providing this app to us, it is really helpful.

But just allow me to aks one question: Will this bug be fixed in an upcoming release or don't you focus on this because your customers have to use the app permanently? 

Thanks for your help.

Regards,
Tobias

Hi James,

thank you for the detailed answer and the query. As it gives no results, we will not need to install the Unknown Attachment App. So we have to wait until the other bug is fixed in the current LTS-version.

Regards,
Michael

Hey All,

I just so happen to have the answer to a question here, so I'll provide it here in the hope it helps others.

29e0d95bce1d - To your questions

1. The plugin won't run SQL to find affected pages as it would require a very expensive regex query that would check every single page in the instance. To avoid this, we've taken an "on-demand" approach which will check pages when they're edited and the placeholder is created. If you would like to check for pages that are impacted, you can try the following query in Postgres (edit <CONFLUENCE_URL> to your Confluence site's URL). I would recommend running this query in a staging environment first to understand how long the query may take and its resource impact. I would strongly recommend running this during a quiet period if it is subsequently run on production. As below
   Identify Unknown Attachment Pages - Postgres Query

   SELECT c.contentid,
          c.contenttype,
          c.title,
          s.spacekey,
          CONCAT('<CONFLUENCE_URL>/pages/viewpage.action?pageId=', c.contentid) AS URL
   FROM CONTENT c
   JOIN BODYCONTENT bc ON c.contentid = bc.contentid
   JOIN SPACES s ON c.spaceid = s.spaceid
   WHERE c.prevver IS NULL
       AND c.contenttype IN ('PAGE',
                             'BLOGPOST')
       AND bc.body LIKE '%/placeholder/unknown-attachment?%';
   

   This will give you a list of pages including links to the pages that you can follow.

2. Unfortunately I don't believe so. From reviewing the developer notes on the issue, I understand that will be addressed by code in Confluence 7.13.0 when it releases

I hope this helps

Thanks,
James Ponting
Premier Support Engineer

Hi James,

Thanks for the detailed explanation. This is very enlightening.

Cheers,
Daniel

Hi,

tanks for the Update and the addon. Is there a SQL-Query to get the pages that the addon will try to repair? We are currently on 7.4.9 and according to the support, we are primary affected by this bug: https://jira.atlassian.com/browse/CONFSERVER-60526 

So I have to questions: 
1. Is there a sql-query, that can show us the affected pages the addon will touch?
2. Does the addon also repair the pages affected by the bug liked above?

Regards,
Michael

Hi All,

I just want to provide an update on our work to provide a fix for existing broken pages, and address the comments above.

Unknown Attachment App

Firstly, we are tracking our work to resolve the issue for pages that are already broken at CONFSERVER-63615: Provide workaround for fixing broken unknown attachment placeholder in storage format. We have just provided an update on that issue detailing an app that is available to resolve some of the broken attachments. I'd encourage you to carefully review the documentation for the app prior to installing it.

If you'd prefer to go straight to the documentation (which includes a link to the app), you can head directly to Unknown attachment error in Confluence Server and Data Center 7.12 and earlier

Understanding the Fix

With regards to the conversation about the efficacy of this fix above involving saikrishna.patel, dbd206fd02ab, and ggautam I would like to explain some things to help understand the fix.

Confluence uses a system called Synchrony to allow collaborative editing of pages by multiple users. When a user edits a page, Confluence (creates if necessary, then) passes a draft to Synchrony which handles the editing of the page. Changes made in Synchrony are sent back to Confluence on a timer (to make sure we don't lose data), and when the editor is closed or the page is published. The issue being tracked here occurs when Confluence passing data back and forwards with Synchrony. This fix stops newly created drafts from being impacted by this issue.

The problem then is that there can be drafts that were created prior to this implementation of this fix that are still impacted. This would give the appearance of the bug appearing for the first time, but in reality the bug has already impacted the page, it's just not visible yet. This is, unfortunately, expected behaviour with this fix, but it is something we can help repair. In cases where this issue is becoming visible for the first time, the app mentioned above will help you resolve this issue.

Given the impact this bug was having on our customers, we split the work of fixing the bug, and repairing impacted pages into two streams to try and get a fix out as soon as possible. This is why the solution to repair impacted pages is not part of this fix and not built into Confluence (yet).

At this time, we understand that the issue is fixed. Our testing has reflected this, and we've been receiving confirmation from impacted customers that they're no longer seeing new occurrences of this issue.

If you are continuing to see a related problem, please open a ticket with support at https://support.atlassian.com and we'll help you look into the issue.

Other Question

To address the other query I can see
58846cf653c8 - There is no fix available for Confluence 7.9.x, however the app that attempts to repair impacted pages can be installed to mitigate the issue (it will try and repair the pages as they break). Even with this app, I would recommend upgrade to Confluence 7.12.2 (the fix is in 7.12.1) to receive the fix.

Final Notes

The development team and I want to extend our thanks for all the data and troubleshooting information that was provided by various people on this ticket and in related support tickets.

Thank you also for your patience and understanding as we worked through this one. We know these issues have a massive impact on you and your users, and we've been working hard on these issues to get them fixed.

Happy editing

Regards,
James Ponting
Premier Support Engineer

Hi saikrishna.patel

> we have tested a page which does not have any broken links or attachments, when we tried to edit the page in Confluence 7.4.9 we saw again "Unknown attachments" error like we used to see previously.

GG - I remember the pages you shown earlier and would like to explain why. Its because of already broken draft and not a new breakage.
Unfortunately when an attachment event was not sent to Synchrony, drafts first get broken as they are auto-saved. You can see broken attachment on the pages where drafts are already broken. I would recommend you to try the replication steps I mentioned in above comment to be sure that the pages are not broken anymore.

Please watch CONFSERVER-63615 for the workaround issue to fix broken content.

Thanks, 
Ganesh

Hi Ganesh,

Thanks for the update. After upgrading confluence from  on 7.4.7 to 7.4.9on  June 4th we have tested a page which doesnot have any broken links or attachments. Before editing the page in confluence 7.4.9 we didnt saw any "Unknow Attachment" error in confluence page. When we tried to edit the page in Confluence 7.4.9 we saw again "Unknown attachments" error like we used to see previously. We did updated the page and in updated page we see the same issue/error again.

Said so even in 7.4.9, fix for this issue didnt stopped for new occurrences of broken pages and attachments.

Thanks & Regards,

Sai

Hi Saikrishna, Daniel, Trevor

Thanks for reaching out.

I would like to highlight that any page whose draft is already broken is not fixed by this issue's fix. The fix for this issue stops new occurrences of broken pages and attachments.

Since this bug used to break the storage format in page entity and draft content themselves, you might still see already broken pages as broken even after the upgrade till the time the workaround for broken pages is put in place.

Please track CONFSERVER-63615 for incoming workaround which will help you in fixing existing broken pages(please see description of this ticket for more info on this) in your installation.

dbd206fd02ab, Yes we confirm that the issue was partially reproducible with 7.4.7 and all new tests for unknown attachment(for unknown placeholder) in our suites pass with 7.4.9.
We have tests in our system which execute the replication steps in many flavours and they pass after the current fix. I would recommend to perform same sanity in your system. Following steps can help you with it: 

1. Ensure Collaborative Editing is switched on.
2. Create a page with this storage format.

   <p><ac:image ac:height="250"><ri:attachment ri:filename="test.jpg"/></ac:image></p>
   

1. The page displays "Unknown attachment".
2. Via '...' > 'Attachments', just insert an attachment 'test.jpg' into the page. You'll see that the page now contains 1 attachment, which is 'test.jpg'.
3. View the page you first created and the image is now there.
4. Edit the page. The image should be there.

Thanks,

Ganesh

We have this bug in our 7.9.1 installation. I can't tell – did the work in this ticket address the 7.9.x series?

We were just in the process of updating. This is very disappointing.
We will carefully check whether this bug is actually fixed for us or not with 7.4.9.
From the way things look these days, worst case scenario is that it is not fixed, and we get some other issues with 7.4.9 compared to 7.4.7....

Atlassian, are you able to reproduce the issue on your end with 7.4.7 to confirm that it is actually fixed in 7.4.9?

Hi Team,

We upgraded confluence to 7.4.9 from 7.4.7 and able to reproduce the issue again for some pages. Bug still exists in 7.4.9.

Excellent. Thanks for the update.

A fix for this issue is available to Server and Data Center customers in Confluence 7.4.9
Upgrade now or check out the Release Notes to see what other issues are resolved.

Hi All,

Updates as promised.

We've just begun the testing process leading into the release of Confluence 7.4.9. As always, we can't commit to time frames, but assuming we don't find any additional issues, I would hope to see the release some time this week. This hope is obviously reliant upon all of our testing passing without problems. If we find problems, we're going to have to address them before we can release. I just want to let you know things are progressing and that hopefully we can release will be soon.

As always, please review the release notes for Confluence 7.4.9 when they become available.

dbd206fd02ab - No worries

e6907d9111f9 - There is not. Assuming you're already on Confluence 7.4.6, I would recommend targeting Confluence 7.4.9 when it releases.

As always, I'll let you know as new information becomes available.

Thanks,
James Ponting
Premier Support Engineer

James,

Is there a full fix for version for 7.4.6?

Hi James,

Got it. Thanks for the update.

Cheers,
Daniel

Hi All,

We now have Confluence 7.11.3 and 7.12.1 out with the fix. We're continue to work on Confluence 7.4.9, and I will provide an update as soon as we have information on that release.

dbd206fd02ab - Not at this time. We'll let you know as soon as we do.

admin1854195196 - This release only stops new occurrences of the bug. We're working on a seperate line of work to fix pages that are already impacted by this issue. That work is being tracked on CONFSERVER-63615: Provide workaround for fixing broken unknown attachment placeholder in storage format. Can you please confirm if you're seeing new occurrences of the bug, or if it's previously affected pages still showing the issue? We are expecting pages that were previously affected by this issue to still be effected.

If anyone has installed the updated version of Confluence, and is still able to reproduce the issue, please open a support ticket at https://support.atlassian.com and we'll begin looking into the issue.

As it stands, we believe the issue is resolved, though we will continue to monitor responses here and on our support platform for any hint otherwise.

Thanks,
James Ponting
Premier Support Engineer

Hi

yesterday i updated to v7.12.1 and the problem still exists with previously created pages.

@james only on previously created pages, no new occurrences so far.

That's great. Do you know the timeline for a fix for version 7.4.9?

A fix for this issue is available to Server and Data Center customers in Confluence 7.12.1
Upgrade now or check out the Release Notes to see what other issues are resolved.

Hi All,

Just providing some updates.

f539b529cd9d - No worries at all. I understood what you meant, I just wanted to make sure it was clear for others without the context we had. We definitely do appreciate such suggestions. Apologies for the belated reply, just trying to avoid spamming everyone by replying to individual comments. I'm going to touch up your previous bounce and add a note to it to see your most recent comment.

76f40437545f - We can't provide you with a date currently. We're continuing to work on Confluence 7.4.9 including multiple high priority fixes, though I understand there is only one outstanding item at this time. Once that item is resolved, we will finalise testing and provide the release. We're trying to walk that fine line of getting fixes out rapidly, and not introducing new issues to our Long Term Support releases. Once I have a release date, I'll include that information for you all to begin planning.

In the meantime, as Confluence 7.11.3 was ready to be shipped so we have released it with the fix for anyone who is in a position to upgrade to it. We're continuing to work on getting the other noted Fixed Versions released, and will continue to provide updates as they're available.

EDIT - I've also re-opened the ticket as it was erroneously marked as closed by our automation. It will not be considered closed until all the fixed versions are released. Apologies for the confusion.

Thanks,
James Ponting
Premier Support Engineer

I don't know if you noticed, 7.4.9 is still not available. Can you give us a date?

A fix for this issue is available to Server and Data Center customers in Confluence 7.11.3
Upgrade now or check out the Release Notes to see what other issues are resolved.

Hi James,

thanks for your feedback. Me referring to "this issue" might be confusing. We get sent plenty pages with "Unknown Attachments" where the reasoning might not always be the bug described in this issue. The query might be worth checking out if you aren't certain the bugged attachment is a result of this issue.

Best regards,

Alexander Puhla

Hey f539b529cd9d,

That's query will certainly work in the scenario you've described.

Unfortunately the issue in this ticket is different to the one you've encountered. That being the case, the above query will not work for the users affected by this bug.

Thank you for sharing your findings though, we always appreciate work arounds identified by our customers

Thanks,
James Ponting
Premier Support Engineer

We found a different approach to resolve this issue for two of our newest occurances. It seemed like the pages were referencing an attachment from a deleted page. We located the page by the following query:

SELECT t2.title AS Title,
       t2.contentid AS PageId,
       t3.spacename AS Spacename,
       t3.spacekey AS Spacekey
FROM
    (SELECT *
     FROM content
     WHERE CONTENTTYPE = 'ATTACHMENT'
         AND title LIKE '%ATTACHMENT-NAME%') t1
JOIN
    (SELECT *
     FROM content
     WHERE CONTENTTYPE = 'PAGE') t2 ON (t1.pageid = t2.contentid)
JOIN spaces t3 ON (t2.spaceid = t3.spaceid)

We restored the deleted page from the bin, downloaded the attachments and added them to the page with the broken attachments. You might wanna give it a try, if the other workarounds don't work for you.

Atlassian Note: f539b529cd9d provides some additional context on this query below - jponting

Hi All,

Just reply to a couple of comments

application.support2 & saikrishna.patel - We don't currently have a release date to share. We're actively working on these releases as both have critical bug fixes in them (including this) that we need to make sure we get to you all. Getting these fixes out to you all is our top priority. When I have more information, I'll let you know.

2b0415d20a38 - We're currently working on methods to automate fixing this issue. We're tracking that work at CONFSERVER-63615: Provide workaround for fixing broken unknown attachment placeholder in storage format. Please add yourself as a watcher on that issue to receive updates as we have them.

brudy - Assuming that behaviour is caused by this bug, then no. This fix will only stop future occurrences. For more information our work to fix existing occurrences of the bug, please see the above link.

As I get new information that is relevant to this issue and those of you impacted by it, I'll continue to provide it here.

Thanks,
James Ponting
Premier Support Engineer

Will this also fix the bug when I have an image successfully attached to a page that worked fine before, only to go back to it later and get the "unknown attachment" error?  

Are we expected to scan all our confluence pages and use workarounds to restore images that were perfectly fine before?

When you will release 7.4.9?

When will you finally release 7.11.3?

wow,me too.
 

We got this bug as well ! in many pages

This is shameful - you should be ashamed, Atlassian. The workarounds do not work for me. Many hours lost. Why am I not using a freeware Wiki, why are we paying for this?

Totally agree with Aleksei. This bug should be fixed ASAP and once for all. It's totally unacceptable to have a bug opened with the Highest priority for almost 3 years without any permanent solution. This approach is totally unprofessional and because of this bug, our documentation is devalued. It means we can not provide our customers with all comprehensive information and features. There are so many issues with Confluence that we are really reconsidering the usage of Atlassian products in the future. 

Hi All,

Just providing an additional update on this one.

We have a fix targeting Confluence 7.4.9 and Confluence 7.11.3, both of which will release in coming days.

Unfortunately we were unable to get the fix into Confluence 7.12.0 prior to release, so we're looking at Confluence 7.12.1 as a candidate on the 7.12 branch. This should result in a fix in the next LTS release which should be Confluence 7.13.0.

To be clear, the fix being merged should stop new occurrences of this bug appearing, however it will not fix pages already affected by this issue.

With that fix in flight and being released to customers, we're now tracking work on correcting already impacted pages at CONFSERVER-63615: Provide workaround for fixing broken pages/shared-drafts because of "Unknown Attachments".

To be clear, whilst the goal is to have some option that just corrects the issue for impacted customers, there may be some cases wherein automatic correction is not possible, and manual intervention may be required. Hopefully we can find a one click solution, but I want to set expectations early and clearly.

Thank you again for your continued patience with this issue.

Thanks,
James Ponting
Premier Support Engineer

Confluence 7.4.7 LTS does not fix it either. We can still see it sporadically appearing on 2 independant Confluence Server instances we operate (both 7.4.7 LTS).

In terms of troubleshooting the only way that helps is:

• revert the page to the last known good version
• re-do all editing steps of the last page update resulting in the faulty page using the pairwise comparision in the history as template

Atlassian Confluence 7.10.0 still has this issue and we noticed a page where the edit recently happened (on Mar 02, 2021).

Reach out if we can help with logs/exports ...

In our organization this problem has arisen after editing pages in which the images already existed. In some cases, they have become 'Unknown attachment'.

We are using Confluence 7.4.5 LTS.

Suggested workaround option 1 is not working

This issue is reported in 7.9.0 version. Performed the below steps and was able to see all my PNG images back.

Option 2 - For Confluence 6.9 and up

1. Open the editor for the affected page
2. Click on the ellipsis button on the bottom right corner of the screen (besides the Close button)
3. Click on revert back to the last published version
4. Check if the attachment appears on the editor now

Hi, same here for Confluence 7.7.4

This reported on 7.8.1 as well.

This was reported by my colleagues,too. We use 

Confluence 7.4.7

Affects 7.8.3 too

Hello,

we are also affected by the bug with almost 500 pages of about 90 areas.

And have a ticket open with Atlassian CSP-289338, but here we are not offered any real help either.
We are supposed to rework all pages by hand to have working pages again. That is an extremely big effort to fix them all manually with the workaround.

The workaround when rolling back page versions is nothing but a workaround, because here I have to maintain the content from the latest page version as well, otherwise I will have an even bigger data loss.
The alternative to rollback page version is to reattach the attachments. But even that is not goal oriented, because often the images don't even exist to reattach them.

So I expect that Atlassian must support us in some way.

I have checked the first two pages from the list of affected pages.
But on the first page, the image is already showing as Unknown Attachment in the first version.
So nothing can be restored here, which is totally unacceptable.

On the second page I found a version of the page where the images are still there correctly.
But here I have to find all the data from the last version and then paste it into the restored version.

For each affected page, it takes about 5-10 minutes to restore the entire site, and with 449 pages, that's one to two weeks of work for one person.

Atlassian should also be aware of this and be willing to assist or compensate us in some way.
Add to that the loss of data for some pages that cannot be recovered.

Can you develop a "Migrate two page versions to one version" app that allows the user to select which version the part can be taken from after each paragraph and for tables each cell.

This would be a solution to really help everyone who has this error. Since would still have to edit each page by hand, but that would significantly reduce the processing time.

Also, I've already reported via comment and in our other ticket that we're affected with version 7.5.1, but again, that's not being reasonably updated in this ticket. For us is also not clear until when a fix for the bug is available, that this can not happen again with pages.

We from ebm-papst see here Atlassian quite clearly in the duty to do more.

Yours sincerely,
Henning

This also affects version 7.8.1

We are also affected by the problem on some pages in different spaces.
In Confluence version 7.5.1

Nice. Thanks for the info, Amanda!