When an anonymous user is creating a ticket and tries to attach a file to it, it gets the following error message:
Cannot attach file: Unknown server error (404).

The network response message is the following:
No project could be found with id '10103'.

This happens because in the project permissions, although the group 'Anyone' has permissions to 'Create Issues' and 'Add Attachments', it does not have permissions to 'Browse Projects'.

Unless there is a workaround, it seems that this functionality to add attachments anonymously is missing.

Variants

It seems that the project could not be found error will be thrown to users without system administrators permission and it is misleading since the real issue is about having Browse Project permission.
Users with system administrator(Global Permission) permission will receive:
Cannot attach file <filename>: You must have the browse project permission to view this project.