I forgot, that the filename column in the fileattachment table is still required to display a filename in JIRA and to send it to the client when downloading. But again, I'd appreciate to get rid of the filenames on disk.

I'd really much appreciate this issue to be resolved in 4.3. We often have problems with badly encoded filenames within jira. Why not just drop the filename column in the fileattachment table and rename all attachments to the ID ?

This change has one unfortunate change:

Current JIRA assumes that "filename on disk" == "filename to display in URL". Obviously there's no reason for that to be true but that's what things like the DefaultThumbnailManager assume. This means that certain URLs (for thumbnails and in the issue navigator) are now just the ID. i.e.

old: /thumbnail/10000/10000_cool_picture.jpg
new: /thumbnail/10000/10000

Back in the ancient days of the web there were browsers that didn't like those kinds of URLs for media. Things are much better nowadays. (Despite the bland URL, browsers will prompt you to save the file with the "correct" name, because we send along all the proper mime headers.) So while the URL is as nice as we'd like, we'll live with it. Fixing this would involve changes to the Thumber, which lives in atlassian-core, and would need to be coordinated with Confluence (and any other products that use the Thumber in atlassian-core).

I think we should also consider biting the bullet and fixing up all existing attachments in an upgrade task. This would mean we could move the current "handle legacy attachment naming" from AttachmentUtils into an upgrade task. We wouldn't be able to fix broken attachments but at least we could generate a list of them and point the customer to a KB article explaining how to fix it outside of JIRA. If we decide to do this, we should also segment the directory hierarchy at the same time (to handle the 32,000+ subdirectory problem). That way there is only a single upgrade task.

We've been afraid that doing this would be a big head ache but Confluence did it for their 3.0 release. I talk to Confluence's support team and they said they haven't had any real support problems from it. That makes me think we are overestimating how many people ever look at the attachment directory in JIRA.

Since I've been unable to reproduce any of the normalization stuff I'm still uncertain how that would actually manifest in the real world. However, the current implementation – which no longer uses the original file name in any way, shape, or form – should be immune from any such things.

We are still left with the problem of what to do with existing attachments. Some kind of upgrade task would be best in an ideal world but the risks (what if the customer has hundreds of thousands of attachments? does it break their incremental backups? etc) outweight the gains. So we'll continue to live with multiple legacy ways attachments can appear.

Thank you for looking this up.

Maybe JIRA should include a manual task which will fix the file names in case of problems ?

Here the perl scripts I have written for our case.

#!/usr/bin/perl
use utf8;
use strict;
use DBI();
use Unicode::Normalize;

my $db = DBI->connect("DBI:mysql:database=jira;host=localhost;port=12034;mysql_socket=/tmp/mysql-jira-prd.sock",
        "jira", "password", {'RaiseError' => 1});

my $stmt = $db->prepare("select p.pkey, i.pkey, f.id, f.FILENAME
         from jiraissue i, fileattachment f, project p
         where i.ID = f.issueid and i.project = p.id");

$stmt->execute();
while (my @row = $stmt->fetchrow_array()) {
#   print "Proj = $row[0], id = $row[1], attch id = $row[2], filename = $row[3]\n";
        my $filename = "/ec/prod/server/citnet/jira/data/jira/data/attachments/"
                 . $row[0] . "/" . $row[1] . "/" . $row[2] . "_" . NFC($row[3]);
        if (!-e $filename) {
                print "$filename\n";
        }
}
$stmt->finish();


$db->disconnect();

#!/usr/bin/perl
use utf8;
use strict;

while (<STDIN>) {
        chomp;
        if (!-e $_) {
                my $folder = substr($_, 0, rindex($_, "/"));
                my $filename = substr($_, rindex($_, "/") + 1);
                my $fileprefix = substr($filename, 0, index($filename, "_"));


                chdir $folder;
                my @fsfiles = glob "$fileprefix*";
                if (scalar(@fsfiles) < 1) {
                        print "FILE NOT FOUND:$_\n";
                } elsif (scalar(@fsfiles) > 1) {
                        print "MULTIPLE FILES WITH SAME ID:$_\n";
                } else {
                        print "RENAMING:$_";
                        print " DB filename [$filename] FS filename [$fsfiles[0]]\n";
                        rename $fsfiles[0], $filename;
                }
        }

        if (!-o $_) {
                print "NOT OWNED BY STUDIO:$_\n";
        }
}

Result from first script can be piped to the second script. For some reason I don't understand, I had to cut the script in those two parts. Those scripts handle only UTF-8 locale.

Be sure to launch it with perl params -CSDL

Please ensure that we provide a note in the release notes/upgrade guide about why this is happening. We will also need to let support know.

I am unable to reproduce (on Mac OS X) the behaviour described in that Stack Overflow Unicode normalization post. I'm not sure what else is required to trigger the behaviour is mentioned there.

See http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4866151 for a similar explanation of this same problem.