fmotglongorder We added a new subdirectory to the attachments directory that allows more subdirectories to be created in the file system. In 7.x, attachments are stored in $JIRA_Home/data/jira/data/attachments/PROJECT_KEY/x0000/ISSUE_KEY/ID, where x0000 is a subdirectory that is created and named for each 10,000 issues. For example, attachments for issues 1-10,000 are stored in /PROJECT_KEY/10000/ISSUE_KEY/ID; issues 10001-20,000 are stored in 20000, etc.

• If you migrate to 7.x, your existing attachments stay in the original location of $JIRA_Home/data/jira/data/attachments/PROJECT_KEY/ISSUE_KEY/ID unless:
  someone views a migrated issue and clicks an attachment thumbnail. JIRA then copies the attachment to the 7.x directory structure.
• someone adds a new attachment to a migrated issue. The new attachment is stored in the 7.x directory structure.

This blog explains in more detail.

Thanks all. This is on our backlog, but we're not committing to have this in 6.3 at this stage.

We will let you know of any progress via this issue.

In the 6.1.x version, the attachment handling is pretty unified (in 5.x, you had to workaround in 4 files), so the change itself would be relatively easy:

/jira-project/jira-components/jira-core/src/main/java/com/atlassian/jira/issue/attachment/DefaultAttachmentStore.java:

public static String getIssuePathPart(String issueKey) {
	String number = String.format("%07d", IssueKey.from(issueKey).getIssueNumber());
	int len = number.length();
	String MMM = number.substring(len - 6, len - 3);
	String UUUU = number.substring(0, len - 6);

	return UUUU + File.separator + MMM;
}

	// Replace the "return" in the else block.
	return new File(projectDirectory + File.separator + getIssuePathPart(issueKey), computeIssueKeyForOriginalProjectKey( originalProjectKey, issueKey));

Build, deploy.

This will split up the directory tree to attachment_root/U/MMM/<PKEY>-<NUMBER>/<attachment_id>_<filename>, this way we will have 2 additional directory level, and at the lowest level we store maximum 1000 issue per directory.
If the NUMBER is smaller than 1000 the issuePathPart will be 0/000.

Warning: I haven't checked, that this rearranges or not the existing attachments!

How does it scale? We have ~1.5M attachments in ~1.3M issues.

I hope that Atlassian will realize the importance of problem and solve this on the fast-track, clearly this must be implemented in 6.2 in order to support clustering. It would not make any sense to have a different file-structure for clustered and non-clustered versions.

In fact Atlassian already have experience doing this on Confluence 3.x when they did the same. I just hope they will perform OS moves instead of trying to copy files as we have terabytes of attachments.

@matt.doar - I just changed the URL to access the page ID and was directed to JIRA Blue Label Pioneer Program.

Sorin, that link failed for me, perhaps because there are two comments with the same content? Anyway, I can imagine other schemes being used for clustered JIRA but you're right, not the current one unchanged

This bug priority should be reconsidered as it will be blocker for implementing a clustered JIRA - https://confluence.atlassian.com/pages/doaddcomment.action?pageId=441745954

Look, what I just found: http://pastebin.com/YKaQBdnJ
It is for 6.1.6, seems they(Atlassian) unified the attachment storage handling, so there is only one place to change things.
If you backport it to 6.1, be sure to check that BulkMoveOperationImpl.java and MoveIssueConfirm.java are using attachmentStorage() or not.
In 5.*, there is no AttachmentManager and AttachmentStorage, you should fix things in AttachmentUtils.

Happy hacking.

I'll do what I can do

I've seen diffs attached before in other issues and no-one at Atlassian had a problem with it. If there is a problem then someone will just remove it and tell us all not to do that. Go for it, it sounds useful!

I've also emailed developer-relations at atlassian to check on this

Does the license allow that?

Can you attach a diff for others to see?

This bug is still in place in 6.1, and we have 1.1M attachments (99% of it belongs to one single project).

Our workaround was to change relevant parts of the jira source (3 places actually) and implemented a directory hierarchy.

I join the others, as changing file system is not a solution nor even a workaround. The solution would be about 17 extra lines of code.

I'd guess it doesn't get fix because the current approach works for most people. The approach of changing the file system to one that supports more than 32K is a good one. My concern about attachments on disk is more about how to distribute JIRA in that case

-∞ for database storage: i have several jira instances with almost 1 TB of attachments, clearly a database would not be able to cope with this and this will make database backup / restore, and export and import almost impossible to run.

Clearly the attachments do have less value than the rest of the database, and keeping them on filesystem is a good approach. The problem is the way of storing them on disk is not scalable due to most filesystems limitations, including the distributed ones. We managed to solve our problem by switching to an XFS which does not have a documented limit (confirmed to work with >2 milion directories).

Still, I hope that Atlassian will not take my comment as a valid workaround for not fixing this bug. Clearly Jira must be fixed to store attachments by splitting them like Confluence or most other software products that have to deal with many files (ex. source control).

+1 for storing attachments in the database, Confluence does it and this would make it so much easier for upgrades...

This could be resolved by storing attachments in the database rather than the file system - refer JRA-806

Wow, we just reached the same problem, I am surprised that nobody from Atlassian dealt with this yet.

Confluence solved the problem few years ago when they changes the storage path in order to make it scale well.

This only occurs if a single JIRA project has more than 32000 issues with attachments, not the total number of attachments in the whole JIRA instance.

http://en.wikipedia.org/wiki/Ext4

"In ext3 a directory can have at most 32,000 subdirectories. In ext4 this limit increased to 64,000. This limit can be increased by using the "dir_nlink" feature (but the parent's link count will not increase). To allow for larger directories and continued performance ext4 turns on Htree indexes (a specialized version of a B-tree) by default. This feature is implemented in Linux 2.6.23. In ext3 Htrees can be used by enabling the dir_index feature."

We hit the same limitation at 99999 files on a attached storage from a NetApp

Confluence did this in 3.0 - http://jira.atlassian.com/browse/CONF-8298

To work around the problem you can move some of the subdirs somewhere else and then create a symlink to them. (i.e. replace a subdirectory with a file) Another workaround is to switch to a filesystem that has higher limits or no limit at all. This is only a problem on ext2/ext3 (and maybe JFS, I'm not positive). ext4 would be the easiest upgrade path since you can convert ext3 in-place.

Jeff,

Thanks for the info., I will check it out.

You're right, Royce, I did also have to patch the move operations. I totally forgot about that.
I attached the source of what I modified and compiled versions.
You'll also need to write a script to move or manually move the old files into the new structure.
The new structure will have bins of 30,000 issues, like this
jira/attachments/XYZ/30000/XYZ-1244
jira/attachments/XYZ/60000/XYZ-40244
Because of the small number of the bins, manually moving them might be fastest. That's what we did...even with 200,000 issues over 10 projects it only took a minute or two to move the old files into the new structure.
Hope that helps.

Sorry to hear that this has nabbed you, too, Royce. We ended up modifying the AttachmentUtils.java class to add a middle directory in the attachment structure that groups the issues in sub-directories of 10,000 issues.

Thanks Jeff. How would your solution affect attachment linking, individual move and bulk move?

There is a patch http://jira.atlassian.com/browse/JRA-16223 but does not work completely, it moved the image attachments but still left behind the thumbnail files jira generated.

Sorry to hear that this has nabbed you, too, Royce. We ended up modifying the AttachmentUtils.java class to add a middle directory in the attachment structure that groups the issues in sub-directories of 10,000 issues.

Same here, ran out of project attachment directory.

/jira_data/attachment/PROJECTA
bash-3.00# ls -l | wc -l
   32766

Our immediately thought is to move some old issues to a separate project (with the help of copy project plug-in https://plugins.atlassian.com/plugin/details/6829, not supported by Atlassian), and then clean up the empty directories under /jira_data/attachment/PROJECTA.

But it seems the bulk move feature won't retain components and version (http://jira.atlassian.com/browse/JRA-8248) and we will have to move issues one by one which is crazy considering the large number of tickets we have. But this can be solve by Bulk Move Restore Lost Fields plugin https://studio.plugins.atlassian.com/wiki/display/JBLKR/Bulk+Move+Restore+Lost+Fields as suggested in http://jira.atlassian.com/browse/JRA-8248 (again, not supported by Atlassian).

But still I encountered another bug with 3.13 Bulk move image attachments (file attachments are fine) remain in previous project folder http://jira.atlassian.com/browse/JRA-16223. Now I have to upgrade to 3.14 and try.

Yikes.