Hi everyone,
We are proud to announce that with Jira 7.10 we have shipped Archiving Projects for Jira Data Center. Now Data Center admins can easily archive and restore projects together with their issues.
Here is a dedicated issue for project archiving: JRASERVER-1450

We are aware that the new capability does not address all the use cases described in this suggestion but we believe some of you may find it very helpful.
Meanwhile, per-issue archiving is considered an addition to our longer term roadmap. We will typically review and update this request in about 6 months from now, at which point we’ll consider whether we need to alter its status.

Kind regards,

Ilya Zinoviev
Jira Developer

For those who needs a very simple archival solution, you can check out the
Issue Archiver for Jira which we have developed.

Do note that the archival is 1-way. It is not possible to restore the archives back into a Jira server again.
However, the archival is human read-able and can fulfil audit requirements.

Hi Watchers and Voters, 

The Archiving in Jira survey is now closed. Thankyou to everyone who took the time to provide feedback, we really appreciate it!

Thanks again, 
The JIRA Software Server team

Hello Watchers and Voters,

Thanks for all your feedback and votes so far, we truly appreciate them. We're currently researching customer's expectations of archiving in JIRA, and we'd really appreciate it if you could take the time to fill out our 15 minute survey, and help us work on something great for you, your team and organization in the near future.

http://www.surveygizmo.com/s3/3789740/Archiving-in-JIRA 

Thanks in advance,

The JIRA Software Server team

An alternate workaround we have found as TAMs is to use an issue sync plugin called Exalate. You can "exalate" selected issues from the production instance to an archive instance which will sync/create copies of the issues to the archive instance. You can then "Unexalate" it to remove the one-one link between synced issues and then delete the original issues from the production instance.

Essentially you can sync the issues to an archiving issues and then delete the issues from the instance in use. The advantage of Exalate is that you can sync the issues moved to the archived instance back to the production instance if necessary.

 This by no means replaces the need for archiving of issues but alleviates the pain using a plugin and a few steps. 

Hi Girish, yes, the issues continue to be indexed.

One way to clean up a bit:

1. Make a new project, and move the issues you actually want to it.
2. Hide the old project with a permission scheme
3. After a suitable retention period, delete the old project

This method is a form of controlled data loss, which is the only way I know to actually "clean" anything, but may not be appropriate for you. Good luck!

Hi,

Does hiding a project using a empty permission scheme still continue to index these issues in the search index?

Is there any progress ?we really need this functionality

Almost exactly 2 years ago I suggested that converting your issues to indexable PDF files, inserting those PDF's into some file search facility, then removing the issues from JIRA, could be an option to implement issue archives.

Looks like implementing this feature in JIRA core is still not happening, and you still have to decide which is more important:

1. to reduce the load on your JIRA server (then you have to move the issues "out" from the server),
2. or the ability to restore the issues (then you play with "visibility" of the issues using JIRA permissions).

If the former is the key, then the PDF method is still a viable option. (For instance, archiving old tickets from Service Desk to a search engine is a great example where restorability is unimportant.)

Let me note that in the last 2 years we have been working on the PDF View Plugin to try to cover this problem more efficiently, and to make the plugin less of a workaround and more of a viable solution. Since 5.1.0 the add-on supports exporting up to 50K issues to a single PDF file (see the shot about a 25K issues test doc) and as it preserves all details of a JIRA Issue in a searchable way, this can be a close-to-comprehensive solution for archiving and storing.

You can use my groovy service like example to archive big tasks:
https://jirasupport.wordpress.com/2016/02/01/new-example-of-groovy-service-for-jira-to-archive-big-tasks/

Will the archiving of JIRA Issues be possible? Attachments images and text to another format?

Don't worry. They can't "archive" it because they don't yet have this functionality.

Good point Jonathan, I retract my comment.
I need to find a way to organize our backlog better though.

Ron.Suhodrev,

Perhaps, according to your last reason, Atlassian should "archive" this and other highly voted issues (since "there are not currently any plans to implement this suggestion").

I have another reason for this I didn't see mentioned. (Sorry if I missed it)
I would like to archive issues because these are features we don't want to currently include (changed our mind). I don't want to just mark them as "Done - Won't Do" because I might change my mind again in a year or two. and meanwhile I don't want them cluttering my backlog

Hi Otto,
It is a minor difference to state that the motivation for the product team is "activating more users" but not "selling more licences". @Greg is on point and his closely resembles our situation.

As a long-term supporter and user of Atlassian products (since Jira 3), I also declined to renew our company's support/maintenance for Jira 6.3 last year due to Atlassian's neglect of core expected features such as this one open for 10 years (talk to @Kayleen Ignacio and @David Mayer if you're interested). The Jira "Cloud" releases and new release of "Service Desk", and the new (more expensive) licensing terms - which I'm sure are successful for Atlassian - are not of any benefit to my company.

Atlassian needs a much stronger focus on their existing core systems, existing customers and their highly-voted tickets; not just large enterprises who could afford an Expert consultation, premium Add-ons, or can attend your Summits. And frankly you probably need to get more developers if a request like this can't get the attention it deserves.

Thanks for the feedback greglions. I'm sorry to hear you moved away from JIRA.

I don't quite follow how it's actually about selling more licenses. If that's an inference that we are not providing archiving to sell more product - that assertion would be entirely wrong. I can't think of a single Atlassian product team that is motivated by the potential of more license sales. We care about activating more users and use that as a metric for many product decisions. This ethos has been a large reason for Atlassian's success.

5-6 years is a long time ago, but it if's who I think it is, Edwin is still with us and runs our JIRA Service Desk Product. Since then we've seen the limits for number of issues pushed back and evaporate with some customers having instances in excess of 2M issues.

Regarding the approach you describe, in more recent times, (since I joined) we've investigated an approach similar to what you mentioned. Essentially we spiked sharding of the lucene index. We found that this approach would keep customers audit compliant and reduce clutter, however, sharding did next to nothing for the objective of increasing performance. Given that performance was the number one reason why customers wanted to archive we decided not to proceed with this approach. Instead we invested directly in improving performance and made significant gains with JIRA 6.4 and beyond.

Its actually about selling more licences….

I spoke to Edwin? ( a previous Jira Product Owner from Atlassian ) and one of the senior Jira engineer (don’t recall his name) about scalability issues with Jira some 5-6 years ago. They came to my office and I even gave them a full dump of oneof our jira to analyse why the instance would crash ( corrupt indexes etc ) when over about 250k tickets.

Anyway, I suggested a simple rule engine that moved all closed tickets of x months to a read-only database/and index so they would not be re-indexed continuously. Should a user wish to find an old ticket, they could use a checkbox “Search Archived Tickets” which would look in the archive.  The engineer thought that would work and very easy to implement and solve several other issues they had…. Never happened. But Atlassian did release “Jira Enterprise” soon after… the highlight of which was a t-shirt from memory.

We now use a different tool for enterprise ticketing.

Thanks for the comments everyone. I really do appreciate the feedback and it helps. I'll continue to share our position openly and without bullsh1t.

Re david.turner@mmal.com.au comments, I'm sorry, I think I didn't explain our position clearly enough. I am not suggesting that audit compliance is a low priority. The point I was trying to make was that when we dig into why archiving is a need for customers, there were essentially three needs (in order of priority);

1. make it faster
2. keep me audit compliant
3. and reduce the clutter.
   The highest priority for virtually all customers was performance. Moreover, when asked "If you could have 2 & 3 only, would you be satisfied?" The answer was "No". i.e. performance was the key driver for archiving.

The additional challenge was that when we look at all the things in JIRA that contribute to it's performance, we believe that the provision of an archiving solution would address 2 & 3 but that we could (and did) make a far more significant impact on performance with direct investment in this area. This investment is ongoing.

I should also clarify that when I talk about cost - I'm only talking about it in relative terms. i.e. when we prioritise work we essntially stack rank initiatives based on the ratio of return vs. cost; Return essentially being how many customers' experiences can we improve how often and cost being how many dev days it will take. Initatives that are high return and low cost rise to the top. Like any organisation, we are limited by the number of people we can throw at a problem. Even if you were to live in a hypothetically unconstrained world where you had unconstrained funds you would still need to prioritise to make relative investment decisions - i.e. how big a team do you put on initative A vs. initative B.

michael.parks comment is really interesting in that it appears to contradict feedback I've heard from customers. I'm glad to hear that performance is no longer a highly ranked concern for Michael. Let me add a little more colour that might provide further context... Every year at Atlassian Summit, all PMs try to get a sense for the highest priorities of our customers. It's a great opportunity for us to have in depth conversations with so many of you. The audience at summit tends to over-index to our enterprise customers and it is also those customers that have clearly told us to improve performance (and are usually the ones asking for archiving to help with performance). Among my biggest take aways from this year's summit was that performance was no longer the top of everyone's list. As you might have seen in the keynote, we've made significant strides in performance and it's heartening to get a signal that says it's being felt by customers. I'm not convinced we've satisfied everyone's performance requirements and we have a lot of NPS data that suggests we need to do more here. However, I'm left wondering if there is a significant change in the needs of our customers that might mean when it comes to archiving de-clutter and audit are more important than performance to the majority of customers. I'd be interested to hear more comments from others as to what's important to them regarding archiving.

Both kurtn and michael.parks mentioned a add-ons that provide archiving. Could you please provide links to these, as I am unaware of any for JIRA and it might be helpful to highlight these as work arounds in the description.

Hopefully you can tell from my reply that Atlassian take all input seriously and that I mean it when I say thank you for sharing it with us. We are lucky to have so many customers that are so engaged and passionate.

Thank you,
Otto

Guys, it's not about performance. JIRA performance, as @Otto mentioned, is fine even with truly absurd numbers of tickets.

@David gave a pretty good laundry list of reasons why this is necessary, but I can add one more: User experience. As you add more and more and more tickets, narrowing down exactly what you want in a filter search becomes more and more difficult. The answer to this is not better filters (since JQL is already about as expressive as can be), but to remove some of those tickets from search.

The fact that it requires a third party addon to provide basic functionality that's been wanted for well over a decade, and then being handed the "it costs too much" argument (especially in light of a recent IPO - do addon devs have more resources than the parent company?!) is... disheartening, and slightly insulting to one's intelligence, to say the least.

@Otto I am confused, there are a number of add-ons that offer this functionality, so its easily possible to add this functionality by non Atlassian folks, it should be simple for you at Atlassian. An open ticket with active comments from 2005 should not have to wait 10 years for a "Sorry were not adding this" reply....

@scott @mike

Hi Otto,

I would disagree that Audit compliance can ever be portrayed as a "lower priority" matter.
And how about large numbers of tickets impacting:

• Long running searches.
• Lucene indexing load/duration.
• Database limits
• Filesystem limits (e.g. GBs of attachments).
• Backups.

We could delete tickets and attachments through scripts, but as many have outlined, it is expected that a leading ticketing application should have a supported archiving method which is non-destructive. Is there any supported work-around solution which does not reduce the richness of the application?

It is simply amazing that highly-requested features should still remain idle on the basis of "cost" when Atlassian just raised billions through their share float last week.

I've got to +1 on this.. archiving old issues is a pretty standard feature on just about every other tracker out there, and as Kunal says, this has been wanted for well over ten years.

Please folks?

Any news on this yet? This request has been open for a decade now!!

Signed up just to comment on this issue, as others have mentioned this feature would be very handy for our project and I'm surprised it isn't standard.

Standard feature of other issue trackers, please add this feature. An open issue since 2005, how is this acceptable for a paid-for product?

dear Atlassian, any news on this request? Thank you

Is this feature available yet? This seems pretty standard in Asana, Lighthouse, other tools. It drives me CRAZY to look at 4 billion tickets that have been resolved but still look like they need attention.

Thanks,

Heather~

Maybe this could help in some way: https://marketplace.atlassian.com/plugins/net.impera.issue-trash

What Nicole said.

We are getting to a point where I want to get things archived off but the difficulty of retain issue data like sprints, issue links, attachments, makes it difficult to attempt an export type of solution. I don't know how successful others have been in their archiving efforts.

Can you please provide an update on the archive functionality? We have over 350K issues (total) in our production system along with a large number of attachments. Offline archiving as it stands now will not work for us, so we are stuck with old projects taking up valuable space that are a drag on performance.

Hi,

Have you got any news about this issue? I would be extremely interested by this feature, so if you need any help to speed up the development, do not hesisate to contact me.

Regards

Hi!

In the recent months we added several new features (API, automation, etc.) to the aforementioned PDF View Plugin, that ease implementing some automatic archiving mechanism in JIRA.
Please note that what I describe here will primarily archive to PDF files in the filesystem (although it can be changed), not to a secondary JIRA instance.

Archiving idea

1. Create a saved filter that returns the list of "archivable issues".
   (You can use the full power of JQL here, and identity those issues by project, by some label, by some status - be creative.)
2. Set up a periodic job (a Groovy script that connects to the JIRA's and the PDF plugin's API), which executes this filter say every night (02:00AM seems a good time).
   If there result set is not empty, render a separate PDF from each issue using the PDF View Plugin's API. The PDF could be titled by the key of the issue like "MYPROJ-123.pdf", but you could also include the (escaped) summary in the filename. Then copy these to a folder in the filesystem.
3. When the PDF was constructed, make some simple programmatic checks on it (filesize not zero or larger than 100 bytes, e.g.) or even some smarter checks (like programatically opening them with an external tool and assert if the first word of the description text, the name of assignee, the name of the project) is contained by the PDF file.
   All this for one purpose, to be sure that the exported PDF is correct.
   (For example, if your plugin license expires or there is a syntax error in your PDF template, the plugin will export a valid PDF with the warning message, but it does not mean that the export was successful.)
4. If everything is looking good. then delete the issue.
5. Then take the next issue (or next batch of issue) and go to step 3.
6. Finally, you could even notify some email address that "this list of issues just got archived", etc.

Advantages

1. It is 100% automatic. You put it in place, and you can pretty much forget about it.
2. Every parts are configurable (frequency of execution, the filter, the logic in the script) and are under your control. The script can be quickly prototyped in Groovy.
3. If you let some internal search engine index the filesystem directory that contains the PDF documents, then you get search for free.

Resources

Tutorial about the PDF view plugin's API:
http://www.midori-global.com/products/jira-pdf-view-plugin/documentation/api

Automation tutorial:
http://www.midori-global.com/products/jira-pdf-view-plugin/documentation/automation
(the idea here is to utilize the super duper JIRA Automation Plugin. it may save you some (all) programming in simpler situations.)

Can we create a secondary schema on the database server - or an archive DB cluster, and then move the projects to the archive datastore. However this requires a functionality in JIRA that will let users select which schema a project can be "Created" or "Archived" to. This way the mission critical production projects can use the primary schema running on primary data cluster. And the archive can be running on a secondary data cluster. KInd of primary vs secondary storage used in the data backup realm. This will ensure that the links still work on the same JIRA instance except the data is moved to another instance/schema.

Roy Krishna,

What do you mean by Seperate Server, can these closed & archived issues be accessed ? If so how would do that....
Can you please throw some information on JIRA Archiving solution .

Is this the best issue for tracking the evolution of a mature archival capability in JIRA? The others that I have seen that are closed for project archival mention work-arounds, but they just hide the data, reducing the noise in the interface, but do not actually archive the data, removing it from the DB/ indexes. I understand that Atlassian has an interest in progressing this capability, but I would like direction on the best place to "watch" for that to happen.

Is there any progress on this issue?

hello everyone,

exporting sets of outdated issues into customizable, searchable PDF documents can be easily implemented with the JIRA PDF View Plugin (disclaimer: i'm a developer in the project)

we have a customer who archives their old issues into these documents, index them in a separate application and delete the original issues in JIRA. (they don't need the "restore" functionality).

it's not the most powerful approach, but it just works.

• feri

Hey guys,

Just thought some of you guys might be interested in the following article/blog:

http://blog.ambientia.fi/?p=700

Cheers,
Eric Salonen

Hi,
I'd like to share some experiences on JIRA scalability.

As an Atlassian Partner I am connected with a company that is planning to create 300K issues per year using the email handler components.
They have set up a powerful test environment and imported into it 1 million issues from emails they generated.
I have seen the system running and can express how surprised as I was to see that JIRA was pretty fast. Response times were excellent, full reindex time was about 1.5 hours.

Some characteristics of the test setup:

• Single project
• 1 million issues
• Few users, default groups
• No custom fields
• Basic workflow
• Many attachments

Index size was about 1.5GB.
This all looked promising although tests are not yet complete.

Do you have any idea what aspects may risk performance? E.g. increasing number of projects, or introducing custom fields, plugins, more users/groups? Any idea is highly appreciated.

I hope to get a case study out of this story towards end of this summer.

Cheers,
Tibor

We currently are using jira 3.13. there are more than 250000 issues. JIRA is very important for our business, so this problem is critical for us.
It should be useful to be able to archive very old closed issues automatically, but they would still be useful for a Knowledge Base.
The large number of issues affects the system performance.
When planned to implement archiving?

PS:archiving project does not solve the problem. This is a major reason why the idea will be considered, choose another product.

Hi Atlassian,

Is ANYTHING being done to make Jira more scalable without throwing more hardware at it? I can tell you DB are looking at other products which dont have this "200k" performance fence and id like Jira to remain a player as it has done very well so far.

Please advise.

Greg

I am not sure if this will help many of you.
But if you just have too many issues in one project, then you can create another project in the same instance as say archive project, and move your issues to archive project. Your links will still work.
This will at least give you less issues to work with in one project. Reporting and searching would be better.
Of course this will not improve the performance as a system, because the JIRA system will still have those issues

Our instance is being used as a help desk (among other things) and we are growing issue count in one project at 25K per year. We now have about 75k issues and our memory consumption is causing garbage collection much too frequently thus slowing down the system. Is there anything we can do to archive issues greater than 1 year old to some other readable text/pdf format and then delete them from the production system? We are thinking that this would free up memory, reduce the frequency of garbage collection, and improve system responsiveness.

Java VM Memory Statistics
Total Memory 1498 MB
Free Memory 110 MB
Used Memory 1388 MB
Total PermGen Memory 304 MB
Free PermGen Memory 169 MB
Used PermGen Memory 134 MB
Memory Graph 7 % Free (Total: 1498 MB) (Force garbage collection)
PermGen Memory Graph 56 % Free (Total: 304 MB)

Database Statistics
Issues 86971
Projects 24
Custom Fields 167
Workflows 21
Users 3783
Groups 35

Hi Team,

We have about 500,000 Issues and the volume is very likely to increase in near future at a much faster pace. This problem is extremely critical for us and we would like your support in getting this resolved quickly.

Also, we have about 29 votes in favor of this issue, and i am sure you will realize that this has become quite important for a lot of clients and is likely to become to more and more important for other clients as well.

Kindly provide your thoughts on this.

Thanks & Regards,
Vishal

Hi.

We have the same need of an "incremental project import" feature, because we can't archive entire projects or split the projects in two instances of JIRA.
For the moment we have more than 280,000 issues for 280 projects.
Each year we've got almost 100,000 more issues. And we've got more and more slowness with JIRA.
So this problem is critical for us.

I understand that less than 1% of customers have more than 200,000 issues; but precisely, these customers are big customers because of their number of issues. If we've got lots of issues, that's because JIRA is very important for our business.

I am very disappointed to see that this request is unassigned and that Atlassian is not working urgently on it.

Cheers.

Hi Greg,

At Atlassian we do not have projects with that many issues. Our largest project is approaching 30,000 issues. Therefore, when required, we archive projects.

We do have customers with more than 200,000 issues. I believe the largest is around 800,000. Usually when going over 200,000 performance starts to degrade, i.e. the system becomes slower, which can be mitigated with "bigger" hardware. However, the more issues get created, the more hardware is required. I believe the instance with 800,000 uses about 8GB of RAM, but the performance is a problem and they are looking at splitting their instance.

We are always paying attention to JIRA's performance and scalability. At the moment we have less than 1% of customers who are over 200,000 issues, and therefore we do not have immediate plans to increase this limit. However, we are planning to readdress this in the future.

Cheers,
Anton

Thanks Anton for the reply.
Project archiving is totally different.

How does atlassian handle archiving - id imagine you have a few projects with a large number of issues?

Can you advise if Atlassian still has the view of 200K issues is the upper "limit" before splitting instances? What happens if 200K is exceeded? Has this been tested recently?

At my employer there is a push to use another product, which is being resisted, but a vendor guidline maximum issue count of 200k does not say that Jira is scalable for what they want to do, and thus an arguement for using the other product. So if this can be given more priority then im sure it would help!

Perhaps its worth Atlassion doing a review of the issue counts from all the "system info" screens attached to Support tickets and see where its users are at?

Cheers,
Greg

Hi Greg,

There is currently no "direct" way to archive issues, but it is possible to archive projects:
http://www.atlassian.com/software/jira/docs/latest/archive_project.html

It is possible to backup the system and then delete old issues in a project. The problem with this is that restoring these issues would not be possible into the same JIRA instance. One would need to setup a new JIRA instance and restore the project from the backup into the new system.

At the moment, due to the large number of other popular feature requests, there are no concrete plans to implement this functionality.

Cheers,
Anton

For some additional information about requested features/behaviour see JRA-13129.

Is there any progress ?