If JIRA could give you the whole XML without memory problems, would that solve your issue? I'm working on rewriting the way that we do XML views to be much more efficient.

Scott

Thanks - it would be great if JIRA's XML interfaces get development effort. We have currently developed a more-or-less effective way to download issues from JIRA using whatever means available (you can take a look at http://almworks.com/jiraclient ). But we still miss the functionality described in this issue.

Regarding your suggestion about the whole XML. We do care about the efficiency of download (total time spent, memory consumed on server, network traffic consumed). But even more we care about the "interactivity", as observed by the user of our application. That means we need to a) start getting issues fast; b) estimate total issues to be downloaded and measure progress; c) be able to cancel the download without extra cost.

Paging is a nice and somewhat reliable way to achieve these goals. By paging here I mean any way to download large amount of issues with a series of requests, each getting a fixed-size chunk.

Getting the whole XML in one request may also work, but it is less flexible and less reliable. For example, suppose that I'm downloading 10000 issues and while 9999'th issue is being transferred the connection is broken. I will then have to waste time and bandwidth to download 9998 issues before I get those two issues that have been left. Also, receiving a large XML from the network will require the usage of SAX parser without constructing the Document (which is fine, but will require more development effort).

I guess the best thing to do is to make it possible to get the XML with all bugs while preserving and enhancing the ability to do paging.
Please note also that we need to specify query conditions to download a subset of issues, as we do now with IssueNavigator and view=rss.

If I may also add, please please include more information in the XML or make it possible to request a more full XML. We currently miss all attachment data (without content, of course), subtask/supertask relationship, work log, change history, available workflow actions. Maybe installed JIRA plug-ins would like to add their content to the XML.

Sorry for being overly wordy This topic is crucial for us.

Cheers,
Igor

I have added 'parent', attachments and subtasks. It should also be much simpler for you to create your own XML view if you wish.

I haven't added paging, but I have tested with very large datasets, and there is no out of memory. I will also enable gzip compression, so that we minimise the amount of data that travels across the network.

Will that work for you? Paging doesn't really work too well at a code level, so I would be interested to know if this solution will work for you.

You will be able to create plugins which produce 'content' for search requests or single issues. You can then create a 'view' that exactly matches what you need for your client. Here is the API (as it stands - this may change!)

public interface SearchRequestView
{
    public void init(SearchRequestViewModuleDescriptor moduleDescriptor);

    public void writeSearchResults(SearchRequest searchRequest, Writer writer);

    public void writeHeaders(SearchRequest searchRequest, RequestHeaders requestHeaders);
}

Cheers,
Scott

Thanks, it's great that we'll have more information in RSS xml. (Starting from version 3.7, I guess?)

Regarding paging - probably this solution will work for us. I will check how fast do we start to get a reply from server if we ask for tens of thousands of issues. As I noted above, we need to have low latency replies from server to be interactive. For some time, we'll stick with our current implementation of paging via "updated" field to support older versions of JIRA, but in the future we will switch to non-paged requests if server performance allows.

By the way, why paging doesn't work well at a code level? Doesn't Issue Navigator implement paging? It might be something similar in case of rss.

Thanks also for the API - we probably write a plug-in someday, but currently we try to work with JIRA and with Bugzilla without any changes on the server side.

Regards,
Igor

The response starts almost immediately, independently of how many issues you have. From my calculations, the RSS view is almost 2 times as fast (overall), with more content. YMMV.

There is no code-level problem with implementing paging, but it is more code that most of our customers would not be using.

I'll see what I can do in terms of a beta for you to test.

Cheers,
Scott

Thanks.
-Brad