Issue Details (XML | Word | Printable)

Key: CONF-5418
Type: Bug Bug
Status: Resolved Resolved
Resolution: Fixed
Priority: Major Major
Assignee: Unassigned
Reporter: Matt Ryall [Atlassian]
Votes: 28
Watchers: 8
Operations

Add/Edit UI Mockup to this issue
If you were logged in you would be able to see more operations.
Confluence

Plugin resource servlet does not provide headers necessary for caching

Created: 06/Feb/06 09:51 PM   Updated: 11/Dec/06 01:58 PM
Component/s: Plugins, Web Interface
Affects Version/s: 2.1.3
Fix Version/s: 2.2

Time Tracking:
Not Specified

Issue Links:
Blocker
 

Participants: Charles Miller [old account, do not assign issues], Dave Loeng [Atlassian], David Peterson, Guy Fraser [Adaptavist.com], Jens Schumacher [Atlassian], Matt Ryall [Atlassian] and Stephan Janssen
Since last comment: 2 years, 36 weeks, 5 days ago
Resolution Date: 19/Mar/06 09:32 PM
Labels:


 Description  « Hide
It appears that files pulled from a resource don't cache properly in the browser so they get loaded over and over and over...

Here's an example:

Response Headers - http://www.randombits.org/download/resources/com.adaptavist.confluence.themes.sitebuilder:sitebuilder/icons/document_plain.png

Date: Wed, 01 Feb 2006 05:52:20 GMT
Server: Resin/3.0.14
Content-Type: image/png
Transfer-Encoding: chunked

200 OK

As you can see, the HTTP header is sparse to say the least.

Here's an example of an image pulled from a normal web server:

Response Headers - http://adaptavist.com/glyphs/about.png

Date: Wed, 01 Feb 2006 06:09:46 GMT
Server: Apache/1.3.29 (Unix) mod_become/1.3 DAV/1.0.3 mod_perl/1.29 PHP/4.3.9 mod_fastcgi/2.4.2
Last-Modified: Thu, 11 Aug 2005 16:28:48 GMT
Etag: "50b29b-30e-42fb7cc0"
Accept-Ranges: bytes
Content-Length: 782
Content-Type: image/png

200 OK

As you can see, there are certain things in the header that would certainly help with caching:

  • Last-Modified - allow browser to know if it has the correct version of the file
  • Content-Length - allow browser to know if the file is the same size
  • Etag - allows the browser to check it's got the same entity reference

Then there is the chunked transfer encoding output from the resource URL - I'm just guessing here, but that's going to make it very difficult indeed for the browser to work out if it has the latest file or not? Pretend to be a web browser for a minute, and decide if you'd want to work out if you already had a chunked file cached or not:

http://www.w3.org/Protocols/rfc2616/rfc2616-sec3.html#sec3.6.1

I don't know which of these things, if any, would actually alter caching, but it seems that if the HTTP headers from resources could be "fleshed out" a little to give some extra information, there would be a higher chance of the browser caching resources.

So, my question is: Can we output additional HTTP headers AND can we have them sent non-chunked?

Without these two vital things, pulling files from resources is going to be slooooowwww. As a growing number of plugins are using resources in this way (images, css, javascript, etc), we're going to see Confluence sites using lots of plugins getting slower and slower.

If the servlet that sends out the files uses the jar's date as the Last-Modified and also sends the file normally and puts it's Content-Length on, things should run a lot quicker for all resources pulled from plugins, ie. a Confluence-level optimisation. Of course, there may be times when a resource needs a different date (eg. dynamically created resources) so there would have to be a way to define this. From my understanding, however, resources that are simply pulled form the jar (ie. a file in the jar) will never be dynamic?

David Peterson will no doubt be along shortly to add to this (eg. the idea about dumping the files on to disk so they can be pulled more easily without needing to go in to the jar each time)...



 All   Comments   Work Log   Change History      Sort Order: Ascending order - Click to sort in descending order
Matt Ryall [Atlassian] added a comment - 06/Feb/06 09:52 PM
Addendum:

It would also be useful to get the content-disposition added (not just for files loaded from resources, but also for word and PDF exports if possible: when I click on the export links, by default it tries to save a file with the same name as the server side action name, but I digress).

Response Headers - http://www.adaptavist.com/download/attachments/918/cubes.png

Date: Wed, 01 Feb 2006 06:31:17 GMT
Server: Resin/3.0.14
Last-Modified: Thu, 12 Jan 2006 19:33:10 GMT
Etag: "1137094390000"
content-disposition: inline; filename="cubes.png"
Content-Type: image/png
Content-Length: 2146

200 OK


Matt Ryall [Atlassian] added a comment - 06/Feb/06 09:52 PM
It seems that the servlet that delivers style sheets produces a more useful header so it should therefore be relatively straightforward to get the resources servlet doing the same?

Response Headers - http://www.statprowiki.com/styles/main-action.css?pluginCompleteKey=org.randombits.confluence.composition:composition-setup&stylesheetName=deck&spaceKey=SER

Date: Wed, 01 Feb 2006 20:37:57 GMT
Server: Resin/3.0.14
Expires: Wed, 01 Feb 2006 21:30:39 GMT
Content-Type: text/css;charset=UTF-8
Content-Length: 1542

200 OK


Guy Fraser [Adaptavist.com] added a comment - 16/Feb/06 07:06 PM
Hi Matt,

Has there been any updates on this issue - it's really causing us a great deal of trouble. As we are using Confluence primarily as a CMS with lots of visual macros pulling resources from the servlet, it's causing some of our sites to grind to an absolute halt.

Guy


Dave Loeng [Atlassian] added a comment - 23/Feb/06 12:44 AM
Guy,

We acknowledge that is an issue, but have yet to schedule time to implement the fix.

Cheers,
Dave


David Peterson added a comment - 23/Feb/06 01:18 AM
I would consider this to be a higher priority issue than 'Minor' - it will have a big effect on many plugins, particularly as they become more sophisticated.

Guy Fraser [Adaptavist.com] added a comment - 23/Feb/06 09:47 PM
We've got some clients who are having problems due to this issue as their sites are viewed via modem - in such cases reloading even the smallest file repeatedly causes massive delays.

So yes, if the priority could be icreased a little it would be most appreciated.


Stephan Janssen added a comment - 24/Feb/06 12:06 PM
The JavaPolis 2006 confluence site would like to use the Adaptavist builder theme (like we're doing for SpringOne.com site)
but if the plugins are not cached we might be forced not to use it because of the response delay of the confluence plugins
So please, please make sure you can fix this in the next release of Confluence !!

Thanks in advance,
Stephan


Charles Miller [old account, do not assign issues] added a comment - 09/Mar/06 12:37 AM
To implement this correctly, the plugin subsystem would need to know when each plugin was last modified. We could then use that for the modification date header.

To implement this hackily, we could just keep the same last-modified time for all plugin resources, and reset it whenever any plugin is enabled/disabled/installed/uninstalled.


Guy Fraser [Adaptavist.com] added a comment - 09/Mar/06 08:56 AM
The hacky way would be fine for now (and probably long term IMHO) - just reset the date each time a plugin is installed, etc., as you say

Guy Fraser [Adaptavist.com] added a comment - 09/Mar/06 11:21 AM
Hi Charles,

Would it be possible for us to get hold of a Confluence 2.2 before the general release - we've got lots of clients who are "chomping at the bit" for the improved performance this will offer so we'd like to test Builder and our other visual macros prior to release rather than have a flood of bugs reported when people suddenly start upgrading ahead of us. We are also aware that it won't be a magic bullet for all performance issues, so we're keen to set client expectations about the performance boost and provide them with tips for making their home page faster (eg. don't flood it with a gazillion graphical plugins like we do hehe).

FYI: We're drooling over some of the additional features and bug fixes in 2.2, especially from a plugin development point of view

Guy


Jens Schumacher [Atlassian] added a comment - 09/Mar/06 10:38 PM
Hi Guy,

We will consider to release a DR [1] next week, based on our progress. Please note that this will be a DR and might not be as stable as the final release. Since 2.0 we haven't had any Developer Releases so it is actually a good idea to provide one if it proves useful for you.

Cheers,
Jens

[1] http://confluence.atlassian.com/display/DOC/Development+Releases


Guy Fraser [Adaptavist.com] added a comment - 15/Mar/06 04:28 PM
Dan gave me a preview of two Confluence installs earlier this evening - one with his patch for the resource servlet, and one without. With "Fasterfox" set to default speed settings, the patched version was twice as fast! Similar performance increases were seen on Internet Exploder and Opera also

Hopefully we'll be able to patch our hosted clients within the next 48 hours or so and we'll certainly be contributing the code back to the community as it really does speed up the performance of plugins that pull assets from the resources servlet (much more than I had expected or hoped for).

Bear in mind that this applies not just to graphics, but javascript files, css files, etc. Cached resources here we come! Yay!


Jens Schumacher [Atlassian] added a comment - 15/Mar/06 09:33 PM
Thanks for the info Guy.

This will be awesome for macros requiring lots of JavaScript like the dynamic task-list.



Charles Miller [old account, do not assign issues] added a comment - 19/Mar/06 09:32 PM
I've rolled this fix into Confluence 2.2 (atlassian-plugins-0.3.22)

Thanks, guys!