New and Improved 3.13 Beta. Highlights: Shareable filters and dashboards and lots of other goodies. Any feedback can be raised as JIRA issues in the JIRA project.
Issue Details (XML | Word | Printable)

Key: CONF-8623
Type: Improvement Improvement
Status: Resolved Resolved
Resolution: Fixed
Priority: Major Major
Assignee: Unassigned
Reporter: Matthew Jensen [Atlassian]
Votes: 3
Watchers: 5
Operations

If you were logged in you would be able to see more operations.
Confluence

Improve and speed up plugin resource loading

Created: 03/Jun/07 08:20 PM   Updated: 29/Jan/08 09:15 PM
Component/s: None
Affects Version/s: None
Fix Version/s: 2.5.6, 2.6.0

Time Tracking:
Not Specified

File Attachments: 1. File DynamicPluginResourceLoader.class (2 kB)
2. Java Source File DynamicPluginResourceLoader.java (1 kB)
3. Text File stacks.txt (40 kB)

Participants: Matt Ryall [Atlassian] and Matthew Jensen [Atlassian]
Since last comment: 30 weeks, 3 days ago
Resolution Date: 23/Jul/07 08:41 PM
Labels:


 Description  « Hide
A response template goes through several stages of decoration, using the #applyDecorator macro specifying a template name. This goes through the sitemesh decorator logic and eventually ends up in the class loading implementation to find the decoration template. Confluence implements the plugin classloaders to scan through the plugins to find the actual resource that satisfies this name.

This involves scanning each plugin jar for the required template resource. If there are a lot of plugins, this can take quite some time.

The three main problems with this implementation are:

  1. During the scan, the individual plugin class loaders (JarClassLoader) is synchronized, meaning all other plugin resource or class loading is executed sequentially
  2. The result is not cached, or recorded in any way, meaning this will happen multiple times on every request
  3. The surrounding DynamicPluginResourceLoader that velocity uses to bridge to the DefaultPluginManager, is synchronized as well, meaning all plugin resource loading that originates from velocity is executed sequentially

A typical stack trace showing this behavior:

"http-127.0.0.1-48080-Processor3" daemon prio=5 tid=0x011248e8 nid=0x23 runnable [0xb7e7c000..0xb7e7fc28]
  - locked <0xb91ac448> (a java.util.zip.Inflater)
  ...
  at java.util.jar.JarInputStream.getNextEntry(JarInputStream.java:114)
  at com.atlassian.plugin.loaders.classloading.JarClassLoader$InnerJar.scanForPrefixes(JarClassLoader.java:53)
  ...
  at com.atlassian.plugin.loaders.classloading.JarClassLoader.getFile(JarClassLoader.java:190)
  - locked <0xc18644e0> (a com.atlassian.plugin.loaders.classloading.JarClassLoader)
  at com.atlassian.plugin.loaders.classloading.PluginsClassLoader.findResource(PluginsClassLoader.java:79)
  at com.atlassian.plugin.loaders.classloading.PluginsClassLoader.getResource(PluginsClassLoader.java:149)
  at java.lang.ClassLoader.getResourceAsStream(ClassLoader.java:995)
  at com.atlassian.plugin.impl.DynamicPlugin.getResourceAsStream(DynamicPlugin.java:33)
  at com.atlassian.plugin.DefaultPluginManager.getDynamicResourceAsStream(DefaultPluginManager.java:676)
  ...  
  at com.atlassian.confluence.setup.velocity.DynamicPluginResourceLoader.getResourceStream(DynamicPluginResourceLoader.java:23)
  - locked <0xbd47f420> (a com.atlassian.confluence.setup.velocity.DynamicPluginResourceLoader)
  at org.apache.velocity.Template.process(Template.java:136)
  at org.apache.velocity.runtime.resource.ResourceManagerImpl.loadResource(ResourceManagerImpl.java:466)
  ...  
  at com.atlassian.confluence.setup.sitemesh.ConfluenceSpaceDecoratorMapper.getDecorator(ConfluenceSpaceDecoratorMapper.java:57)
  ...


 All   Comments   Work Log   Change History      Sort Order: Ascending order - Click to sort in descending order
[ Permlink | « Hide ]
Matthew Jensen [Atlassian] added a comment - 03/Jun/07 08:28 PM - edited
Removing the need to synchronize on all (but perhaps the first) request would improve things dramatically.

A couple of ideas/notes:

  • Perhaps when found extract the resources into a temporary directory so they can be found without a jar scan
  • Perhaps even store the resources in memory, not sure how the memory usage is, maybe a weak referenced map cache would be good here. Commonly used templates would be loaded from memory
  • When fixing this we don't want to affect the class loading, just the resource loading.

[ Permlink | « Hide ]
Matthew Jensen [Atlassian] added a comment - 03/Jun/07 08:34 PM - edited
It's possible that there is a certain number (or size) of plugins that starts to expose this issue. The more plugins, means more time spent in the scan. Add a high load to this and you will get some major slowdown as more and more requests will back up waiting for a lock on the resource and class loaders.

[ Permlink | « Hide ]
Matthew Jensen [Atlassian] added a comment - 03/Jun/07 11:00 PM
Complete stack traces which show this potential problem.

[ Permlink | « Hide ]
Matthew Jensen [Atlassian] added a comment - 23/Jul/07 08:41 PM
This fix has been backported to 2.5.6.

[ Permlink | « Hide ]
Matthew Jensen [Atlassian] added a comment - 23/Jul/07 08:57 PM - edited
Source for patched resource loader

[ Permlink | « Hide ]
Matt Ryall [Atlassian] added a comment - 29/Jan/08 09:15 PM
To install this patch, put the attached class file in the following directory, creating paths under WEB-INF/classes/ as necessary.

confluence/WEB-INF/classes/com/atlassian/confluence/setup/velocity/

If the class file already exists in this location, please keep a back-up of this file in case something goes wrong.
Powered by Atlassian JIRA the Professional Issue Tracker. (Enterprise Edition, Version: 3.13_beta1-#328) - Bug/feature request - Atlassian news - Contact Administrators