Details
-
Bug
-
Resolution: Fixed
-
Medium
-
2.5.1
-
None
-
None
Description
Thread dumps report:
Found one Java-level deadlock:
=============================
"btpool0-3136":
waiting to lock monitor 0x00002aac013418f0 (object 0x00002aab3ca97360, a org.mortbay.io.nio.SelectorManager$SelectSet),
which is held by "btpool0-1 - Acceptor0 ProxyPassingSelectChannelConnector@0.0.0.0:8100"
"btpool0-1 - Acceptor0 ProxyPassingSelectChannelConnector@0.0.0.0:8100":
waiting to lock monitor 0x00002aac006aa078 (object 0x00002aab1697b6e0, a org.mortbay.jetty.nio.SelectChannelConnector$ConnectorEndPoint),
which is held by "btpool0-3136"Java stack information for the threads listed above:
===================================================
"btpool0-3136":
at org.mortbay.io.nio.SelectorManager$SelectSet.scheduleIdle(SelectorManager.java:782)waiting to lock <0x00002aab3ca97360> (a org.mortbay.io.nio.SelectorManager$SelectSet)
at org.mortbay.io.nio.SelectChannelEndPoint.scheduleIdle(SelectChannelEndPoint.java:159)
at org.mortbay.io.nio.SelectChannelEndPoint.blockWritable(SelectChannelEndPoint.java:293)
locked <0x00002aab1697b6e0> (a org.mortbay.jetty.nio.SelectChannelConnector$ConnectorEndPoint)
at org.mortbay.jetty.AbstractGenerator$Output.blockForOutput(AbstractGenerator.java:544)
at org.mortbay.jetty.AbstractGenerator$Output.flush(AbstractGenerator.java:571)
at org.mortbay.jetty.HttpConnection$Output.flush(HttpConnection.java:1010)
at org.mortbay.jetty.AbstractGenerator$Output.write(AbstractGenerator.java:650)
at org.mortbay.jetty.AbstractGenerator$Output.write(AbstractGenerator.java:579)
at org.mortbay.util.IO.copy(IO.java:196)
at org.mortbay.resource.Resource.writeTo(Resource.java:562)
at org.mortbay.jetty.servlet.DefaultServlet.sendData(DefaultServlet.java:755)
at org.mortbay.jetty.servlet.DefaultServlet.doGet(DefaultServlet.java:451)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
at org.mortbay.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1217)
at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:46)
at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:66)
at com.atlassian.applinks.core.rest.context.ContextFilter.doFilter(ContextFilter.java:25)
at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:74)
at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42)
at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:77)
at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:63)
at org.mortbay.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1209)
at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:46)
at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:77)
at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:63)
at org.mortbay.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1209)
at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:83)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76)
at org.mortbay.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1209)
at com.cenqua.crucible.filters.CrucibleFilter.doFilter(CrucibleFilter.java:141)
at org.mortbay.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1209)
at com.cenqua.fisheye.web.filters.TotalityFilter.doFilter(TotalityFilter.java:188)
at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:236)
at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:167)
at org.mortbay.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1209)
at com.atlassian.security.auth.trustedapps.filter.TrustedApplicationsFilter.doFilter(TrustedApplicationsFilter.java:98)
at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:236)
at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:167)
at org.mortbay.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1209)
at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:46)
at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter$1.doFilter(DelegatingPluginFilter.java:66)
at com.atlassian.oauth.serviceprovider.internal.servlet.OAuthFilter.doFilter(OAuthFilter.java:69)
at com.atlassian.plugin.servlet.filter.DelegatingPluginFilter.doFilter(DelegatingPluginFilter.java:74)
at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:42)
at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:77)
at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:63)
at org.mortbay.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1209)
at com.cenqua.fisheye.web.filters.CustomIncludableGzipFilter.doFilter(CustomIncludableGzipFilter.java:19)
at org.mortbay.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1209)
at com.cenqua.fisheye.web.filters.ProductInfoFilter.doFilter(ProductInfoFilter.java:36)
at org.mortbay.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1209)
at com.atlassian.plugin.servlet.filter.IteratingFilterChain.doFilter(IteratingFilterChain.java:46)
at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:77)
at com.atlassian.plugin.servlet.filter.ServletFilterModuleContainerFilter.doFilter(ServletFilterModuleContainerFilter.java:63)
at org.mortbay.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1209)
at com.cenqua.fisheye.web.filters.UpfrontFilter.doFilter(UpfrontFilter.java:52)
at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:236)
at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:167)
at org.mortbay.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1209)
at com.atlassian.fecru.profiling.ProfilingServletFilter.doFilter(ProfilingServletFilter.java:73)
at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:236)
at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:167)
at org.mortbay.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1209)
at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:388)
at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:418)
at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at org.mortbay.jetty.Server.handle(Server.java:326)
at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:926)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
at org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.java:451)
"btpool0-1 - Acceptor0 ProxyPassingSelectChannelConnector@0.0.0.0:8100":
at org.mortbay.io.nio.SelectChannelEndPoint.updateKey(SelectChannelEndPoint.java:320)
waiting to lock <0x00002aab1697b6e0> (a org.mortbay.jetty.nio.SelectChannelConnector$ConnectorEndPoint)
at org.mortbay.io.nio.SelectChannelEndPoint.close(SelectChannelEndPoint.java:456)
at org.mortbay.jetty.nio.SelectChannelConnector$ConnectorEndPoint.close(SelectChannelConnector.java:362)
at org.mortbay.io.nio.SelectChannelEndPoint.idleExpired(SelectChannelEndPoint.java:174)
at org.mortbay.io.nio.SelectChannelEndPoint$IdleTask.expire(SelectChannelEndPoint.java:489)
at org.mortbay.thread.Timeout.tick(Timeout.java:137)
locked <0x00002aab3ca97360> (a org.mortbay.io.nio.SelectorManager$SelectSet)
at org.mortbay.thread.Timeout.tick(Timeout.java:153)
at org.mortbay.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:753)
at org.mortbay.io.nio.SelectorManager.doSelect(SelectorManager.java:191)
at org.mortbay.jetty.nio.SelectChannelConnector.accept(SelectChannelConnector.java:124)
at org.mortbay.jetty.AbstractConnector$Acceptor.run(AbstractConnector.java:708)
at org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.java:451)
Found 1 deadlock.
This appears to be Jetty bug:
http://jira.codehaus.org/browse/JETTY-1264
fixed in 6.1.26
Latest Fisheye version 2.5.1 is running on 6.1.24
We should upgrade Jetty to help prevent this issue.