Suppress Excessive java.io.IOException: Broken pipe ERROR when audit log export fails

XMLWordPrintable

    • 1

      We have hundreds of ERRORs getting reported when a single audit log export fails. This leads to unnecessary log file rotation of atlassian-jira.log. We had JRASERVER-68615 similarly, we will need to suppress the unwanted java.io.IOException: Broken pipe .

      2022-03-23 17:05:14,674+0000 http-nio-8080-exec-3 ERROR admin 1025x47x1 1392vyj 172.29.176.147,172.50.0.3 /rest/auditing/1.0/events [c.a.audit.csv.AuditCsvWriter] Failed to append row to outputStream
      org.apache.catalina.connector.ClientAbortException: java.io.IOException: Broken pipe
      	at org.apache.catalina.connector.OutputBuffer.realWriteBytes(OutputBuffer.java:371)
      	at org.apache.catalina.connector.OutputBuffer.flushByteBuffer(OutputBuffer.java:845)
      	at org.apache.catalina.connector.OutputBuffer.append(OutputBuffer.java:750)
      	at org.apache.catalina.connector.OutputBuffer.writeBytes(OutputBuffer.java:406)
      	at org.apache.catalina.connector.OutputBuffer.write(OutputBuffer.java:384)
      	... 1 filtered
      	at com.atlassian.gzipfilter.RoutableServletOutputStream.write(RoutableServletOutputStream.java:134)
      	... 1 filtered
      	at com.sun.jersey.spi.container.ContainerResponse$CommittingOutputStream.write(ContainerResponse.java:135)
      	at java.base/sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:233)
      	at java.base/sun.nio.cs.StreamEncoder.implWrite(StreamEncoder.java:303)
      	at java.base/sun.nio.cs.StreamEncoder.implWrite(StreamEncoder.java:281)
      	at java.base/sun.nio.cs.StreamEncoder.write(StreamEncoder.java:125)
      	at java.base/java.io.OutputStreamWriter.write(OutputStreamWriter.java:208)
      	at java.base/java.io.BufferedWriter.write(BufferedWriter.java:184)
      	at java.base/java.io.BufferedWriter.flushBuffer(BufferedWriter.java:120)
      	at java.base/java.io.BufferedWriter.write(BufferedWriter.java:233)
      	at java.base/java.io.Writer.write(Writer.java:249)
      	at org.supercsv.io.AbstractCsvWriter.writeRow(AbstractCsvWriter.java:196)
      	at org.supercsv.io.AbstractCsvWriter.writeRow(AbstractCsvWriter.java:162)
      	at org.supercsv.io.CsvMapWriter.write(CsvMapWriter.java:60)
      	at com.atlassian.audit.csv.AuditCsvWriter.appendRow(AuditCsvWriter.java:118)
      	at com.atlassian.audit.ao.dao.AoAuditEntityDao.lambda$null$2(AoAuditEntityDao.java:137)
      	at net.java.ao.EntityManager.stream(EntityManager.java:907)
      	at com.atlassian.activeobjects.internal.EntityManagedActiveObjects.stream(EntityManagedActiveObjects.java:174)
      	at com.atlassian.activeobjects.osgi.TenantAwareActiveObjects.stream(TenantAwareActiveObjects.java:322)
      	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
      	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343)
      	at org.eclipse.gemini.blueprint.service.importer.support.internal.aop.ServiceInvoker.doInvoke(ServiceInvoker.java:56)
      	at org.eclipse.gemini.blueprint.service.importer.support.internal.aop.ServiceInvoker.invoke(ServiceInvoker.java:60)
      	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
      	at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:136)
      	at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:124)
      	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
      	at org.eclipse.gemini.blueprint.service.util.internal.aop.ServiceTCCLInterceptor.invokeUnprivileged(ServiceTCCLInterceptor.java:70)
      	at org.eclipse.gemini.blueprint.service.util.internal.aop.ServiceTCCLInterceptor.invoke(ServiceTCCLInterceptor.java:53)
      	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
      	at org.eclipse.gemini.blueprint.service.importer.support.LocalBundleContextAdvice.invoke(LocalBundleContextAdvice.java:57)
      	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
      	at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:136)
      	at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:124)
      	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
      	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212)
      	at com.sun.proxy.$Proxy746.stream(Unknown Source)
      	at com.atlassian.audit.ao.dao.AoAuditEntityDao.lambda$stream$3(AoAuditEntityDao.java:137)
      	at com.atlassian.sal.core.transaction.HostContextTransactionTemplate$1.doInTransaction(HostContextTransactionTemplate.java:21)
      	at com.atlassian.jira.DefaultHostContextAccessor.doInTransaction(DefaultHostContextAccessor.java:50)
      	at jdk.internal.reflect.GeneratedMethodAccessor246.invoke(Unknown Source)
      	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
      	at com.atlassian.plugin.util.ContextClassLoaderSettingInvocationHandler.invoke(ContextClassLoaderSettingInvocationHandler.java:26)
      	at com.sun.proxy.$Proxy422.doInTransaction(Unknown Source)
      	at jdk.internal.reflect.GeneratedMethodAccessor246.invoke(Unknown Source)
      	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
      	at com.atlassian.plugin.osgi.bridge.external.HostComponentFactoryBean$DynamicServiceInvocationHandler.invoke(HostComponentFactoryBean.java:130)
      	at com.sun.proxy.$Proxy422.doInTransaction(Unknown Source)
      	at com.atlassian.sal.core.transaction.HostContextTransactionTemplate.execute(HostContextTransactionTemplate.java:18)
      	at jdk.internal.reflect.GeneratedMethodAccessor263.invoke(Unknown Source)
      	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
      	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343)
      	at org.eclipse.gemini.blueprint.service.importer.support.internal.aop.ServiceInvoker.doInvoke(ServiceInvoker.java:56)
      	at org.eclipse.gemini.blueprint.service.importer.support.internal.aop.ServiceInvoker.invoke(ServiceInvoker.java:60)
      	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
      	at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:136)
      	at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:124)
      	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
      	at org.eclipse.gemini.blueprint.service.util.internal.aop.ServiceTCCLInterceptor.invokeUnprivileged(ServiceTCCLInterceptor.java:70)
      	at org.eclipse.gemini.blueprint.service.util.internal.aop.ServiceTCCLInterceptor.invoke(ServiceTCCLInterceptor.java:53)
      	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
      	at org.eclipse.gemini.blueprint.service.importer.support.LocalBundleContextAdvice.invoke(LocalBundleContextAdvice.java:57)
      	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
      	at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:136)
      	at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:124)
      	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
      	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212)
      	at com.sun.proxy.$Proxy747.execute(Unknown Source)
      	at com.atlassian.audit.ao.dao.AoAuditEntityDao.stream(AoAuditEntityDao.java:136)
      	at com.atlassian.audit.ao.service.DatabaseAuditSearchService.stream(DatabaseAuditSearchService.java:48)
      	at com.atlassian.audit.ao.service.RateLimitedSearchService.tryStream(RateLimitedSearchService.java:93)
      	at com.atlassian.audit.ao.service.RateLimitedSearchService.stream(RateLimitedSearchService.java:81)
      	at com.atlassian.audit.ao.service.AnalyticsTrackedAuditSearchService.stream(AnalyticsTrackedAuditSearchService.java:65)
      	at com.atlassian.audit.ao.service.AuditedSearchService.stream(AuditedSearchService.java:81)
      	at com.atlassian.audit.ao.service.RestrictiveSearchService.stream(RestrictiveSearchService.java:47)
      	at com.atlassian.audit.csv.AuditCsvExporter.export(AuditCsvExporter.java:40)
      	at com.atlassian.audit.rest.AuditRestResource.lambda$generateCsvResponse$2(AuditRestResource.java:310)
      	at com.sun.jersey.core.impl.provider.entity.StreamingOutputProvider.writeTo(StreamingOutputProvider.java:71)
      	at com.sun.jersey.core.impl.provider.entity.StreamingOutputProvider.writeTo(StreamingOutputProvider.java:57)
      	at com.sun.jersey.spi.container.ContainerResponse.write(ContainerResponse.java:302)
      	... 5 filtered
      	at com.atlassian.plugins.rest.module.RestDelegatingServletFilter$JerseyOsgiServletContainer.doFilter(RestDelegatingServletFilter.java:160)
      	... 1 filtered
      	at com.atlassian.plugins.rest.module.RestDelegatingServletFilter.doFilter(RestDelegatingServletFilter.java:70)
      	... 36 filtered
      	at com.atlassian.jira.plugin.mobile.web.filter.MobileAppRequestFilter.doFilter(MobileAppRequestFilter.java:59)
      	... 4 filtered
      	at com.atlassian.jira.plugin.mobile.login.MobileLoginSuccessFilter.doFilter(MobileLoginSuccessFilter.java:54)
      	... 3 filtered
      	at com.atlassian.diagnostics.internal.platform.monitor.http.HttpRequestMonitoringFilter.doFilter(HttpRequestMonitoringFilter.java:55)
      	... 8 filtered
      	at com.atlassian.web.servlet.plugin.request.RedirectInterceptingFilter.doFilter(RedirectInterceptingFilter.java:21)
      	... 43 filtered
      	at com.atlassian.ratelimiting.internal.filter.RateLimitFilter.doFilter(RateLimitFilter.java:73)
      	... 3 filtered
      	at com.atlassian.troubleshooting.thready.filter.AbstractThreadNamingFilter.doFilter(AbstractThreadNamingFilter.java:46)
      	... 17 filtered
      	at com.atlassian.jira.security.JiraSecurityFilter.lambda$doFilter$0(JiraSecurityFilter.java:66)
      	... 1 filtered
      	at com.atlassian.jira.security.JiraSecurityFilter.doFilter(JiraSecurityFilter.java:64)
      	... 16 filtered
      	at com.atlassian.plugins.rest.module.servlet.RestSeraphFilter.doFilter(RestSeraphFilter.java:38)
      	... 3 filtered
      	at com.atlassian.pats.web.filter.TokenBasedAuthenticationFilter.doFilter(TokenBasedAuthenticationFilter.java:83)
      	... 19 filtered
      	at com.atlassian.jira.servermetrics.CorrelationIdPopulatorFilter.doFilter(CorrelationIdPopulatorFilter.java:30)
      	... 5 filtered
      	at com.atlassian.plugins.authentication.impl.basicauth.filter.DisableBasicAuthFilter.doFilter(DisableBasicAuthFilter.java:70)
      	... 8 filtered
      	at com.atlassian.ratelimiting.internal.filter.RateLimitPreAuthFilter.doFilter(RateLimitPreAuthFilter.java:71)
      	... 3 filtered
      	at com.atlassian.web.servlet.plugin.request.RedirectInterceptingFilter.doFilter(RedirectInterceptingFilter.java:21)
      	... 4 filtered
      	at com.atlassian.troubleshooting.thready.filter.AbstractThreadNamingFilter.doFilter(AbstractThreadNamingFilter.java:46)
      	... 3 filtered
      	at com.atlassian.web.servlet.plugin.LocationCleanerFilter.doFilter(LocationCleanerFilter.java:36)
      	... 26 filtered
      	at com.atlassian.jira.servermetrics.MetricsCollectorFilter.doFilter(MetricsCollectorFilter.java:25)
      	... 25 filtered
      	at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191)
      	at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
      	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
      	at java.base/java.lang.Thread.run(Thread.java:829)
      Caused by: java.io.IOException: Broken pipe
      	at java.base/sun.nio.ch.FileDispatcherImpl.write0(Native Method)
      	at java.base/sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47)
      	at java.base/sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:113)
      	at java.base/sun.nio.ch.IOUtil.write(IOUtil.java:79)
      	at java.base/sun.nio.ch.IOUtil.write(IOUtil.java:50)
      	... 12 filtered
      	... 348 more
      2022-03-23 17:05:14,677+0000 http-nio-8080-exec-3 ERROR admin 1025x47x1 1392vyj 172.29.176.147,172.50.0.3 /rest/auditing/1.0/events [c.a.audit.csv.AuditCsvExporter] Failed to close mapWriter
      org.apache.catalina.connector.ClientAbortException: java.io.IOException: Broken pipe
      	at org.apache.catalina.connector.OutputBuffer.realWriteBytes(OutputBuffer.java:371)
      	at org.apache.catalina.connector.OutputBuffer.flushByteBuffer(OutputBuffer.java:845)
      	at org.apache.catalina.connector.OutputBuffer.append(OutputBuffer.java:750)
      	at org.apache.catalina.connector.OutputBuffer.writeBytes(OutputBuffer.java:406)
      	at org.apache.catalina.connector.OutputBuffer.write(OutputBuffer.java:384)
      	... 1 filtered
      	at com.atlassian.gzipfilter.RoutableServletOutputStream.write(RoutableServletOutputStream.java:134)
      	... 1 filtered
      	at com.sun.jersey.spi.container.ContainerResponse$CommittingOutputStream.write(ContainerResponse.java:135)
      	at java.base/sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:233)
      	at java.base/sun.nio.cs.StreamEncoder.implWrite(StreamEncoder.java:303)
      	at java.base/sun.nio.cs.StreamEncoder.implWrite(StreamEncoder.java:281)
      	at java.base/sun.nio.cs.StreamEncoder.write(StreamEncoder.java:125)
      	at java.base/java.io.OutputStreamWriter.write(OutputStreamWriter.java:208)
      	at java.base/java.io.BufferedWriter.flushBuffer(BufferedWriter.java:120)
      	at java.base/java.io.BufferedWriter.close(BufferedWriter.java:268)
      	at java.base/java.io.BufferedWriter.close(BufferedWriter.java:269)
      	at org.supercsv.io.AbstractCsvWriter.close(AbstractCsvWriter.java:77)
      	at com.atlassian.audit.csv.AuditCsvWriter.close(AuditCsvWriter.java:178)
      	at com.atlassian.audit.csv.AuditCsvExporter.export(AuditCsvExporter.java:41)
      	at com.atlassian.audit.rest.AuditRestResource.lambda$generateCsvResponse$2(AuditRestResource.java:310)
      	at com.sun.jersey.core.impl.provider.entity.StreamingOutputProvider.writeTo(StreamingOutputProvider.java:71)
      	at com.sun.jersey.core.impl.provider.entity.StreamingOutputProvider.writeTo(StreamingOutputProvider.java:57)
      	at com.sun.jersey.spi.container.ContainerResponse.write(ContainerResponse.java:302)
      	... 5 filtered
      	at com.atlassian.plugins.rest.module.RestDelegatingServletFilter$JerseyOsgiServletContainer.doFilter(RestDelegatingServletFilter.java:160)
      	... 1 filtered
      	at com.atlassian.plugins.rest.module.RestDelegatingServletFilter.doFilter(RestDelegatingServletFilter.java:70)
      	... 36 filtered
      	at com.atlassian.jira.plugin.mobile.web.filter.MobileAppRequestFilter.doFilter(MobileAppRequestFilter.java:59)
      	... 4 filtered
      	at com.atlassian.jira.plugin.mobile.login.MobileLoginSuccessFilter.doFilter(MobileLoginSuccessFilter.java:54)
      	... 3 filtered
      	at com.atlassian.diagnostics.internal.platform.monitor.http.HttpRequestMonitoringFilter.doFilter(HttpRequestMonitoringFilter.java:55)
      	... 8 filtered
      	at com.atlassian.web.servlet.plugin.request.RedirectInterceptingFilter.doFilter(RedirectInterceptingFilter.java:21)
      	... 43 filtered
      	at com.atlassian.ratelimiting.internal.filter.RateLimitFilter.doFilter(RateLimitFilter.java:73)
      	... 3 filtered
      	at com.atlassian.troubleshooting.thready.filter.AbstractThreadNamingFilter.doFilter(AbstractThreadNamingFilter.java:46)
      	... 17 filtered
      	at com.atlassian.jira.security.JiraSecurityFilter.lambda$doFilter$0(JiraSecurityFilter.java:66)
      	... 1 filtered
      	at com.atlassian.jira.security.JiraSecurityFilter.doFilter(JiraSecurityFilter.java:64)
      	... 16 filtered
      	at com.atlassian.plugins.rest.module.servlet.RestSeraphFilter.doFilter(RestSeraphFilter.java:38)
      	... 3 filtered
      	at com.atlassian.pats.web.filter.TokenBasedAuthenticationFilter.doFilter(TokenBasedAuthenticationFilter.java:83)
      	... 19 filtered
      	at com.atlassian.jira.servermetrics.CorrelationIdPopulatorFilter.doFilter(CorrelationIdPopulatorFilter.java:30)
      	... 5 filtered
      	at com.atlassian.plugins.authentication.impl.basicauth.filter.DisableBasicAuthFilter.doFilter(DisableBasicAuthFilter.java:70)
      	... 8 filtered
      	at com.atlassian.ratelimiting.internal.filter.RateLimitPreAuthFilter.doFilter(RateLimitPreAuthFilter.java:71)
      	... 3 filtered
      	at com.atlassian.web.servlet.plugin.request.RedirectInterceptingFilter.doFilter(RedirectInterceptingFilter.java:21)
      	... 4 filtered
      	at com.atlassian.troubleshooting.thready.filter.AbstractThreadNamingFilter.doFilter(AbstractThreadNamingFilter.java:46)
      	... 3 filtered
      	at com.atlassian.web.servlet.plugin.LocationCleanerFilter.doFilter(LocationCleanerFilter.java:36)
      	... 26 filtered
      	at com.atlassian.jira.servermetrics.MetricsCollectorFilter.doFilter(MetricsCollectorFilter.java:25)
      	... 25 filtered
      	at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191)
      	at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
      	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
      	at java.base/java.lang.Thread.run(Thread.java:829)
      	Suppressed: org.apache.catalina.connector.ClientAbortException: java.io.IOException: Broken pipe
      	... 5 filtered
      		at com.atlassian.gzipfilter.RoutableServletOutputStream.close(RoutableServletOutputStream.java:62)
      	... 1 filtered
      		at com.sun.jersey.spi.container.ContainerResponse$CommittingOutputStream.close(ContainerResponse.java:152)
      		at java.base/sun.nio.cs.StreamEncoder.implClose(StreamEncoder.java:341)
      		at java.base/sun.nio.cs.StreamEncoder.close(StreamEncoder.java:161)
      		at java.base/java.io.OutputStreamWriter.close(OutputStreamWriter.java:255)
      		at java.base/java.io.BufferedWriter.close(BufferedWriter.java:267)
      		... 264 more
      	Caused by: java.io.IOException: Broken pipe
      		at java.base/sun.nio.ch.FileDispatcherImpl.write0(Native Method)
      		at java.base/sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47)
      		at java.base/sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:113)
      		at java.base/sun.nio.ch.IOUtil.write(IOUtil.java:79)
      		at java.base/sun.nio.ch.IOUtil.write(IOUtil.java:50)
      		at java.base/sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:462)
      	... 12 filtered
      		... 275 more
      Caused by: java.io.IOException: Broken pipe
      	at java.base/sun.nio.ch.FileDispatcherImpl.write0(Native Method)
      	at java.base/sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47)
      	at java.base/sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:113)
      	at java.base/sun.nio.ch.IOUtil.write(IOUtil.java:79)
      	at java.base/sun.nio.ch.IOUtil.write(IOUtil.java:50)
      	... 12 filtered
      	... 279 more
      

            Assignee:
            Unassigned
            Reporter:
            Nikhil Janardhanan
            Votes:
            3 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated: