Uploaded image for project: 'Jira Data Center'
  1. Jira Data Center
  2. JRASERVER-70637

Incoming mail is consuming all caesium threads blocking other scheduled tasks from running

XMLWordPrintable

      Issue Summary

      In larger instances of the software, there may be dozens of mail handlers configured or there me so much mail load that needs to be processed that it's possible for incoming mail to consume all four caesium threads causing other tasks such as notifications, DVCS syncs, etc to fail silently.
      Ideally, we want to have a large number of caesium workes, see JRASERVER-65809.

      Steps to Reproduce

      1. create a large instance
      2. configure dozens of mail handlers OR a few with a tremendous amount of load, in one instance there were 680+ new messages from one handler in a period of less than 24 hours

      Expected Results

      JIRA gracefully handles all scheduled tasks and allows other tasks besides mail handlers to be processed

      Actual Results

      incoming mail will hold all four threads until the queue is processed

      Thread dumps show all four caesium threads running with a similar stack trace: 

      at java.net.SocketInputStream.socketRead0(java.base@11.0.6/Native Method)
	at java.net.SocketInputStream.socketRead(java.base@11.0.6/SocketInputStream.java:115)
	at java.net.SocketInputStream.read(java.base@11.0.6/SocketInputStream.java:168)
	at java.net.SocketInputStream.read(java.base@11.0.6/SocketInputStream.java:140)
	at sun.security.ssl.SSLSocketInputRecord.read(java.base@11.0.6/SSLSocketInputRecord.java:448)
	at sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(java.base@11.0.6/SSLSocketInputRecord.java:68)
	at sun.security.ssl.SSLSocketImpl.readApplicationRecord(java.base@11.0.6/SSLSocketImpl.java:1103)
	at sun.security.ssl.SSLSocketImpl$AppInputStream.read(java.base@11.0.6/SSLSocketImpl.java:823)
	- locked <0x000000095c6597b0> (a sun.security.ssl.SSLSocketImpl$AppInputStream)
	at com.sun.mail.util.TraceInputStream.read(TraceInputStream.java:126)
	at java.io.BufferedInputStream.fill(java.base@11.0.6/BufferedInputStream.java:252)
	at java.io.BufferedInputStream.read(java.base@11.0.6/BufferedInputStream.java:271)
	- locked <0x000000095c65a020> (a java.io.BufferedInputStream)
	at com.sun.mail.iap.ResponseInputStream.readResponse(ResponseInputStream.java:103)
	at com.sun.mail.iap.Response.<init>(Response.java:131)
	at com.sun.mail.imap.protocol.IMAPResponse.<init>(IMAPResponse.java:60)
	at com.sun.mail.imap.protocol.IMAPProtocol.readResponse(IMAPProtocol.java:407)
	at com.sun.mail.iap.Protocol.command(Protocol.java:355)
	- locked <0x000000095c65a170> (a com.sun.mail.imap.protocol.IMAPProtocol)
	at com.sun.mail.imap.protocol.IMAPProtocol.fetch(IMAPProtocol.java:2151)
	at com.sun.mail.imap.protocol.IMAPProtocol.fetch(IMAPProtocol.java:2143)
	at com.sun.mail.imap.IMAPMessage.loadEnvelope(IMAPMessage.java:1493)
	- locked <0x0000001287fc4830> (a java.lang.Object)
	- locked <0x0000001287fe5040> (a com.sun.mail.imap.IMAPMessage)
	at com.sun.mail.imap.IMAPMessage.getSubject(IMAPMessage.java:441)
	at com.atlassian.jira.service.services.mail.MailFetcherService$MessageProviderImpl.getAndProcessMail(MailFetcherService.java:260)

      These may vary based on mailbox type but the threads are all running and checking mail.

      Workaround

      Until a fix is implemented in JRASERVER-65809, it may be best practice to stagger how often mail handlers are checking mail.

              Unassigned Unassigned
              rvolkmar Ronnie Volkmar
              Votes:
              37 Vote for this issue
              Watchers:
              42 Start watching this issue

                Created:
                Updated: