Issue Summary

During construction of outbound email content, the Velocity template uses MailAttachmentsManagerImpl.inlineImages() (according to the interface docs) to:

• Looks for <img> tags and tries to inline image if it's an internal resource.
• Otherwise adds JWT token for every /secure/* image or leaves unchanged for
• Counts how many images there are that start with /secure/* so hasSecuredImage() can calculate the result needed to construct email footer
• Adds default 'alt' attribute which is an attachment file name in case img has none

This process relies on regular expressions, the performance of which varies depending on the length of the email. Processing appears to be CPU bound, on a single core (batching off == a single caseium thread)

Another effect is that mails are processed sequentially - if a mail takes a long time to process, other outbound mails in the queue must wait, resulting in mails not being delivered in a reasonable amount of time.

This is similar to JRASERVER-65963 affecting diff() on the mail content, the fix being to limit the amount of characters attempted to process.

Steps to Reproduce

1. Jira 9.12.16
2. Turn batching email notifications off
3. Have a low performance CPU core
   and/or
4. Have a very long email (for example, artificially increase jira.text.field.character.limit=1638350, make a custom text custom field, and max out the content)
5. Try to send a notification

Expected Results

• The mail is sent in a reasonable amount of time

Actual Results

• The mail sending is heavily delayed
• A single core CPU is very high (eg 99%), but the total CPU consumption is within expectation
• The mail queue flush performance is reduced or completely stalled
• The mail error queue size stays typical
• When examining thread dumps, the Caseium thread sending mail remains mostly or entirely on this stack trace:

  "Sending mailitem com.atlassian.jira.mail.IssueMailQueueItem@5165dc15[issue=com.atlassian.jira.issue.IssueImpl@1ca4ad09<SNIP>" #1564 daemon prio=5 os_prio=0 cpu=347827883.22ms elapsed=1962050.43s tid=0x00007f54f37df000 nid=0x105363 runnable  [0x00007f5085572000]
     java.lang.Thread.State: RUNNABLE
  	at java.util.regex.Pattern$Curly.match1(java.base@11.0.3/Pattern.java:4419)
  	at java.util.regex.Pattern$Curly.match(java.base@11.0.3/Pattern.java:4368)
  	at java.util.regex.Pattern$Ques.match(java.base@11.0.3/Pattern.java:4241)
  	at java.util.regex.Pattern$GroupHead.match(java.base@11.0.3/Pattern.java:4791)
  	at java.util.regex.Pattern$GroupTail.match(java.base@11.0.3/Pattern.java:4850)
  	at java.util.regex.Pattern$CharPropertyGreedy.match(java.base@11.0.3/Pattern.java:4293)
  	at java.util.regex.Pattern$GroupHead.match(java.base@11.0.3/Pattern.java:4791)
  	at java.util.regex.Pattern$GroupTail.match(java.base@11.0.3/Pattern.java:4850)
  	at java.util.regex.Pattern$Ques.match(java.base@11.0.3/Pattern.java:4240)
  	at java.util.regex.Pattern$BmpCharPropertyGreedy.match(java.base@11.0.3/Pattern.java:4331)
  	at java.util.regex.Pattern$BmpCharProperty.match(java.base@11.0.3/Pattern.java:3951)
  	at java.util.regex.Pattern$BmpCharPropertyGreedy.match(java.base@11.0.3/Pattern.java:4331)
  	at java.util.regex.Pattern$Slice.match(java.base@11.0.3/Pattern.java:4099)
  	at java.util.regex.Pattern$Curly.match1(java.base@11.0.3/Pattern.java:4419)
  	at java.util.regex.Pattern$Curly.match(java.base@11.0.3/Pattern.java:4368)
  	at java.util.regex.Pattern$Slice.match(java.base@11.0.3/Pattern.java:4099)
  	at java.util.regex.Pattern$GroupHead.match(java.base@11.0.3/Pattern.java:4791)
  	at java.util.regex.Pattern$Start.match(java.base@11.0.3/Pattern.java:3606)
  	at java.util.regex.Matcher.search(java.base@11.0.3/Matcher.java:1729)
  	at java.util.regex.Matcher.find(java.base@11.0.3/Matcher.java:746)
  	at com.atlassian.jira.mail.util.MailAttachmentsManagerImpl.processImages(MailAttachmentsManagerImpl.java:264)
  	at com.atlassian.jira.mail.util.MailAttachmentsManagerImpl.inlineImages(MailAttachmentsManagerImpl.java:308)
  	at jdk.internal.reflect.GeneratedMethodAccessor40483.invoke(Unknown Source)
  	at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@11.0.3/DelegatingMethodAccessorImpl.java:43)
  	at java.lang.reflect.Method.invoke(java.base@11.0.3/Method.java:566)
  	at org.apache.velocity.util.introspection.UberspectImpl$VelMethodImpl.doInvoke(UberspectImpl.java:385)
  	at org.apache.velocity.util.introspection.UberspectImpl$VelMethodImpl.invoke(UberspectImpl.java:374)
  	at com.atlassian.velocity.htmlsafe.introspection.UnboxingMethod.invoke(UnboxingMethod.java:28)
  	at org.apache.velocity.runtime.parser.node.ASTMethod.execute(ASTMethod.java:270) 
  <SNIP>

   

Workaround

Currently there is no known workaround for this behavior. A workaround will be added here when available