[JSDSERVER-6099] Emails with Large Size Attachments Fail/Are slow To Generate Ticket in IMAP Mailbox

Type: Bug
Resolution: Fixed
Priority: Highest
Fix Version/s: 4.9.0
Affects Version/s: 3.14.2
Component/s: Email - Incoming
Labels:
None

Support reference count:
20
Symptom Severity:
Severity 1 - Critical
UIS:
42
Bug Fix Policy:
View Atlassian Server bug fix policy

Summary

Emails sent to an IMAP custom Service Desk inbox with attachments bigger than 2Mb will not create a ticket in their JSD project. The email is read in the mailbox though but there is no record of JSD ever processing it. The issue is reproducible by the customer as the email is currently sitting in their inbox as read, and has been marked as unread several times. The email gets then marked as read with no success in creating the ticket in JSD.

Steps to Reproduce

Setup custom secure IMAP mail server
Receive an email with attachment of around 2Mb or larger
Notice the email will be read, but no record is logged in Jira or the Attachment takes 15 mins to be processed.

NOTE: This could not be reproduced in our Atlassian test environment.

Expected Results

Email is set with attachment
Ticket is created and a log created in Jira

Actual Results

Email is marked as read but JSD ticket does not get generated and the following stack trace is found in the logs:

2018-10-17 08:05:21,044 ERROR [] Caesium-1-4 ServiceRunner     Error while serializing message
com.sun.mail.util.FolderClosedIOException
        at com.sun.mail.imap.IMAPInputStream.forceCheckExpunged(IMAPInputStream.java:107) [javax.mail-1.6.0.jar:1.6.0]
        at com.sun.mail.imap.IMAPInputStream.fill(IMAPInputStream.java:158) [javax.mail-1.6.0.jar:1.6.0]
        at com.sun.mail.imap.IMAPInputStream.read(IMAPInputStream.java:218) [javax.mail-1.6.0.jar:1.6.0]
        at com.sun.mail.imap.IMAPInputStream.read(IMAPInputStream.java:244) [javax.mail-1.6.0.jar:1.6.0]
        at com.sun.mail.imap.IMAPMessage.writeTo(IMAPMessage.java:849) [javax.mail-1.6.0.jar:1.6.0]
        at com.atlassian.jira.internal.mail.processor.util.MimeMessageBase64ChunkStream.fullyLoadMessageToMemory(MimeMessageBase64ChunkStream.java:32) [?:?]
        at com.atlassian.jira.internal.mail.processor.util.MimeMessageBase64ChunkStream.<init>(MimeMessageBase64ChunkStream.java:22) [?:?]
        at com.atlassian.jira.internal.mail.processor.feature.puller.mailitem.MailItemStore.createMailItemChunk(MailItemStore.java:239) [?:?]
        at com.atlassian.jira.internal.mail.processor.feature.puller.mailitem.MailItemStore.lambda$null$4(MailItemStore.java:214) [?:?]
        at io.atlassian.fugue.Option.map(Option.java:259) [fugue-4.5.0.jar:?]
        at com.atlassian.jira.internal.mail.processor.feature.puller.mailitem.MailItemStore.lambda$createMailItem$5(MailItemStore.java:212) [?:?]
        at com.atlassian.pocketknife.internal.querydsl.DatabaseAccessorImpl.lambda$execute$0(DatabaseAccessorImpl.java:69) [?:?]
        at com.atlassian.sal.core.rdbms.DefaultTransactionalExecutor.executeInternal(DefaultTransactionalExecutor.java:86) [?:?]
        at com.atlassian.sal.core.rdbms.DefaultTransactionalExecutor.lambda$execute$0(DefaultTransactionalExecutor.java:42) [?:?]
        at com.atlassian.jira.database.DatabaseAccessorImpl.runInManagedTransaction(DatabaseAccessorImpl.java:125) [classes/:?]
        at sun.reflect.GeneratedMethodAccessor234.invoke(Unknown Source) [?:?]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [?:1.8.0_102]
        at java.lang.reflect.Method.invoke(Method.java:498) [?:1.8.0_102]
        at com.atlassian.plugin.util.ContextClassLoaderSettingInvocationHandler.invoke(ContextClassLoaderSettingInvocationHandler.java:26) [atlassian-plugins-core-4.5.1.jar:?]
        at com.sun.proxy.$Proxy114.runInManagedTransaction(Unknown Source) [?:?]
        at sun.reflect.GeneratedMethodAccessor234.invoke(Unknown Source) [?:?]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [?:1.8.0_102]
        at java.lang.reflect.Method.invoke(Method.java:498) [?:1.8.0_102]
        at com.atlassian.plugin.osgi.bridge.external.HostComponentFactoryBean$DynamicServiceInvocationHandler.invoke(HostComponentFactoryBean.java:136) [atlassian-plugins-osgi-bridge-4.5.1.jar:?]
        at com.sun.proxy.$Proxy114.runInManagedTransaction(Unknown Source) [?:?]
        at com.atlassian.sal.jira.rdbms.JiraHostConnectionAccessor.runInStartedOrExistingTransaction(JiraHostConnectionAccessor.java:130) [jira-sal-plugin-7.11.2_1533715698000.jar:?]
        at com.atlassian.sal.jira.rdbms.JiraHostConnectionAccessor.execute(JiraHostConnectionAccessor.java:60) [jira-sal-plugin-7.11.2_1533715698000.jar:?]
        at com.atlassian.sal.core.rdbms.DefaultTransactionalExecutor.execute(DefaultTransactionalExecutor.java:39) [jira-sal-plugin-7.11.2_1533715698000.jar:?]
        at com.atlassian.pocketknife.internal.querydsl.DatabaseAccessorImpl.execute(DatabaseAccessorImpl.java:67) [jira-email-processor-plugin-2.11.0.jar:?]
        at com.atlassian.pocketknife.internal.querydsl.DatabaseAccessorImpl.runInTransaction(DatabaseAccessorImpl.java:43) [jira-email-processor-plugin-2.11.0.jar:?]
        at com.atlassian.pocketknife.api.querydsl.DatabaseAccessor.runInTransaction(DatabaseAccessor.java:134) [jira-email-processor-plugin-2.11.0.jar:?]
        at com.atlassian.jira.internal.mail.processor.querydsl.QueryDslHelper.runInTransaction(QueryDslHelper.java:53) [jira-email-processor-plugin-2.11.0.jar:?]
        at com.atlassian.jira.internal.mail.processor.feature.puller.mailitem.MailItemStore.createMailItem(MailItemStore.java:208) [jira-email-processor-plugin-2.11.0.jar:?]
        at com.atlassian.jira.internal.mail.processor.feature.puller.mailitem.MailItemManager.createMailItemFromMessage(MailItemManager.java:38) [jira-email-processor-plugin-2.11.0.jar:?]
        at com.atlassian.jira.internal.mail.processor.feature.puller.MailPullerWorker.persistMailMessage(MailPullerWorker.java:209) [jira-email-processor-plugin-2.11.0.jar:?]
        at com.atlassian.jira.internal.mail.processor.feature.puller.MailPullerWorker.pullEmailForConnection(MailPullerWorker.java:113) [jira-email-processor-plugin-2.11.0.jar:?]
        at com.atlassian.jira.internal.mail.processor.feature.puller.MailPullerWorker.pullMailFromAllValidChannels(MailPullerWorker.java:76) [jira-email-processor-plugin-2.11.0.jar:?]
        at com.atlassian.jira.internal.mail.processor.feature.puller.MailPullerService.run(MailPullerService.java:33) [jira-email-processor-plugin-2.11.0.jar:?]
        at com.atlassian.jira.internal.mail.processor.services.MailPullerExecutor.run(MailPullerExecutor.java:29) [jira-email-processor-plugin-2.11.0.jar:?]
        at com.atlassian.jira.internal.mail.processor.services.AbstractMailExecutor.execute(AbstractMailExecutor.java:45) [jira-email-processor-plugin-2.11.0.jar:?]
        at com.atlassian.jira.internal.mail.processor.services.MailJobRunner.runJob(MailJobRunner.java:35) [jira-email-processor-plugin-2.11.0.jar:?]
        at com.atlassian.scheduler.core.JobLauncher.runJob(JobLauncher.java:153) [atlassian-scheduler-core-1.7.0.jar:?]
        at com.atlassian.scheduler.core.JobLauncher.launchAndBuildResponse(JobLauncher.java:118) [atlassian-scheduler-core-1.7.0.jar:?]
        at com.atlassian.scheduler.core.JobLauncher.launch(JobLauncher.java:97) [atlassian-scheduler-core-1.7.0.jar:?]
        at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService.launchJob(CaesiumSchedulerService.java:443) [atlassian-scheduler-caesium-1.3.3.jar:?]
        at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService.executeClusteredJob(CaesiumSchedulerService.java:438) [atlassian-scheduler-caesium-1.3.3.jar:?]
        at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService.executeClusteredJobWithRecoveryGuard(CaesiumSchedulerService.java:462) [atlassian-scheduler-caesium-1.3.3.jar:?]
        at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService.executeQueuedJob(CaesiumSchedulerService.java:390) [atlassian-scheduler-caesium-1.3.3.jar:?]
        at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService$1.consume(CaesiumSchedulerService.java:285) [atlassian-scheduler-caesium-1.3.3.jar:?]
        at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService$1.consume(CaesiumSchedulerService.java:282) [atlassian-scheduler-caesium-1.3.3.jar:?]
        at com.atlassian.scheduler.caesium.impl.SchedulerQueueWorker.executeJob(SchedulerQueueWorker.java:65) [atlassian-scheduler-caesium-1.3.3.jar:?]
        at com.atlassian.scheduler.caesium.impl.SchedulerQueueWorker.executeNextJob(SchedulerQueueWorker.java:59) [atlassian-scheduler-caesium-1.3.3.jar:?]
        at com.atlassian.scheduler.caesium.impl.SchedulerQueueWorker.run(SchedulerQueueWorker.java:34) [atlassian-scheduler-caesium-1.3.3.jar:?]
        at java.lang.Thread.run(Thread.java:745) [?:1.8.0_102]
2018-10-17 08:05:21,106 DEBUG [] Caesium-1-4 ServiceRunner     The result of deletion 0
2018-10-17 08:05:21,106 DEBUG [] Caesium-1-4 ServiceRunner     The result of deletion 1

This always happens for all the emails that are slow or fail to be processed.

Notes

By looking at the connectivity logs for that inbox, we can see several of [AUTHENTICATIONFAILED] Invalid credentials (Failure) errors in the last month.
Forward the email directly to the default Jira inbox, a ticket was created

Workaround

Workaround 1

There's been some evidence that moving to POP mitigates the bug

Workaround 2

There's been some evidence that adding the 2 following JVM parameters and re-starting the Jira application also helps mitigate the bug:

-Dmail.imap.fetchsize=1048576
-Dmail.imaps.fetchsize=1048576

These settings will increase the Java Mail Prefetch size from 16kb to 1mb, in order to limit the number of requests sent from the Service Desk mail handler to the network/

The steps are:

Stop the Jira application (stop all the Jira nodes if using Jira Data Center)

Add the following JVM parameters as per this documentation

-Dmail.imap.fetchsize=1048576
-Dmail.imaps.fetchsize=1048576

Backup the Jira Database
Run the following DELETE query in the Jira Database (this step will help reset the Service Desk Mail Puller job, in case it was stuck. Note the row corresponding to this job will be automatically re-created in the rundetails table after a Jira re-start)
```
delete from rundetails where job_id = 'com.atlassian.jira.internal.mail.services.MailPullerJobRunner';
```

Start the Jira application (or start all the Jira nodes id using Jira Data Center)

is cloned from

JSDCLOUD-6172 Emails with Large Size Attachments Fail To Generate Ticket in IMAP Mailbox

Closed

is cloned by: JSMDC-5460 You do not have permission to view this issue

mentioned in: Page Failed to load; Page Failed to load; Page Failed to load

Form Name

Jeramy added a comment - 26/Sep/2024 4:53 PM - edited

This is still an issue with JSM 5.12.12 for Data Center. And the first workaround isn't even feasible anymore. I was informed by our IT department that Microsoft killed POP.

We added workaround #2, but that only fixed the issue for a few days.

Jeramy added a comment - 26/Sep/2024 4:53 PM - edited This is still an issue with JSM 5.12.12 for Data Center. And the first workaround isn't even feasible anymore. I was informed by our IT department that Microsoft killed POP. We added workaround #2, but that only fixed the issue for a few days.

Nélio Vasconcelos added a comment - 17/Jan/2024 12:17 PM - edited

I had slowness issues when retrieving emails from the office365 IMAP server with my apache camel application.
Adding JVM parameters resolved the problem for me.
Thank you very much.

Nélio Vasconcelos added a comment - 17/Jan/2024 12:17 PM - edited I had slowness issues when retrieving emails from the office365 IMAP server with my apache camel application. Adding JVM parameters resolved the problem for me. Thank you very much.

Allan Gandelman made changes - 02/Jun/2023 9:09 PM

Remote Link

Original: This issue links to "JSDS-5460 (Bulldog)" [ 466876 ]

New: This issue links to "JSMDC-5460 (JIRA Server (Bulldog))" [ 466876 ]

Jose Aguilera added a comment - 11/Aug/2022 3:11 PM

We are experiencing this issue in version 4.13.18, so perhaps the bug was re-introduced?

Jose Aguilera added a comment - 11/Aug/2022 3:11 PM We are experiencing this issue in version 4.13.18 , so perhaps the bug was re-introduced?

Julien Rey made changes - 06/Oct/2021 10:22 AM

Description

Original: h3. Summary

Emails sent to an IMAP custom Service Desk inbox with attachments bigger than 2Mb will not create a ticket in their JSD project. The email is read in the mailbox though but there is no record of JSD ever processing it. The issue is reproducible by the customer as the email is currently sitting in their inbox as read, and has been marked as unread several times. The email gets then marked as read with no success in creating the ticket in JSD.
h3. Steps to Reproduce
# Setup custom secure IMAP mail server
# Receive an email with attachment of around 2Mb or larger
# Notice the email will be read, but no record is logged in Jira or the Attachment takes 15 mins to be processed.

NOTE: This could not be reproduced in our Atlassian test environment.
h3. Expected Results
# Email is set with attachment
# Ticket is created and a log created in Jira

h3. Actual Results

Email is marked as read but JSD ticket does not get generated and the following stack trace is found in the logs:
{code:java}
2018-10-17 08:05:21,044 ERROR [] Caesium-1-4 ServiceRunner Error while serializing message
com.sun.mail.util.FolderClosedIOException
        at com.sun.mail.imap.IMAPInputStream.forceCheckExpunged(IMAPInputStream.java:107) [javax.mail-1.6.0.jar:1.6.0]
        at com.sun.mail.imap.IMAPInputStream.fill(IMAPInputStream.java:158) [javax.mail-1.6.0.jar:1.6.0]
        at com.sun.mail.imap.IMAPInputStream.read(IMAPInputStream.java:218) [javax.mail-1.6.0.jar:1.6.0]
        at com.sun.mail.imap.IMAPInputStream.read(IMAPInputStream.java:244) [javax.mail-1.6.0.jar:1.6.0]
        at com.sun.mail.imap.IMAPMessage.writeTo(IMAPMessage.java:849) [javax.mail-1.6.0.jar:1.6.0]
        at com.atlassian.jira.internal.mail.processor.util.MimeMessageBase64ChunkStream.fullyLoadMessageToMemory(MimeMessageBase64ChunkStream.java:32) [?:?]
        at com.atlassian.jira.internal.mail.processor.util.MimeMessageBase64ChunkStream.<init>(MimeMessageBase64ChunkStream.java:22) [?:?]
        at com.atlassian.jira.internal.mail.processor.feature.puller.mailitem.MailItemStore.createMailItemChunk(MailItemStore.java:239) [?:?]
        at com.atlassian.jira.internal.mail.processor.feature.puller.mailitem.MailItemStore.lambda$null$4(MailItemStore.java:214) [?:?]
        at io.atlassian.fugue.Option.map(Option.java:259) [fugue-4.5.0.jar:?]
        at com.atlassian.jira.internal.mail.processor.feature.puller.mailitem.MailItemStore.lambda$createMailItem$5(MailItemStore.java:212) [?:?]
        at com.atlassian.pocketknife.internal.querydsl.DatabaseAccessorImpl.lambda$execute$0(DatabaseAccessorImpl.java:69) [?:?]
        at com.atlassian.sal.core.rdbms.DefaultTransactionalExecutor.executeInternal(DefaultTransactionalExecutor.java:86) [?:?]
        at com.atlassian.sal.core.rdbms.DefaultTransactionalExecutor.lambda$execute$0(DefaultTransactionalExecutor.java:42) [?:?]
        at com.atlassian.jira.database.DatabaseAccessorImpl.runInManagedTransaction(DatabaseAccessorImpl.java:125) [classes/:?]
        at sun.reflect.GeneratedMethodAccessor234.invoke(Unknown Source) [?:?]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [?:1.8.0_102]
        at java.lang.reflect.Method.invoke(Method.java:498) [?:1.8.0_102]
        at com.atlassian.plugin.util.ContextClassLoaderSettingInvocationHandler.invoke(ContextClassLoaderSettingInvocationHandler.java:26) [atlassian-plugins-core-4.5.1.jar:?]
        at com.sun.proxy.$Proxy114.runInManagedTransaction(Unknown Source) [?:?]
        at sun.reflect.GeneratedMethodAccessor234.invoke(Unknown Source) [?:?]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [?:1.8.0_102]
        at java.lang.reflect.Method.invoke(Method.java:498) [?:1.8.0_102]
        at com.atlassian.plugin.osgi.bridge.external.HostComponentFactoryBean$DynamicServiceInvocationHandler.invoke(HostComponentFactoryBean.java:136) [atlassian-plugins-osgi-bridge-4.5.1.jar:?]
        at com.sun.proxy.$Proxy114.runInManagedTransaction(Unknown Source) [?:?]
        at com.atlassian.sal.jira.rdbms.JiraHostConnectionAccessor.runInStartedOrExistingTransaction(JiraHostConnectionAccessor.java:130) [jira-sal-plugin-7.11.2_1533715698000.jar:?]
        at com.atlassian.sal.jira.rdbms.JiraHostConnectionAccessor.execute(JiraHostConnectionAccessor.java:60) [jira-sal-plugin-7.11.2_1533715698000.jar:?]
        at com.atlassian.sal.core.rdbms.DefaultTransactionalExecutor.execute(DefaultTransactionalExecutor.java:39) [jira-sal-plugin-7.11.2_1533715698000.jar:?]
        at com.atlassian.pocketknife.internal.querydsl.DatabaseAccessorImpl.execute(DatabaseAccessorImpl.java:67) [jira-email-processor-plugin-2.11.0.jar:?]
        at com.atlassian.pocketknife.internal.querydsl.DatabaseAccessorImpl.runInTransaction(DatabaseAccessorImpl.java:43) [jira-email-processor-plugin-2.11.0.jar:?]
        at com.atlassian.pocketknife.api.querydsl.DatabaseAccessor.runInTransaction(DatabaseAccessor.java:134) [jira-email-processor-plugin-2.11.0.jar:?]
        at com.atlassian.jira.internal.mail.processor.querydsl.QueryDslHelper.runInTransaction(QueryDslHelper.java:53) [jira-email-processor-plugin-2.11.0.jar:?]
        at com.atlassian.jira.internal.mail.processor.feature.puller.mailitem.MailItemStore.createMailItem(MailItemStore.java:208) [jira-email-processor-plugin-2.11.0.jar:?]
        at com.atlassian.jira.internal.mail.processor.feature.puller.mailitem.MailItemManager.createMailItemFromMessage(MailItemManager.java:38) [jira-email-processor-plugin-2.11.0.jar:?]
        at com.atlassian.jira.internal.mail.processor.feature.puller.MailPullerWorker.persistMailMessage(MailPullerWorker.java:209) [jira-email-processor-plugin-2.11.0.jar:?]
        at com.atlassian.jira.internal.mail.processor.feature.puller.MailPullerWorker.pullEmailForConnection(MailPullerWorker.java:113) [jira-email-processor-plugin-2.11.0.jar:?]
        at com.atlassian.jira.internal.mail.processor.feature.puller.MailPullerWorker.pullMailFromAllValidChannels(MailPullerWorker.java:76) [jira-email-processor-plugin-2.11.0.jar:?]
        at com.atlassian.jira.internal.mail.processor.feature.puller.MailPullerService.run(MailPullerService.java:33) [jira-email-processor-plugin-2.11.0.jar:?]
        at com.atlassian.jira.internal.mail.processor.services.MailPullerExecutor.run(MailPullerExecutor.java:29) [jira-email-processor-plugin-2.11.0.jar:?]
        at com.atlassian.jira.internal.mail.processor.services.AbstractMailExecutor.execute(AbstractMailExecutor.java:45) [jira-email-processor-plugin-2.11.0.jar:?]
        at com.atlassian.jira.internal.mail.processor.services.MailJobRunner.runJob(MailJobRunner.java:35) [jira-email-processor-plugin-2.11.0.jar:?]
        at com.atlassian.scheduler.core.JobLauncher.runJob(JobLauncher.java:153) [atlassian-scheduler-core-1.7.0.jar:?]
        at com.atlassian.scheduler.core.JobLauncher.launchAndBuildResponse(JobLauncher.java:118) [atlassian-scheduler-core-1.7.0.jar:?]
        at com.atlassian.scheduler.core.JobLauncher.launch(JobLauncher.java:97) [atlassian-scheduler-core-1.7.0.jar:?]
        at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService.launchJob(CaesiumSchedulerService.java:443) [atlassian-scheduler-caesium-1.3.3.jar:?]
        at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService.executeClusteredJob(CaesiumSchedulerService.java:438) [atlassian-scheduler-caesium-1.3.3.jar:?]
        at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService.executeClusteredJobWithRecoveryGuard(CaesiumSchedulerService.java:462) [atlassian-scheduler-caesium-1.3.3.jar:?]
        at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService.executeQueuedJob(CaesiumSchedulerService.java:390) [atlassian-scheduler-caesium-1.3.3.jar:?]
        at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService$1.consume(CaesiumSchedulerService.java:285) [atlassian-scheduler-caesium-1.3.3.jar:?]
        at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService$1.consume(CaesiumSchedulerService.java:282) [atlassian-scheduler-caesium-1.3.3.jar:?]
        at com.atlassian.scheduler.caesium.impl.SchedulerQueueWorker.executeJob(SchedulerQueueWorker.java:65) [atlassian-scheduler-caesium-1.3.3.jar:?]
        at com.atlassian.scheduler.caesium.impl.SchedulerQueueWorker.executeNextJob(SchedulerQueueWorker.java:59) [atlassian-scheduler-caesium-1.3.3.jar:?]
        at com.atlassian.scheduler.caesium.impl.SchedulerQueueWorker.run(SchedulerQueueWorker.java:34) [atlassian-scheduler-caesium-1.3.3.jar:?]
        at java.lang.Thread.run(Thread.java:745) [?:1.8.0_102]
2018-10-17 08:05:21,106 DEBUG [] Caesium-1-4 ServiceRunner The result of deletion 0
2018-10-17 08:05:21,106 DEBUG [] Caesium-1-4 ServiceRunner The result of deletion 1
{code}

This always happens for all the emails that are slow or fail to be processed.
h3. Notes
* By looking at the connectivity logs for that inbox, we can see several of [AUTHENTICATIONFAILED] Invalid credentials (Failure) errors in the last month.
  * Forward the email directly to the default Jira inbox, a ticket was created

h3. Workaround

h4. Workaround 1

There's been some evidence that moving to POP mitigates the bug

h4. Workaround 2

There's been some evidence that adding the 2 following JVM parameters and re-starting the Jira application also helps mitigate the bug:
{code}
-Dmail.imap.fetchsize=1048576
-Dmail.imaps.fetchsize=1048576
{code}

These settings will increase the Java Mail Prefetch size from 16kb to 1mb, in order to limit the number of requests sent from the Service Desk mail handler to the network/

The steps are:
- Stop the Jira application (stop all the Jira nodes if using Jira Data Center)
- Add the following JVM parameters as per [this documentation|https://confluence.atlassian.com/adminjiraserver/setting-properties-and-options-on-startup-938847831.html]
{code}
-Dmail.imap.fetchsize=1048576
-Dmail.imaps.fetchsize=1048576
{code}
- Start the Jira application (or start all the Jira nodes id using Jira Data Center)

New: h3. Summary

Emails sent to an IMAP custom Service Desk inbox with attachments bigger than 2Mb will not create a ticket in their JSD project. The email is read in the mailbox though but there is no record of JSD ever processing it. The issue is reproducible by the customer as the email is currently sitting in their inbox as read, and has been marked as unread several times. The email gets then marked as read with no success in creating the ticket in JSD.
h3. Steps to Reproduce
# Setup custom secure IMAP mail server
# Receive an email with attachment of around 2Mb or larger
# Notice the email will be read, but no record is logged in Jira or the Attachment takes 15 mins to be processed.

NOTE: This could not be reproduced in our Atlassian test environment.
h3. Expected Results
# Email is set with attachment
# Ticket is created and a log created in Jira

h3. Actual Results

Email is marked as read but JSD ticket does not get generated and the following stack trace is found in the logs:
{code:java}
2018-10-17 08:05:21,044 ERROR [] Caesium-1-4 ServiceRunner Error while serializing message
com.sun.mail.util.FolderClosedIOException
        at com.sun.mail.imap.IMAPInputStream.forceCheckExpunged(IMAPInputStream.java:107) [javax.mail-1.6.0.jar:1.6.0]
        at com.sun.mail.imap.IMAPInputStream.fill(IMAPInputStream.java:158) [javax.mail-1.6.0.jar:1.6.0]
        at com.sun.mail.imap.IMAPInputStream.read(IMAPInputStream.java:218) [javax.mail-1.6.0.jar:1.6.0]
        at com.sun.mail.imap.IMAPInputStream.read(IMAPInputStream.java:244) [javax.mail-1.6.0.jar:1.6.0]
        at com.sun.mail.imap.IMAPMessage.writeTo(IMAPMessage.java:849) [javax.mail-1.6.0.jar:1.6.0]
        at com.atlassian.jira.internal.mail.processor.util.MimeMessageBase64ChunkStream.fullyLoadMessageToMemory(MimeMessageBase64ChunkStream.java:32) [?:?]
        at com.atlassian.jira.internal.mail.processor.util.MimeMessageBase64ChunkStream.<init>(MimeMessageBase64ChunkStream.java:22) [?:?]
        at com.atlassian.jira.internal.mail.processor.feature.puller.mailitem.MailItemStore.createMailItemChunk(MailItemStore.java:239) [?:?]
        at com.atlassian.jira.internal.mail.processor.feature.puller.mailitem.MailItemStore.lambda$null$4(MailItemStore.java:214) [?:?]
        at io.atlassian.fugue.Option.map(Option.java:259) [fugue-4.5.0.jar:?]
        at com.atlassian.jira.internal.mail.processor.feature.puller.mailitem.MailItemStore.lambda$createMailItem$5(MailItemStore.java:212) [?:?]
        at com.atlassian.pocketknife.internal.querydsl.DatabaseAccessorImpl.lambda$execute$0(DatabaseAccessorImpl.java:69) [?:?]
        at com.atlassian.sal.core.rdbms.DefaultTransactionalExecutor.executeInternal(DefaultTransactionalExecutor.java:86) [?:?]
        at com.atlassian.sal.core.rdbms.DefaultTransactionalExecutor.lambda$execute$0(DefaultTransactionalExecutor.java:42) [?:?]
        at com.atlassian.jira.database.DatabaseAccessorImpl.runInManagedTransaction(DatabaseAccessorImpl.java:125) [classes/:?]
        at sun.reflect.GeneratedMethodAccessor234.invoke(Unknown Source) [?:?]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [?:1.8.0_102]
        at java.lang.reflect.Method.invoke(Method.java:498) [?:1.8.0_102]
        at com.atlassian.plugin.util.ContextClassLoaderSettingInvocationHandler.invoke(ContextClassLoaderSettingInvocationHandler.java:26) [atlassian-plugins-core-4.5.1.jar:?]
        at com.sun.proxy.$Proxy114.runInManagedTransaction(Unknown Source) [?:?]
        at sun.reflect.GeneratedMethodAccessor234.invoke(Unknown Source) [?:?]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [?:1.8.0_102]
        at java.lang.reflect.Method.invoke(Method.java:498) [?:1.8.0_102]
        at com.atlassian.plugin.osgi.bridge.external.HostComponentFactoryBean$DynamicServiceInvocationHandler.invoke(HostComponentFactoryBean.java:136) [atlassian-plugins-osgi-bridge-4.5.1.jar:?]
        at com.sun.proxy.$Proxy114.runInManagedTransaction(Unknown Source) [?:?]
        at com.atlassian.sal.jira.rdbms.JiraHostConnectionAccessor.runInStartedOrExistingTransaction(JiraHostConnectionAccessor.java:130) [jira-sal-plugin-7.11.2_1533715698000.jar:?]
        at com.atlassian.sal.jira.rdbms.JiraHostConnectionAccessor.execute(JiraHostConnectionAccessor.java:60) [jira-sal-plugin-7.11.2_1533715698000.jar:?]
        at com.atlassian.sal.core.rdbms.DefaultTransactionalExecutor.execute(DefaultTransactionalExecutor.java:39) [jira-sal-plugin-7.11.2_1533715698000.jar:?]
        at com.atlassian.pocketknife.internal.querydsl.DatabaseAccessorImpl.execute(DatabaseAccessorImpl.java:67) [jira-email-processor-plugin-2.11.0.jar:?]
        at com.atlassian.pocketknife.internal.querydsl.DatabaseAccessorImpl.runInTransaction(DatabaseAccessorImpl.java:43) [jira-email-processor-plugin-2.11.0.jar:?]
        at com.atlassian.pocketknife.api.querydsl.DatabaseAccessor.runInTransaction(DatabaseAccessor.java:134) [jira-email-processor-plugin-2.11.0.jar:?]
        at com.atlassian.jira.internal.mail.processor.querydsl.QueryDslHelper.runInTransaction(QueryDslHelper.java:53) [jira-email-processor-plugin-2.11.0.jar:?]
        at com.atlassian.jira.internal.mail.processor.feature.puller.mailitem.MailItemStore.createMailItem(MailItemStore.java:208) [jira-email-processor-plugin-2.11.0.jar:?]
        at com.atlassian.jira.internal.mail.processor.feature.puller.mailitem.MailItemManager.createMailItemFromMessage(MailItemManager.java:38) [jira-email-processor-plugin-2.11.0.jar:?]
        at com.atlassian.jira.internal.mail.processor.feature.puller.MailPullerWorker.persistMailMessage(MailPullerWorker.java:209) [jira-email-processor-plugin-2.11.0.jar:?]
        at com.atlassian.jira.internal.mail.processor.feature.puller.MailPullerWorker.pullEmailForConnection(MailPullerWorker.java:113) [jira-email-processor-plugin-2.11.0.jar:?]
        at com.atlassian.jira.internal.mail.processor.feature.puller.MailPullerWorker.pullMailFromAllValidChannels(MailPullerWorker.java:76) [jira-email-processor-plugin-2.11.0.jar:?]
        at com.atlassian.jira.internal.mail.processor.feature.puller.MailPullerService.run(MailPullerService.java:33) [jira-email-processor-plugin-2.11.0.jar:?]
        at com.atlassian.jira.internal.mail.processor.services.MailPullerExecutor.run(MailPullerExecutor.java:29) [jira-email-processor-plugin-2.11.0.jar:?]
        at com.atlassian.jira.internal.mail.processor.services.AbstractMailExecutor.execute(AbstractMailExecutor.java:45) [jira-email-processor-plugin-2.11.0.jar:?]
        at com.atlassian.jira.internal.mail.processor.services.MailJobRunner.runJob(MailJobRunner.java:35) [jira-email-processor-plugin-2.11.0.jar:?]
        at com.atlassian.scheduler.core.JobLauncher.runJob(JobLauncher.java:153) [atlassian-scheduler-core-1.7.0.jar:?]
        at com.atlassian.scheduler.core.JobLauncher.launchAndBuildResponse(JobLauncher.java:118) [atlassian-scheduler-core-1.7.0.jar:?]
        at com.atlassian.scheduler.core.JobLauncher.launch(JobLauncher.java:97) [atlassian-scheduler-core-1.7.0.jar:?]
        at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService.launchJob(CaesiumSchedulerService.java:443) [atlassian-scheduler-caesium-1.3.3.jar:?]
        at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService.executeClusteredJob(CaesiumSchedulerService.java:438) [atlassian-scheduler-caesium-1.3.3.jar:?]
        at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService.executeClusteredJobWithRecoveryGuard(CaesiumSchedulerService.java:462) [atlassian-scheduler-caesium-1.3.3.jar:?]
        at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService.executeQueuedJob(CaesiumSchedulerService.java:390) [atlassian-scheduler-caesium-1.3.3.jar:?]
        at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService$1.consume(CaesiumSchedulerService.java:285) [atlassian-scheduler-caesium-1.3.3.jar:?]
        at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService$1.consume(CaesiumSchedulerService.java:282) [atlassian-scheduler-caesium-1.3.3.jar:?]
        at com.atlassian.scheduler.caesium.impl.SchedulerQueueWorker.executeJob(SchedulerQueueWorker.java:65) [atlassian-scheduler-caesium-1.3.3.jar:?]
        at com.atlassian.scheduler.caesium.impl.SchedulerQueueWorker.executeNextJob(SchedulerQueueWorker.java:59) [atlassian-scheduler-caesium-1.3.3.jar:?]
        at com.atlassian.scheduler.caesium.impl.SchedulerQueueWorker.run(SchedulerQueueWorker.java:34) [atlassian-scheduler-caesium-1.3.3.jar:?]
        at java.lang.Thread.run(Thread.java:745) [?:1.8.0_102]
2018-10-17 08:05:21,106 DEBUG [] Caesium-1-4 ServiceRunner The result of deletion 0
2018-10-17 08:05:21,106 DEBUG [] Caesium-1-4 ServiceRunner The result of deletion 1
{code}
This always happens for all the emails that are slow or fail to be processed.
h3. Notes
* By looking at the connectivity logs for that inbox, we can see several of [AUTHENTICATIONFAILED] Invalid credentials (Failure) errors in the last month.
* Forward the email directly to the default Jira inbox, a ticket was created

h3. Workaround
h4. Workaround 1

There's been some evidence that moving to POP mitigates the bug
h4. Workaround 2

There's been some evidence that adding the 2 following JVM parameters and re-starting the Jira application also helps mitigate the bug:
{code:java}
-Dmail.imap.fetchsize=1048576
-Dmail.imaps.fetchsize=1048576
{code}
These settings will increase the Java Mail Prefetch size from 16kb to 1mb, in order to limit the number of requests sent from the Service Desk mail handler to the network/

The steps are:
- Stop the Jira application (stop all the Jira nodes if using Jira Data Center)
- Add the following JVM parameters as per [this documentation|https://confluence.atlassian.com/adminjiraserver/setting-properties-and-options-on-startup-938847831.html]
{code:java}
-Dmail.imap.fetchsize=1048576
-Dmail.imaps.fetchsize=1048576
{code}

- Backup the Jira Database
- Run the following DELETE query in the Jira Database (this step will help reset the Service Desk Mail Puller job, in case it was stuck. Note the row corresponding to this job will be automatically re-created in the rundetails table after a Jira re-start)
{code:java}
delete from rundetails where job_id = 'com.atlassian.jira.internal.mail.services.MailPullerJobRunner';
{code}

- Start the Jira application (or start all the Jira nodes id using Jira Data Center)

Julien Rey made changes - 05/Oct/2021 3:27 PM

Description

Original: h3. Summary

Emails sent to an IMAP custom Service Desk inbox with attachments bigger than 2Mb will not create a ticket in their JSD project. The email is read in the mailbox though but there is no record of JSD ever processing it. The issue is reproducible by the customer as the email is currently sitting in their inbox as read, and has been marked as unread several times. The email gets then marked as read with no success in creating the ticket in JSD.
h3. Steps to Reproduce
# Setup custom secure IMAP mail server
# Receive an email with attachment of around 2Mb or larger
# Notice the email will be read, but no record is logged in Jira or the Attachment takes 15 mins to be processed.

NOTE: This could not be reproduced in our Atlassian test environment.
h3. Expected Results
# Email is set with attachment
# Ticket is created and a log created in Jira

h3. Actual Results

Email is marked as read but JSD ticket does not get generated and the following stack trace is found in the logs:
{code:java}
2018-10-17 08:05:21,044 ERROR [] Caesium-1-4 ServiceRunner Error while serializing message
com.sun.mail.util.FolderClosedIOException
        at com.sun.mail.imap.IMAPInputStream.forceCheckExpunged(IMAPInputStream.java:107) [javax.mail-1.6.0.jar:1.6.0]
        at com.sun.mail.imap.IMAPInputStream.fill(IMAPInputStream.java:158) [javax.mail-1.6.0.jar:1.6.0]
        at com.sun.mail.imap.IMAPInputStream.read(IMAPInputStream.java:218) [javax.mail-1.6.0.jar:1.6.0]
        at com.sun.mail.imap.IMAPInputStream.read(IMAPInputStream.java:244) [javax.mail-1.6.0.jar:1.6.0]
        at com.sun.mail.imap.IMAPMessage.writeTo(IMAPMessage.java:849) [javax.mail-1.6.0.jar:1.6.0]
        at com.atlassian.jira.internal.mail.processor.util.MimeMessageBase64ChunkStream.fullyLoadMessageToMemory(MimeMessageBase64ChunkStream.java:32) [?:?]
        at com.atlassian.jira.internal.mail.processor.util.MimeMessageBase64ChunkStream.<init>(MimeMessageBase64ChunkStream.java:22) [?:?]
        at com.atlassian.jira.internal.mail.processor.feature.puller.mailitem.MailItemStore.createMailItemChunk(MailItemStore.java:239) [?:?]
        at com.atlassian.jira.internal.mail.processor.feature.puller.mailitem.MailItemStore.lambda$null$4(MailItemStore.java:214) [?:?]
        at io.atlassian.fugue.Option.map(Option.java:259) [fugue-4.5.0.jar:?]
        at com.atlassian.jira.internal.mail.processor.feature.puller.mailitem.MailItemStore.lambda$createMailItem$5(MailItemStore.java:212) [?:?]
        at com.atlassian.pocketknife.internal.querydsl.DatabaseAccessorImpl.lambda$execute$0(DatabaseAccessorImpl.java:69) [?:?]
        at com.atlassian.sal.core.rdbms.DefaultTransactionalExecutor.executeInternal(DefaultTransactionalExecutor.java:86) [?:?]
        at com.atlassian.sal.core.rdbms.DefaultTransactionalExecutor.lambda$execute$0(DefaultTransactionalExecutor.java:42) [?:?]
        at com.atlassian.jira.database.DatabaseAccessorImpl.runInManagedTransaction(DatabaseAccessorImpl.java:125) [classes/:?]
        at sun.reflect.GeneratedMethodAccessor234.invoke(Unknown Source) [?:?]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [?:1.8.0_102]
        at java.lang.reflect.Method.invoke(Method.java:498) [?:1.8.0_102]
        at com.atlassian.plugin.util.ContextClassLoaderSettingInvocationHandler.invoke(ContextClassLoaderSettingInvocationHandler.java:26) [atlassian-plugins-core-4.5.1.jar:?]
        at com.sun.proxy.$Proxy114.runInManagedTransaction(Unknown Source) [?:?]
        at sun.reflect.GeneratedMethodAccessor234.invoke(Unknown Source) [?:?]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [?:1.8.0_102]
        at java.lang.reflect.Method.invoke(Method.java:498) [?:1.8.0_102]
        at com.atlassian.plugin.osgi.bridge.external.HostComponentFactoryBean$DynamicServiceInvocationHandler.invoke(HostComponentFactoryBean.java:136) [atlassian-plugins-osgi-bridge-4.5.1.jar:?]
        at com.sun.proxy.$Proxy114.runInManagedTransaction(Unknown Source) [?:?]
        at com.atlassian.sal.jira.rdbms.JiraHostConnectionAccessor.runInStartedOrExistingTransaction(JiraHostConnectionAccessor.java:130) [jira-sal-plugin-7.11.2_1533715698000.jar:?]
        at com.atlassian.sal.jira.rdbms.JiraHostConnectionAccessor.execute(JiraHostConnectionAccessor.java:60) [jira-sal-plugin-7.11.2_1533715698000.jar:?]
        at com.atlassian.sal.core.rdbms.DefaultTransactionalExecutor.execute(DefaultTransactionalExecutor.java:39) [jira-sal-plugin-7.11.2_1533715698000.jar:?]
        at com.atlassian.pocketknife.internal.querydsl.DatabaseAccessorImpl.execute(DatabaseAccessorImpl.java:67) [jira-email-processor-plugin-2.11.0.jar:?]
        at com.atlassian.pocketknife.internal.querydsl.DatabaseAccessorImpl.runInTransaction(DatabaseAccessorImpl.java:43) [jira-email-processor-plugin-2.11.0.jar:?]
        at com.atlassian.pocketknife.api.querydsl.DatabaseAccessor.runInTransaction(DatabaseAccessor.java:134) [jira-email-processor-plugin-2.11.0.jar:?]
        at com.atlassian.jira.internal.mail.processor.querydsl.QueryDslHelper.runInTransaction(QueryDslHelper.java:53) [jira-email-processor-plugin-2.11.0.jar:?]
        at com.atlassian.jira.internal.mail.processor.feature.puller.mailitem.MailItemStore.createMailItem(MailItemStore.java:208) [jira-email-processor-plugin-2.11.0.jar:?]
        at com.atlassian.jira.internal.mail.processor.feature.puller.mailitem.MailItemManager.createMailItemFromMessage(MailItemManager.java:38) [jira-email-processor-plugin-2.11.0.jar:?]
        at com.atlassian.jira.internal.mail.processor.feature.puller.MailPullerWorker.persistMailMessage(MailPullerWorker.java:209) [jira-email-processor-plugin-2.11.0.jar:?]
        at com.atlassian.jira.internal.mail.processor.feature.puller.MailPullerWorker.pullEmailForConnection(MailPullerWorker.java:113) [jira-email-processor-plugin-2.11.0.jar:?]
        at com.atlassian.jira.internal.mail.processor.feature.puller.MailPullerWorker.pullMailFromAllValidChannels(MailPullerWorker.java:76) [jira-email-processor-plugin-2.11.0.jar:?]
        at com.atlassian.jira.internal.mail.processor.feature.puller.MailPullerService.run(MailPullerService.java:33) [jira-email-processor-plugin-2.11.0.jar:?]
        at com.atlassian.jira.internal.mail.processor.services.MailPullerExecutor.run(MailPullerExecutor.java:29) [jira-email-processor-plugin-2.11.0.jar:?]
        at com.atlassian.jira.internal.mail.processor.services.AbstractMailExecutor.execute(AbstractMailExecutor.java:45) [jira-email-processor-plugin-2.11.0.jar:?]
        at com.atlassian.jira.internal.mail.processor.services.MailJobRunner.runJob(MailJobRunner.java:35) [jira-email-processor-plugin-2.11.0.jar:?]
        at com.atlassian.scheduler.core.JobLauncher.runJob(JobLauncher.java:153) [atlassian-scheduler-core-1.7.0.jar:?]
        at com.atlassian.scheduler.core.JobLauncher.launchAndBuildResponse(JobLauncher.java:118) [atlassian-scheduler-core-1.7.0.jar:?]
        at com.atlassian.scheduler.core.JobLauncher.launch(JobLauncher.java:97) [atlassian-scheduler-core-1.7.0.jar:?]
        at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService.launchJob(CaesiumSchedulerService.java:443) [atlassian-scheduler-caesium-1.3.3.jar:?]
        at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService.executeClusteredJob(CaesiumSchedulerService.java:438) [atlassian-scheduler-caesium-1.3.3.jar:?]
        at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService.executeClusteredJobWithRecoveryGuard(CaesiumSchedulerService.java:462) [atlassian-scheduler-caesium-1.3.3.jar:?]
        at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService.executeQueuedJob(CaesiumSchedulerService.java:390) [atlassian-scheduler-caesium-1.3.3.jar:?]
        at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService$1.consume(CaesiumSchedulerService.java:285) [atlassian-scheduler-caesium-1.3.3.jar:?]
        at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService$1.consume(CaesiumSchedulerService.java:282) [atlassian-scheduler-caesium-1.3.3.jar:?]
        at com.atlassian.scheduler.caesium.impl.SchedulerQueueWorker.executeJob(SchedulerQueueWorker.java:65) [atlassian-scheduler-caesium-1.3.3.jar:?]
        at com.atlassian.scheduler.caesium.impl.SchedulerQueueWorker.executeNextJob(SchedulerQueueWorker.java:59) [atlassian-scheduler-caesium-1.3.3.jar:?]
        at com.atlassian.scheduler.caesium.impl.SchedulerQueueWorker.run(SchedulerQueueWorker.java:34) [atlassian-scheduler-caesium-1.3.3.jar:?]
        at java.lang.Thread.run(Thread.java:745) [?:1.8.0_102]
2018-10-17 08:05:21,106 DEBUG [] Caesium-1-4 ServiceRunner The result of deletion 0
2018-10-17 08:05:21,106 DEBUG [] Caesium-1-4 ServiceRunner The result of deletion 1
{code}

This always happens for all the emails that are slow or fail to be processed.
h3. Notes
* By looking at the connectivity logs for that inbox, we can see several of [AUTHENTICATIONFAILED] Invalid credentials (Failure) errors in the last month.
  * Forward the email directly to the default Jira inbox, a ticket was created

h3. Workaround

There's been some evidence that moving to POP mitigates the bug

New: h3. Summary

Emails sent to an IMAP custom Service Desk inbox with attachments bigger than 2Mb will not create a ticket in their JSD project. The email is read in the mailbox though but there is no record of JSD ever processing it. The issue is reproducible by the customer as the email is currently sitting in their inbox as read, and has been marked as unread several times. The email gets then marked as read with no success in creating the ticket in JSD.
h3. Steps to Reproduce
# Setup custom secure IMAP mail server
# Receive an email with attachment of around 2Mb or larger
# Notice the email will be read, but no record is logged in Jira or the Attachment takes 15 mins to be processed.

NOTE: This could not be reproduced in our Atlassian test environment.
h3. Expected Results
# Email is set with attachment
# Ticket is created and a log created in Jira

h3. Actual Results

Email is marked as read but JSD ticket does not get generated and the following stack trace is found in the logs:
{code:java}
2018-10-17 08:05:21,044 ERROR [] Caesium-1-4 ServiceRunner Error while serializing message
com.sun.mail.util.FolderClosedIOException
        at com.sun.mail.imap.IMAPInputStream.forceCheckExpunged(IMAPInputStream.java:107) [javax.mail-1.6.0.jar:1.6.0]
        at com.sun.mail.imap.IMAPInputStream.fill(IMAPInputStream.java:158) [javax.mail-1.6.0.jar:1.6.0]
        at com.sun.mail.imap.IMAPInputStream.read(IMAPInputStream.java:218) [javax.mail-1.6.0.jar:1.6.0]
        at com.sun.mail.imap.IMAPInputStream.read(IMAPInputStream.java:244) [javax.mail-1.6.0.jar:1.6.0]
        at com.sun.mail.imap.IMAPMessage.writeTo(IMAPMessage.java:849) [javax.mail-1.6.0.jar:1.6.0]
        at com.atlassian.jira.internal.mail.processor.util.MimeMessageBase64ChunkStream.fullyLoadMessageToMemory(MimeMessageBase64ChunkStream.java:32) [?:?]
        at com.atlassian.jira.internal.mail.processor.util.MimeMessageBase64ChunkStream.<init>(MimeMessageBase64ChunkStream.java:22) [?:?]
        at com.atlassian.jira.internal.mail.processor.feature.puller.mailitem.MailItemStore.createMailItemChunk(MailItemStore.java:239) [?:?]
        at com.atlassian.jira.internal.mail.processor.feature.puller.mailitem.MailItemStore.lambda$null$4(MailItemStore.java:214) [?:?]
        at io.atlassian.fugue.Option.map(Option.java:259) [fugue-4.5.0.jar:?]
        at com.atlassian.jira.internal.mail.processor.feature.puller.mailitem.MailItemStore.lambda$createMailItem$5(MailItemStore.java:212) [?:?]
        at com.atlassian.pocketknife.internal.querydsl.DatabaseAccessorImpl.lambda$execute$0(DatabaseAccessorImpl.java:69) [?:?]
        at com.atlassian.sal.core.rdbms.DefaultTransactionalExecutor.executeInternal(DefaultTransactionalExecutor.java:86) [?:?]
        at com.atlassian.sal.core.rdbms.DefaultTransactionalExecutor.lambda$execute$0(DefaultTransactionalExecutor.java:42) [?:?]
        at com.atlassian.jira.database.DatabaseAccessorImpl.runInManagedTransaction(DatabaseAccessorImpl.java:125) [classes/:?]
        at sun.reflect.GeneratedMethodAccessor234.invoke(Unknown Source) [?:?]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [?:1.8.0_102]
        at java.lang.reflect.Method.invoke(Method.java:498) [?:1.8.0_102]
        at com.atlassian.plugin.util.ContextClassLoaderSettingInvocationHandler.invoke(ContextClassLoaderSettingInvocationHandler.java:26) [atlassian-plugins-core-4.5.1.jar:?]
        at com.sun.proxy.$Proxy114.runInManagedTransaction(Unknown Source) [?:?]
        at sun.reflect.GeneratedMethodAccessor234.invoke(Unknown Source) [?:?]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [?:1.8.0_102]
        at java.lang.reflect.Method.invoke(Method.java:498) [?:1.8.0_102]
        at com.atlassian.plugin.osgi.bridge.external.HostComponentFactoryBean$DynamicServiceInvocationHandler.invoke(HostComponentFactoryBean.java:136) [atlassian-plugins-osgi-bridge-4.5.1.jar:?]
        at com.sun.proxy.$Proxy114.runInManagedTransaction(Unknown Source) [?:?]
        at com.atlassian.sal.jira.rdbms.JiraHostConnectionAccessor.runInStartedOrExistingTransaction(JiraHostConnectionAccessor.java:130) [jira-sal-plugin-7.11.2_1533715698000.jar:?]
        at com.atlassian.sal.jira.rdbms.JiraHostConnectionAccessor.execute(JiraHostConnectionAccessor.java:60) [jira-sal-plugin-7.11.2_1533715698000.jar:?]
        at com.atlassian.sal.core.rdbms.DefaultTransactionalExecutor.execute(DefaultTransactionalExecutor.java:39) [jira-sal-plugin-7.11.2_1533715698000.jar:?]
        at com.atlassian.pocketknife.internal.querydsl.DatabaseAccessorImpl.execute(DatabaseAccessorImpl.java:67) [jira-email-processor-plugin-2.11.0.jar:?]
        at com.atlassian.pocketknife.internal.querydsl.DatabaseAccessorImpl.runInTransaction(DatabaseAccessorImpl.java:43) [jira-email-processor-plugin-2.11.0.jar:?]
        at com.atlassian.pocketknife.api.querydsl.DatabaseAccessor.runInTransaction(DatabaseAccessor.java:134) [jira-email-processor-plugin-2.11.0.jar:?]
        at com.atlassian.jira.internal.mail.processor.querydsl.QueryDslHelper.runInTransaction(QueryDslHelper.java:53) [jira-email-processor-plugin-2.11.0.jar:?]
        at com.atlassian.jira.internal.mail.processor.feature.puller.mailitem.MailItemStore.createMailItem(MailItemStore.java:208) [jira-email-processor-plugin-2.11.0.jar:?]
        at com.atlassian.jira.internal.mail.processor.feature.puller.mailitem.MailItemManager.createMailItemFromMessage(MailItemManager.java:38) [jira-email-processor-plugin-2.11.0.jar:?]
        at com.atlassian.jira.internal.mail.processor.feature.puller.MailPullerWorker.persistMailMessage(MailPullerWorker.java:209) [jira-email-processor-plugin-2.11.0.jar:?]
        at com.atlassian.jira.internal.mail.processor.feature.puller.MailPullerWorker.pullEmailForConnection(MailPullerWorker.java:113) [jira-email-processor-plugin-2.11.0.jar:?]
        at com.atlassian.jira.internal.mail.processor.feature.puller.MailPullerWorker.pullMailFromAllValidChannels(MailPullerWorker.java:76) [jira-email-processor-plugin-2.11.0.jar:?]
        at com.atlassian.jira.internal.mail.processor.feature.puller.MailPullerService.run(MailPullerService.java:33) [jira-email-processor-plugin-2.11.0.jar:?]
        at com.atlassian.jira.internal.mail.processor.services.MailPullerExecutor.run(MailPullerExecutor.java:29) [jira-email-processor-plugin-2.11.0.jar:?]
        at com.atlassian.jira.internal.mail.processor.services.AbstractMailExecutor.execute(AbstractMailExecutor.java:45) [jira-email-processor-plugin-2.11.0.jar:?]
        at com.atlassian.jira.internal.mail.processor.services.MailJobRunner.runJob(MailJobRunner.java:35) [jira-email-processor-plugin-2.11.0.jar:?]
        at com.atlassian.scheduler.core.JobLauncher.runJob(JobLauncher.java:153) [atlassian-scheduler-core-1.7.0.jar:?]
        at com.atlassian.scheduler.core.JobLauncher.launchAndBuildResponse(JobLauncher.java:118) [atlassian-scheduler-core-1.7.0.jar:?]
        at com.atlassian.scheduler.core.JobLauncher.launch(JobLauncher.java:97) [atlassian-scheduler-core-1.7.0.jar:?]
        at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService.launchJob(CaesiumSchedulerService.java:443) [atlassian-scheduler-caesium-1.3.3.jar:?]
        at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService.executeClusteredJob(CaesiumSchedulerService.java:438) [atlassian-scheduler-caesium-1.3.3.jar:?]
        at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService.executeClusteredJobWithRecoveryGuard(CaesiumSchedulerService.java:462) [atlassian-scheduler-caesium-1.3.3.jar:?]
        at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService.executeQueuedJob(CaesiumSchedulerService.java:390) [atlassian-scheduler-caesium-1.3.3.jar:?]
        at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService$1.consume(CaesiumSchedulerService.java:285) [atlassian-scheduler-caesium-1.3.3.jar:?]
        at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService$1.consume(CaesiumSchedulerService.java:282) [atlassian-scheduler-caesium-1.3.3.jar:?]
        at com.atlassian.scheduler.caesium.impl.SchedulerQueueWorker.executeJob(SchedulerQueueWorker.java:65) [atlassian-scheduler-caesium-1.3.3.jar:?]
        at com.atlassian.scheduler.caesium.impl.SchedulerQueueWorker.executeNextJob(SchedulerQueueWorker.java:59) [atlassian-scheduler-caesium-1.3.3.jar:?]
        at com.atlassian.scheduler.caesium.impl.SchedulerQueueWorker.run(SchedulerQueueWorker.java:34) [atlassian-scheduler-caesium-1.3.3.jar:?]
        at java.lang.Thread.run(Thread.java:745) [?:1.8.0_102]
2018-10-17 08:05:21,106 DEBUG [] Caesium-1-4 ServiceRunner The result of deletion 0
2018-10-17 08:05:21,106 DEBUG [] Caesium-1-4 ServiceRunner The result of deletion 1
{code}

This always happens for all the emails that are slow or fail to be processed.
h3. Notes
* By looking at the connectivity logs for that inbox, we can see several of [AUTHENTICATIONFAILED] Invalid credentials (Failure) errors in the last month.
  * Forward the email directly to the default Jira inbox, a ticket was created

h3. Workaround

h4. Workaround 1

There's been some evidence that moving to POP mitigates the bug

h4. Workaround 2

There's been some evidence that adding the 2 following JVM parameters and re-starting the Jira application also helps mitigate the bug:
{code}
-Dmail.imap.fetchsize=1048576
-Dmail.imaps.fetchsize=1048576
{code}

These settings will increase the Java Mail Prefetch size from 16kb to 1mb, in order to limit the number of requests sent from the Service Desk mail handler to the network/

The steps are:
- Stop the Jira application (stop all the Jira nodes if using Jira Data Center)
- Add the following JVM parameters as per [this documentation|https://confluence.atlassian.com/adminjiraserver/setting-properties-and-options-on-startup-938847831.html]
{code}
-Dmail.imap.fetchsize=1048576
-Dmail.imaps.fetchsize=1048576
{code}
- Start the Jira application (or start all the Jira nodes id using Jira Data Center)

Julien Rey made changes - 09/Mar/2021 5:54 PM

Remote Link

Original: This issue links to "Page (Confluence)" [ 534650 ]

Julien Rey made changes - 09/Mar/2021 3:13 PM

Remote Link

New: This issue links to "Page (Confluence)" [ 534650 ]

Julien Rey made changes - 22/Feb/2021 4:11 PM

Remote Link

Original: This issue links to "Page (Confluence)" [ 415464 ]

Julien Rey made changes - 22/Feb/2021 4:07 PM

Remote Link

New: This issue links to "Page (Confluence)" [ 532058 ]

Assignee:: Unassigned

Reporter:: Niro (Inactive)

Affected customers:: 12 This affects my team

Watchers:: 22 Start watching this issue

Created:: 24/Oct/2018 8:11 AM

Updated:: 26/Sep/2024 5:11 PM

Resolved:: 20/May/2020 8:52 AM

Details

Description

Summary

Steps to Reproduce

Expected Results

Actual Results

Notes

Workaround

Workaround 1

Workaround 2

Attachments

Issue Links

Forms

Activity

Collapse comment: Jeramy added a comment - 26/Sep/2024 4:53 PM, Edited by Jeramy - 26/Sep/2024 5:11 PM

Expand comment: Jeramy added a comment - 26/Sep/2024 4:53 PM, Edited by Jeramy - 26/Sep/2024 5:11 PM

Collapse comment: Nélio Vasconcelos added a comment - 17/Jan/2024 12:17 PM, Edited by Nélio Vasconcelos - 17/Jan/2024 12:20 PM

Expand comment: Nélio Vasconcelos added a comment - 17/Jan/2024 12:17 PM, Edited by Nélio Vasconcelos - 17/Jan/2024 12:20 PM

Collapse comment: Jose Aguilera added a comment - 11/Aug/2022 3:11 PM

Expand comment: Jose Aguilera added a comment - 11/Aug/2022 3:11 PM

People

Dates

Backbone Issue Sync