Details
-
Bug
-
Resolution: Fixed
-
Highest
-
3.14.2
-
None
-
20
-
Severity 1 - Critical
-
42
-
Description
Summary
Emails sent to an IMAP custom Service Desk inbox with attachments bigger than 2Mb will not create a ticket in their JSD project. The email is read in the mailbox though but there is no record of JSD ever processing it. The issue is reproducible by the customer as the email is currently sitting in their inbox as read, and has been marked as unread several times. The email gets then marked as read with no success in creating the ticket in JSD.
Steps to Reproduce
- Setup custom secure IMAP mail server
- Receive an email with attachment of around 2Mb or larger
- Notice the email will be read, but no record is logged in Jira or the Attachment takes 15 mins to be processed.
NOTE: This could not be reproduced in our Atlassian test environment.
Expected Results
- Email is set with attachment
- Ticket is created and a log created in Jira
Actual Results
Email is marked as read but JSD ticket does not get generated and the following stack trace is found in the logs:
2018-10-17 08:05:21,044 ERROR [] Caesium-1-4 ServiceRunner Error while serializing message com.sun.mail.util.FolderClosedIOException at com.sun.mail.imap.IMAPInputStream.forceCheckExpunged(IMAPInputStream.java:107) [javax.mail-1.6.0.jar:1.6.0] at com.sun.mail.imap.IMAPInputStream.fill(IMAPInputStream.java:158) [javax.mail-1.6.0.jar:1.6.0] at com.sun.mail.imap.IMAPInputStream.read(IMAPInputStream.java:218) [javax.mail-1.6.0.jar:1.6.0] at com.sun.mail.imap.IMAPInputStream.read(IMAPInputStream.java:244) [javax.mail-1.6.0.jar:1.6.0] at com.sun.mail.imap.IMAPMessage.writeTo(IMAPMessage.java:849) [javax.mail-1.6.0.jar:1.6.0] at com.atlassian.jira.internal.mail.processor.util.MimeMessageBase64ChunkStream.fullyLoadMessageToMemory(MimeMessageBase64ChunkStream.java:32) [?:?] at com.atlassian.jira.internal.mail.processor.util.MimeMessageBase64ChunkStream.<init>(MimeMessageBase64ChunkStream.java:22) [?:?] at com.atlassian.jira.internal.mail.processor.feature.puller.mailitem.MailItemStore.createMailItemChunk(MailItemStore.java:239) [?:?] at com.atlassian.jira.internal.mail.processor.feature.puller.mailitem.MailItemStore.lambda$null$4(MailItemStore.java:214) [?:?] at io.atlassian.fugue.Option.map(Option.java:259) [fugue-4.5.0.jar:?] at com.atlassian.jira.internal.mail.processor.feature.puller.mailitem.MailItemStore.lambda$createMailItem$5(MailItemStore.java:212) [?:?] at com.atlassian.pocketknife.internal.querydsl.DatabaseAccessorImpl.lambda$execute$0(DatabaseAccessorImpl.java:69) [?:?] at com.atlassian.sal.core.rdbms.DefaultTransactionalExecutor.executeInternal(DefaultTransactionalExecutor.java:86) [?:?] at com.atlassian.sal.core.rdbms.DefaultTransactionalExecutor.lambda$execute$0(DefaultTransactionalExecutor.java:42) [?:?] at com.atlassian.jira.database.DatabaseAccessorImpl.runInManagedTransaction(DatabaseAccessorImpl.java:125) [classes/:?] at sun.reflect.GeneratedMethodAccessor234.invoke(Unknown Source) [?:?] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [?:1.8.0_102] at java.lang.reflect.Method.invoke(Method.java:498) [?:1.8.0_102] at com.atlassian.plugin.util.ContextClassLoaderSettingInvocationHandler.invoke(ContextClassLoaderSettingInvocationHandler.java:26) [atlassian-plugins-core-4.5.1.jar:?] at com.sun.proxy.$Proxy114.runInManagedTransaction(Unknown Source) [?:?] at sun.reflect.GeneratedMethodAccessor234.invoke(Unknown Source) [?:?] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [?:1.8.0_102] at java.lang.reflect.Method.invoke(Method.java:498) [?:1.8.0_102] at com.atlassian.plugin.osgi.bridge.external.HostComponentFactoryBean$DynamicServiceInvocationHandler.invoke(HostComponentFactoryBean.java:136) [atlassian-plugins-osgi-bridge-4.5.1.jar:?] at com.sun.proxy.$Proxy114.runInManagedTransaction(Unknown Source) [?:?] at com.atlassian.sal.jira.rdbms.JiraHostConnectionAccessor.runInStartedOrExistingTransaction(JiraHostConnectionAccessor.java:130) [jira-sal-plugin-7.11.2_1533715698000.jar:?] at com.atlassian.sal.jira.rdbms.JiraHostConnectionAccessor.execute(JiraHostConnectionAccessor.java:60) [jira-sal-plugin-7.11.2_1533715698000.jar:?] at com.atlassian.sal.core.rdbms.DefaultTransactionalExecutor.execute(DefaultTransactionalExecutor.java:39) [jira-sal-plugin-7.11.2_1533715698000.jar:?] at com.atlassian.pocketknife.internal.querydsl.DatabaseAccessorImpl.execute(DatabaseAccessorImpl.java:67) [jira-email-processor-plugin-2.11.0.jar:?] at com.atlassian.pocketknife.internal.querydsl.DatabaseAccessorImpl.runInTransaction(DatabaseAccessorImpl.java:43) [jira-email-processor-plugin-2.11.0.jar:?] at com.atlassian.pocketknife.api.querydsl.DatabaseAccessor.runInTransaction(DatabaseAccessor.java:134) [jira-email-processor-plugin-2.11.0.jar:?] at com.atlassian.jira.internal.mail.processor.querydsl.QueryDslHelper.runInTransaction(QueryDslHelper.java:53) [jira-email-processor-plugin-2.11.0.jar:?] at com.atlassian.jira.internal.mail.processor.feature.puller.mailitem.MailItemStore.createMailItem(MailItemStore.java:208) [jira-email-processor-plugin-2.11.0.jar:?] at com.atlassian.jira.internal.mail.processor.feature.puller.mailitem.MailItemManager.createMailItemFromMessage(MailItemManager.java:38) [jira-email-processor-plugin-2.11.0.jar:?] at com.atlassian.jira.internal.mail.processor.feature.puller.MailPullerWorker.persistMailMessage(MailPullerWorker.java:209) [jira-email-processor-plugin-2.11.0.jar:?] at com.atlassian.jira.internal.mail.processor.feature.puller.MailPullerWorker.pullEmailForConnection(MailPullerWorker.java:113) [jira-email-processor-plugin-2.11.0.jar:?] at com.atlassian.jira.internal.mail.processor.feature.puller.MailPullerWorker.pullMailFromAllValidChannels(MailPullerWorker.java:76) [jira-email-processor-plugin-2.11.0.jar:?] at com.atlassian.jira.internal.mail.processor.feature.puller.MailPullerService.run(MailPullerService.java:33) [jira-email-processor-plugin-2.11.0.jar:?] at com.atlassian.jira.internal.mail.processor.services.MailPullerExecutor.run(MailPullerExecutor.java:29) [jira-email-processor-plugin-2.11.0.jar:?] at com.atlassian.jira.internal.mail.processor.services.AbstractMailExecutor.execute(AbstractMailExecutor.java:45) [jira-email-processor-plugin-2.11.0.jar:?] at com.atlassian.jira.internal.mail.processor.services.MailJobRunner.runJob(MailJobRunner.java:35) [jira-email-processor-plugin-2.11.0.jar:?] at com.atlassian.scheduler.core.JobLauncher.runJob(JobLauncher.java:153) [atlassian-scheduler-core-1.7.0.jar:?] at com.atlassian.scheduler.core.JobLauncher.launchAndBuildResponse(JobLauncher.java:118) [atlassian-scheduler-core-1.7.0.jar:?] at com.atlassian.scheduler.core.JobLauncher.launch(JobLauncher.java:97) [atlassian-scheduler-core-1.7.0.jar:?] at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService.launchJob(CaesiumSchedulerService.java:443) [atlassian-scheduler-caesium-1.3.3.jar:?] at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService.executeClusteredJob(CaesiumSchedulerService.java:438) [atlassian-scheduler-caesium-1.3.3.jar:?] at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService.executeClusteredJobWithRecoveryGuard(CaesiumSchedulerService.java:462) [atlassian-scheduler-caesium-1.3.3.jar:?] at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService.executeQueuedJob(CaesiumSchedulerService.java:390) [atlassian-scheduler-caesium-1.3.3.jar:?] at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService$1.consume(CaesiumSchedulerService.java:285) [atlassian-scheduler-caesium-1.3.3.jar:?] at com.atlassian.scheduler.caesium.impl.CaesiumSchedulerService$1.consume(CaesiumSchedulerService.java:282) [atlassian-scheduler-caesium-1.3.3.jar:?] at com.atlassian.scheduler.caesium.impl.SchedulerQueueWorker.executeJob(SchedulerQueueWorker.java:65) [atlassian-scheduler-caesium-1.3.3.jar:?] at com.atlassian.scheduler.caesium.impl.SchedulerQueueWorker.executeNextJob(SchedulerQueueWorker.java:59) [atlassian-scheduler-caesium-1.3.3.jar:?] at com.atlassian.scheduler.caesium.impl.SchedulerQueueWorker.run(SchedulerQueueWorker.java:34) [atlassian-scheduler-caesium-1.3.3.jar:?] at java.lang.Thread.run(Thread.java:745) [?:1.8.0_102] 2018-10-17 08:05:21,106 DEBUG [] Caesium-1-4 ServiceRunner The result of deletion 0 2018-10-17 08:05:21,106 DEBUG [] Caesium-1-4 ServiceRunner The result of deletion 1
This always happens for all the emails that are slow or fail to be processed.
Notes
- By looking at the connectivity logs for that inbox, we can see several of [AUTHENTICATIONFAILED] Invalid credentials (Failure) errors in the last month.
- Forward the email directly to the default Jira inbox, a ticket was created
Workaround
Workaround 1
There's been some evidence that moving to POP mitigates the bug
Workaround 2
There's been some evidence that adding the 2 following JVM parameters and re-starting the Jira application also helps mitigate the bug:
-Dmail.imap.fetchsize=1048576 -Dmail.imaps.fetchsize=1048576
These settings will increase the Java Mail Prefetch size from 16kb to 1mb, in order to limit the number of requests sent from the Service Desk mail handler to the network/
The steps are:
- Stop the Jira application (stop all the Jira nodes if using Jira Data Center)
- Add the following JVM parameters as per this documentation
-Dmail.imap.fetchsize=1048576 -Dmail.imaps.fetchsize=1048576
- Backup the Jira Database
- Run the following DELETE query in the Jira Database (this step will help reset the Service Desk Mail Puller job, in case it was stuck. Note the row corresponding to this job will be automatically re-created in the rundetails table after a Jira re-start)
delete from rundetails where job_id = 'com.atlassian.jira.internal.mail.services.MailPullerJobRunner';
- Start the Jira application (or start all the Jira nodes id using Jira Data Center)
Attachments
Issue Links
- is cloned from
-
-
- Closed
-
- is cloned by
-
JSMDC-5460 Loading...