Uploaded image for project: 'Jira Data Center'
  1. Jira Data Center
  2. JRASERVER-42044

Race condition when multiple mail handlers access the same mail box

XMLWordPrintable

      NOTE: This bug report is for JIRA Server. Using JIRA Cloud? See the corresponding bug report.

      Order of events:

      1. mail handler 1 starts processing a message with attachments; log: 
        Issue XX-1234 created
      2. mail handler 2 starts processing the same message and MailFetcherService's getAssociatedIssueKey detects existing issue with the Message-ID; log: 
        Deleting message 'foo' without processing in order to avoid creating duplicate issues/comments. This message has already been partially processed, associated issue key: XX-1234
      3. mail handler 1 tries to process attachment; log: 
        Exception while adding attachments to XX-1234. Some attachments from the message might be missing. Making a note on the issue.
java.io.IOException
        at javax.mail.internet.MimePartDataSource.getInputStream(MimePartDataSource.java:113)
        at javax.activation.DataSourceDataContentHandler.getContent(DataHandler.java:797)
        at javax.activation.DataHandler.getContent(DataHandler.java:542)
        at javax.mail.internet.MimeBodyPart.getContent(MimeBodyPart.java:637)
        at com.atlassian.mail.MailUtils.isContentEmpty(MailUtils.java:638)
        at com.atlassian.jira.plugins.mail.handlers.AbstractMessageHandler.attachAttachmentsParts(AbstractMessageHandler.java:912)
        at com.atlassian.jira.plugins.mail.handlers.AbstractMessageHandler.shouldAttach(AbstractMessageHandler.java:801)
        at com.atlassian.jira.plugins.mail.handlers.AbstractMessageHandler$1.handlePart(AbstractMessageHandler.java:603)
        at com.atlassian.jira.plugins.mail.handlers.AbstractMessageHandler.handleMultipart(AbstractMessageHandler.java:700)
        at com.atlassian.jira.plugins.mail.handlers.AbstractMessageHandler.createAttachmentsForMessage(AbstractMessageHandler.java:661)
        at com.atlassian.jira.plugins.mail.handlers.CreateIssueHandler.addCcWatchersAndAttachments(CreateIssueHandler.java:300)
        at com.atlassian.jira.plugins.mail.handlers.CreateIssueHandler.handleMessage(CreateIssueHandler.java:276)
        at com.metainf.jira.plugin.emailissue.handler.CreateAndLinkHandler.handleMessage(CreateAndLinkHandler.java:48)
        at com.metainf.jira.plugin.emailissue.handler.AbstractMailHandler.handleMessage(AbstractMailHandler.java:96)
        at com.metainf.jira.plugin.emailissue.handler.EmailThisIssueMailHandler.handleMessage(EmailThisIssueMailHandler.java:419)
        at com.atlassian.jira.service.services.mail.MailFetcherService$1.process(MailFetcherService.java:438)
        at com.atlassian.jira.service.services.mail.MailFetcherService$MessageProviderImpl.getAndProcessMail(MailFetcherService.java:304)
        at com.atlassian.jira.service.services.mail.MailFetcherService.runImpl(MailFetcherService.java:426)
        at com.atlassian.jira.service.services.file.AbstractMessageHandlingService.run(AbstractMessageHandlingService.java:263)
        at com.atlassian.jira.service.JiraServiceContainerImpl.run(JiraServiceContainerImpl.java:66)
        at com.atlassian.jira.service.ServiceRunner.runService(ServiceRunner.java:75)
        at com.atlassian.jira.service.ServiceRunner.runServiceId(ServiceRunner.java:53)
        at com.atlassian.jira.service.ServiceRunner.runJob(ServiceRunner.java:36)
        at com.atlassian.scheduler.core.JobLauncher.runJob(JobLauncher.java:135)
        at com.atlassian.scheduler.core.JobLauncher.launchAndBuildResponse(JobLauncher.java:101)
        at com.atlassian.scheduler.core.JobLauncher.launch(JobLauncher.java:80)
        at com.atlassian.scheduler.quartz1.Quartz1Job.execute(Quartz1Job.java:32)
        at org.quartz.core.JobRunShell.run(JobRunShell.java:223)
        at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:549)

        and

        javax.mail.MessageRemovedException
        at com.sun.mail.imap.IMAPMessage.checkExpunged(IMAPMessage.java:205)
        at com.sun.mail.imap.IMAPMessage.getFrom(IMAPMessage.java:236)
        at com.metainf.jira.plugin.emailissue.handler.EmailThisIssueMailHandler.saveSenderInField(EmailThisIssueMailHandler.java:622)
        at com.metainf.jira.plugin.emailissue.handler.EmailThisIssueMailHandler.access$100(EmailThisIssueMailHandler.java:59)
        at com.metainf.jira.plugin.emailissue.handler.EmailThisIssueMailHandler$1.createComment(EmailThisIssueMailHandler.java:160)
        at com.metainf.jira.plugin.emailissue.handler.IssueSupportingMessageHandlerContext.createComment(IssueSupportingMessageHandlerContext.java:39)
        at com.metainf.jira.plugin.emailissue.handler.IssueSupportingMessageHandlerContext.createComment(IssueSupportingMessageHandlerContext.java:39)
        at com.atlassian.jira.plugins.mail.handlers.AbstractMessageHandler.addCommentIndicatingAttachmentFailureAndMarkForDeletion(AbstractMessageHandler.java:865)
        at com.atlassian.jira.plugins.mail.handlers.CreateIssueHandler.addCcWatchersAndAttachments(CreateIssueHandler.java:313)
        at com.atlassian.jira.plugins.mail.handlers.CreateIssueHandler.handleMessage(CreateIssueHandler.java:276)
        at com.metainf.jira.plugin.emailissue.handler.CreateAndLinkHandler.handleMessage(CreateAndLinkHandler.java:48)
        at com.metainf.jira.plugin.emailissue.handler.AbstractMailHandler.handleMessage(AbstractMailHandler.java:96)
        at com.metainf.jira.plugin.emailissue.handler.EmailThisIssueMailHandler.handleMessage(EmailThisIssueMailHandler.java:419)
        at com.atlassian.jira.service.services.mail.MailFetcherService$1.process(MailFetcherService.java:438)
        at com.atlassian.jira.service.services.mail.MailFetcherService$MessageProviderImpl.getAndProcessMail(MailFetcherService.java:304)
        at com.atlassian.jira.service.services.mail.MailFetcherService.runImpl(MailFetcherService.java:426)
        at com.atlassian.jira.service.services.file.AbstractMessageHandlingService.run(AbstractMessageHandlingService.java:263)
        at com.atlassian.jira.service.JiraServiceContainerImpl.run(JiraServiceContainerImpl.java:66)
        at com.atlassian.jira.service.ServiceRunner.runService(ServiceRunner.java:75)
        at com.atlassian.jira.service.ServiceRunner.runServiceId(ServiceRunner.java:53)
        at com.atlassian.jira.service.ServiceRunner.runJob(ServiceRunner.java:36)
        at com.atlassian.scheduler.core.JobLauncher.runJob(JobLauncher.java:135)
        at com.atlassian.scheduler.core.JobLauncher.launchAndBuildResponse(JobLauncher.java:101)
        at com.atlassian.scheduler.core.JobLauncher.launch(JobLauncher.java:80)
        at com.atlassian.scheduler.quartz1.Quartz1Job.execute(Quartz1Job.java:32)
        at org.quartz.core.JobRunShell.run(JobRunShell.java:223)
        at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:549)

          Form Name

            [JRASERVER-42044] Race condition when multiple mail handlers access the same mail box

            Mauro Badii (Inactive) added a comment -

            That is right jaakko.linnosaari
            If you encounter any problems, please create a support request and we'll be happy to help.

            Cheers!

            Mauro Badii (Inactive) added a comment - That is right jaakko.linnosaari If you encounter any problems, please create a support request and we'll be happy to help. Cheers!

            Jaakko Linnosaari added a comment -

            So does that mean that one can now use multiple mail handlers with distinct Catch Email Address -parameters on the same mailbox folder with 7.0.2 and newer? That'd be useful.

            Jaakko Linnosaari added a comment - So does that mean that one can now use multiple mail handlers with distinct Catch Email Address -parameters on the same mailbox folder with 7.0.2 and newer? That'd be useful.

            Mauro Badii (Inactive) added a comment -

            This race condition should be now fixed by the work done on JRA-41831

            Mauro Badii (Inactive) added a comment - This race condition should be now fixed by the work done on JRA-41831

            ChrisV added a comment -

            Thanks Os!

            ChrisV added a comment - Thanks Os!

            Oswaldo Hernandez (Inactive) added a comment -

            Hi chris.lepetit,

            We introduced the change of behaviour with the fix done at JRA-35138 (JIRA 6.3.7)

            After reviewing the solution to that bug with the team we determined that we did not contemplate the scenario where multiple mail handlers have been setup to monitor a single mailbox.

            This initially sounds like a configuration error, but after review we determined that it's ok to configure multiple exclusive mail handlers (for example using the "Catch email address" option). As long as the configuration is such that for A single email there will be only be one mail handler that will pick it up, then this is a valid set-up. We have developed a fix for this scenario under JRA-41831 which should ship with JIRA 7.0.2

            On the other hand, a configuration error would be having multiple mail handlers that would actually try to handle the same email - this is a race condition because each mail handler individually would try to delete the message. This is unsupported and it is a configuration error that should be fixed by the JIRA Administrator.

            Hope that clarifies the status of this one.

            Cheers,
            Os.

            Oswaldo Hernandez (Inactive) added a comment - Hi chris.lepetit , We introduced the change of behaviour with the fix done at JRA-35138 (JIRA 6.3.7) After reviewing the solution to that bug with the team we determined that we did not contemplate the scenario where multiple mail handlers have been setup to monitor a single mailbox. This initially sounds like a configuration error, but after review we determined that it's ok to configure multiple exclusive mail handlers (for example using the "Catch email address" option). As long as the configuration is such that for A single email there will be only be one mail handler that will pick it up, then this is a valid set-up. We have developed a fix for this scenario under JRA-41831 which should ship with JIRA 7.0.2 On the other hand, a configuration error would be having multiple mail handlers that would actually try to handle the same email - this is a race condition because each mail handler individually would try to delete the message. This is unsupported and it is a configuration error that should be fixed by the JIRA Administrator. Hope that clarifies the status of this one. Cheers, Os.

            ChrisV added a comment - - edited

            ohernandez Did this behaviour change in 6.3.15 or a different version? It would be handy to know which version introduced the change so we can look out for this situation during upgrades.

            ChrisV added a comment - - edited ohernandez Did this behaviour change in 6.3.15 or a different version? It would be handy to know which version introduced the change so we can look out for this situation during upgrades.

            Oswaldo Hernandez (Inactive) added a comment -

            Hi jaakko.linnosaari,

            Indeed, I have asked our tech writers to update the mail handler documentation accordingly

            Regards,

            Oswaldo Hernández.
            JIRA Bugmaster.
            [Atlassian].

            Oswaldo Hernandez (Inactive) added a comment - Hi jaakko.linnosaari , Indeed, I have asked our tech writers to update the mail handler documentation accordingly Regards, Oswaldo Hernández. JIRA Bugmaster. [Atlassian] .

            Jaakko Linnosaari added a comment -

            Ok.

            It should be noted, however, that these mail handlers do not have to have matching Catch Email Address -parameters as the duplicate check is performed before checking any other headers. Documentation should at least have warning of the possibility of irreversible data loss (email is deleted before being fully processed) if mail handlers are configured this way.

            Cheers,
            Jaakko

            Jaakko Linnosaari added a comment - Ok. It should be noted, however, that these mail handlers do not have to have matching Catch Email Address -parameters as the duplicate check is performed before checking any other headers. Documentation should at least have warning of the possibility of irreversible data loss (email is deleted before being fully processed) if mail handlers are configured this way. Cheers, Jaakko

            Oswaldo Hernandez (Inactive) added a comment -

            Hi jaakko.linnosaari,

            I have just a discussion with the team and the current design of the mail handler sub-system is such that there is an underlying assumption that only one mail handler will be accessing a given mailbox at the time.

            Even though configurable, the system is not designed to cope with two or more different mail handlers accessing the same mailbox and It is advised to only configure one handler to read messages from a given mailbox.

            Hope the information helps.

            Regards,

            Oswaldo Hernández.
            JIRA Bugmaster.
            [Atlassian].

            Oswaldo Hernandez (Inactive) added a comment - Hi jaakko.linnosaari , I have just a discussion with the team and the current design of the mail handler sub-system is such that there is an underlying assumption that only one mail handler will be accessing a given mailbox at the time. Even though configurable, the system is not designed to cope with two or more different mail handlers accessing the same mailbox and It is advised to only configure one handler to read messages from a given mailbox. Hope the information helps. Regards, Oswaldo Hernández. JIRA Bugmaster. [Atlassian] .

            Jaakko Linnosaari added a comment -

            It might be worth mentioning, that we observed this issue when accessing a mailbox on Microsoft's Office365 cloud service.

            Jaakko Linnosaari added a comment - It might be worth mentioning, that we observed this issue when accessing a mailbox on Microsoft's Office365 cloud service.

              Unassigned Unassigned
              9da142218f90 Jaakko Linnosaari
              Affected customers:
              1 This affects my team
              Watchers:
              20 Start watching this issue

                Created:
                Updated:
                Resolved: