JIRA creates duplicate issues when the standard mail handler has to process a large number of emails or large amounts of data..

Summary

Duplicate issues are created when the mail handler has to process a mailbox with a large number of emails or a small number of emails with large attachments as the caesium threads get stuck processing emails.

Although there is not exact figure to define "a large number of emails" in an attempt to provide a guideline for future issues that can be diagnose as this bug what it was found is that the mailbox has to have more 500 emails to be processed and it was also observed that when the first job processing emails reached approximately the 200th is when duplicates issues start appearing in the system.

It is also important to note that the same problem can happen when the mail handler has to process large amounts of data rather than a large number of emails, for example a few emails with big attachments (files larger than 8 or 10 MB), as the problem is related to the fact that if the running job on one node (job-A on node-1)takes longer than the time that the next job of the same type (job-B) is scheduled to run and it is scheduled to run on a different node than node-1 (for example node-2) then we will see both jobs (job-A and job-B) running at the same time on two different nodes (node-A and node-B).

To summarize, the problem is that the scheduler ensures that a clustered job will not run on the same node that is already running but will not check the same for other nodes in the JIRA DC cluster.

Environment

• JIRA Data Center 7.2.9 with two or mode nodes active in the cluster.
• Jira standard mail handler configured
• POP or IMAP mail box

Steps to Reproduce

1. Create or find a POP or IMAP mailbox with either more than 25 emails having big files attached to them or more than 500 emails without attachments. (Please see the Additional notes section below)
2. Install JIRA Data Center 7.2.9 and start up two nodes
3. Go to the Incoming Mail screen (<IP>:<PORT>/secure/admin/IncomingMailServers.jspa)
4. Add and configure one mail server (it does not matter if it is a pop or imap)
5. Add an incoming mail handler
6. Configure the incoming mail handler as follows:
   1. Specify a name
   2. Select the mail server previously created
   3. Delay --> 1
   4. Handler --> Create a new issue from each email message
   5. Project --> choose a project name
   6. Default Reporter --> choose the user that will be the reporter of the created issues
   7. Leave all other fields not mentioned above as per the respective default settings
   8. Click on Save
7. Wait until the mail handler starts connecting to the mail box and starts processing email.
8. Monitor the issues being created

Expected Results

It is expected that only one issue per email processed is created in the respective project.

Actual Results

Duplicate issues are created when having two nodes in the JIRA Data Center cluster.

Note

Problem caused by 2 factors:

• JIRA Datacenter scheduler doesn't guaranty that only one job will be running at specific moment of time across the cluster (Singleton). See: JRASERVER-65599
• There is no cluster lock for Mail Handler.

Workaround

It is less likely to reproduce the issue when the "Delay" parameter (in the incoming mail handler configuration) has a higher value than 1 minute, however that also depends on how much data needs to be processed and hence in some cases increasing it to 3 minutes can workaround the issue whereas in others it has to be 5 minutes or higher.

Set the "Delay" parameter to a higher value than 1 minute (keep increasing it if you still see duplicates) and monitor the system until no duplicates are created.