Loading...

Type: Bug
Resolution: Unresolved
Priority: Low
Fix Version/s: None
Affects Version/s: 6.3.15, 6.4.13, 7.3.0
Component/s: Email notifications
Labels:

Introduced in Version:
6.03
Support reference count:
68
Symptom Severity:
Severity 3 - Minor
UIS:
9
Bug Fix Policy:
View Atlassian Server bug fix policy

NOTE: This bug report is for JIRA Server. Using JIRA Cloud? See the corresponding bug report.

Summary

JIRA Outgoing mail stop sending mails out to users for roughly 4 hours when it is processing Group Filter Subscription with the particular group has more than 1700 users.

Environment

500,000 issues
2000 users
SMTP outgoing mail configured

Steps to Reproduce

Create a JQL filter that uses currentUser() function
Has a group that consist of 1700 users
Set a Group Filter Subscription to use the group and JQL filter created

Diagnostic

Generating thread dumps when Mail queue is stuck show long running thread of the following:

"Sending mailitem com.atlassian.jira.mail.SubscriptionMailQueueItem owner: 'username(username)'" #110 prio=4 os_prio=-1 tid=0x0000000085c7e800 nid=0x1640 runnable [0x000000008fb1e000]
   java.lang.Thread.State: RUNNABLE
	at org.apache.lucene.index.SegmentTermDocs.read(SegmentTermDocs.java:157)
	at org.apache.lucene.search.MultiTermQueryWrapperFilter.getDocIdSet(MultiTermQueryWrapperFilter.java:122)
	at org.apache.lucene.search.ConstantScoreQuery$ConstantWeight.scorer(ConstantScoreQuery.java:139)
	at org.apache.lucene.search.BooleanQuery$BooleanWeight.scorer(BooleanQuery.java:298)
	at org.apache.lucene.search.IndexSearcher.searchWithFilter(IndexSearcher.java:542)
	at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:532)
	at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:463)
	at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:433)
	at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:356)
	at com.atlassian.jira.index.DelegateSearcher.search(DelegateSearcher.java:165)
	at com.atlassian.jira.issue.search.providers.LuceneSearchProvider.runSearch(LuceneSearchProvider.java:566)
	at com.atlassian.jira.issue.search.providers.LuceneSearchProvider.getHits(LuceneSearchProvider.java:294)
	at com.atlassian.jira.issue.search.providers.LuceneSearchProvider.search(LuceneSearchProvider.java:421)
	at com.atlassian.jira.issue.search.providers.LuceneSearchProvider.search(LuceneSearchProvider.java:413)
	at com.atlassian.jira.issue.search.providers.LuceneSearchProvider.search(LuceneSearchProvider.java:104)
	at com.atlassian.jira.issue.search.providers.LuceneSearchProvider.search(LuceneSearchProvider.java:92)
	at com.atlassian.jira.mail.SubscriptionMailQueueItem.getContextParams(SubscriptionMailQueueItem.java:336)
	at com.atlassian.jira.mail.SubscriptionMailQueueItem.sendSearchRequestEmail(SubscriptionMailQueueItem.java:174)
	at com.atlassian.jira.mail.SubscriptionMailQueueItem.send(SubscriptionMailQueueItem.java:126)
	at com.atlassian.mail.queue.MailQueueImpl.sendBuffer(MailQueueImpl.java:66)

Notice that an entry is logged to atlassian-jira.log, showing the number of recipients on the subscription that was triggered, if it is above 30:

2024-08-15 12:00:01,501-0400 Sending mailitem com.atlassian.jira.mail.SubscriptionMailQueueItem id: '100001' owner: 'myuser' INFO anonymous    Mail Queue Service [c.a.jira.mail.SubscriptionMailQueueItem] Sending subscription '100001' of filter '123123' to 987 recipients.

Expected Results

Outgoing mail continue to send out while Group Filter Subscription is processing the mails to be added in the Mail queue

Actual Results

Mail queue is stuck, no outgoing mail is send. Manually flushing mail queue does not work.

Notes

Filter Subscription that will generate the mails for each users will obtain a lock on the Mail queue on MailQueueImpl#addItem. The mail sender couldn't take lock for MailQueueImpl#sendBuffer to send those mails that already in the Mail queue due to it being locked by Filter subscription.

Workaround

Depending on use-case:

Set Group Filter Subscription to use a group that has a smaller user count <200 users.
Create a mailing list in Mail Server to be configured for the Filter Subscription. Create a dummy user in JIRA to be assign with the mailing list and the user will be the one subscribing to the Filter Subscription. Users who should be receiving mail subscription needs to be part of the mailing list.

is related to

JRASERVER-74338 Filter subscriptions for too many recipients may result in performance degradation or OutOfMemoryError in Jira

Closed

JRASERVER-31588 Large filter subscriptions can crash a JIRA instance with an OutOfMemoryError

Closed

JRASERVER-68697 Subscribe Filter To A Large Group Of User Causing JIRA Trigger Subscription Mail To All User In The Group Even If "email this filter, even if there are no issues found" Was Unchecked.

Gathering Impact

JRASERVER-61548 Show status of Outgoing Mail

Gathering Interest

relates to

JRASERVER-31588 Large filter subscriptions can crash a JIRA instance with an OutOfMemoryError

Closed

JRACLOUD-61543 Outgoing Mail Stopped Working due to Large Group Filter Subscription

Closed

JRASERVER-65963 Outgoing mail queue processing is delayed due to thread taking a long time to process what was changed in an issue

Gathering Impact

JRASERVER-71503 As an Jira Administrator I want to trace of the email creation process and check it's speed

Gathering Interest

mentioned in: Page Loading...; Page Loading...; Page Loading...; Page Loading...; Page Loading...; Page Loading...; Page Loading...

(3 relates to, 7 mentioned in)

Details

Description

Summary

Environment

Steps to Reproduce

Diagnostic

Expected Results

Actual Results

Notes

Workaround

Attachments

Issue Links

Forms

Activity

People

Dates