Low Issue Summary
When the sd.internal.base.db.backed.completion.events.enabled dark feature is enabled, JSM uses the "DB backed" off thread system. The thread pool used to consume events added to PSMQ is named SdSerialisedOffThreadProcessor-n
This pool is fixed to a size of 5, which breaks scaling message consumption when this pool type is in effect.
Customers who have a significant amount of issue events (update/comment etc) could find that this fixed pool size is not enough to consume events more than they are added, leading to an infinite build up of the AO_319474_MESSAGE table
Steps to Reproduce
- Set Dark Feature sd.internal.base.db.backed.completion.events.enabled
- Put load onto Jira
- Note that the thread pool is fixed to 5
Setting the defined workaround to increase the SdOffThreadEventJobRunner pool size, sd.event.processing.async.thread.pool.count has no effect
Expected Results
It's not clear. If we expect the previous workaround to take effect, the workaround should result in.
If not, then, we should add a new property to define the pool size - preferably, one that does not require DB manipulation.
Actual Results
The pool size is fixed and does not grow
Workaround
It is not possible to increase the SdSerialisedOffThreadProcessor pool size.
The only options are to either:
- Go back to the deadlock risk SdOffThreadEventJobRunner pool, by removing the dark feature internal.base.db.backed.completion.events.enabled
- Disable off-threading completely by setting dark features
sd.internal.bounded.off.thread.on.completion.events.disabled sd.internal.base.off.thread.on.completion.events.disabled
- is detailed by
-
JSMDC-10161 You do not have permission to view this issue
| Form Name | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
[JSDSERVER-8502] SdSerialisedOffThreadProcessor thread pool is fixed to 5 threads and cannot be increased
Hi Alex,
- Yes, the query (queries) to change sd.event.processing.serialised.thread.pool.count is exactly the same.
- I mentioned re-enabling that feature because the workaround suggested removing the dark feature for the time being and falling back to the older thread pool. I was saying that if someone has done that, they would now be able to re-enable the feature. Otherwise they can just set the sd.event.processing.serialised.thread.pool.count value in the DB.
Thanks for quickly triaging this a06a9964b472 . Could I plese ask
- Is the query to change sd.event.processing.serialised.thread.pool.count the same as "sd.event.processing.async.thread.pool.count" ?
- You mention re-enabling sd.internal.base.db.backed.completion.events.enabled . Why should we do that? It is removed automatically for customers who have it when upgrade to 4.19.0?
The SdSerialisedOffThreadProcessor thread pool, in order to not conflict with the SdOffThreadJobRunner thread pool name, has its own database setting "sd.event.processing.serialised.thread.pool.count" that is different from the previous "sd.event.processing.async.thread.pool.count" setting, as of 4.19.0.
Therefore, as of 4.19.0 you should be able to set this value in the database, re-enable the internal.base.db.backed.completion.events.enabled dark feature, and restart the service desk instance to be able to set the number of threads in the thread pool.
Awesome, thanks for confirming!