|
History
| Log In
|
|
|
|
Hi Alexey,
Removing the synchronization from JiraLuceneFieldCache is tracked via JRA-13296. The memory consumption is a separate issue. Unfortuantely the way things are done now (creating arrays) is the way Lucene does sorting. The reason for this is speed. We have not measured the effects of this but as far as we know the speed is affected to a large degree if values are compared directly from Indexex as a trip to disk for the document needs to be made for each value. We do plan to look at this in the future however at the moment I do not have a timeline for this. We will need to ensure that the changes that we make do not bring people's dashboards to a grinding hault. Cheers, I'm guess, creation 260-280mb objects per everyoone issue update is too much speed affected issue as reading from index by doc offset (via RandomAccessFile, i suppose). And i'm realy think that it is will be faster. I'm not will patch JIRA for this, and will measure results. If it will OK, i'm notify you.
Alexey,
It should only create objects for the comparators that are actually used. So if people do sort by 45 different columns between issue updates, then it will load this into memory. However, most people only sort by a certain subset of fields. It should be stored in a WeakHashMap, so if the sorts are not being used, then they will be reclaimed by memory. So it should not be a memory issue that you run into. We have seen memory issues if you have hundreds of people concurrently searching at the same time that you have issue updates, so there become many copies of this comparator in memory. Another thing we could do is reduce the size of those objects in memory. I haven't looked at each sortcomparator, but there is a chance that we could store smaller objects in memory than we do. One example of how we do this is the text sort comparators, where we just store the first 30 bytes in memory, and not the entire text field. This optimisation may be able to be applied to other fields. I'd love to see any results you get from looking into this. It has been a problem with other customers also, so any creative solutions we can brainstorm together would be great. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
 |
|
Task
Resolved
Minor
Remove JiraLuceneFieldCache cache
This comparator should compare values directly from Indexer, do not cache values.
Please look on com.atlassian.jira.issue.search.parameters.lucene.sort.JiraLuceneFieldCache - it is a smell class, please remove it all (synchronizations and dummy creation of objects not making GC happy).
Also please look onto com.atlassian.jira.issue.search.parameters.lucene.sort.MappedSortComparator - we have suddenly OutOfMemory errors for JIRA intensive using. See on field cachedValues - it is an array with size = number of issues in JIRA (in our case ~80000), total count of comparators ~45 in our case. 45 * 80000 with values ~260mb retain size objects in memory. But on everyone issue update, you must recreate all 45 comparators with 80000 arrays in each of them.
Please review this place again and make customers happy with out patches...
This comparator should compare values directly from Indexer, do not cache values.