Hi Ivar and Jed,

Without any knowledge of the forum discussion or JRA-16347, last night I wrote and ran a JSP very similar to Ivar's suggestion in JRA-16347 so that we could 'silently' re-index our whole system and leave it online and not have to take Jira down for 40 minutes to re-index. Index Optimization was off. It ran and no errors were reported. It took 5 hours to re-index 130,000 issues.

We thought everything was OK but then this morning we had 2 reports of filters displaying incorrect values for 1 issue each that did not match each issue's actual values. To re-re-index any issues that might have become corrupted, I put a comment on all 60 issues touched during the 5 hours.

Now I've read this forum discussion and JRA-16347 and I'm wondering if Thread.yield() might avoided the problem.
However, if there was lock contention from a user action, wouldn't the contention have caused an exception which I would see in the logs? I have no errors reported in the logs during the 5 hours that the re-index JSP was running.
Could having the OfBizListIterator open for 5 hours have caused the problem? But if it had wouldn't that have reported an exception?

I would really love to see JRA-7842 implemented.

Thanks,

Jeff

Please see this forum discussion for some more information on this.

Ivar,

The details of the implementation aren't important, the feature is the same that's why it is a duplicate. If we implement this ourselves we will probably go with a solution like yours anyway.

Your approach will pull the entire ResultSet into memory, so it will not scale well if there are very large numbers of issues. This should be OK as long as you only use it in situations where you know the number of issues is not that large and you monitor the memory usage of the application carefully.

Please make sure you call Thread.yield(); after you index an issue, this allows any other thread that is waiting for the index lock to proceed without you reacquiring the lock beforehand (locking in Java is not fair).

JIRA will perform an optimisation on the index after a certain number of updates. You should make sure that this is not triggered during your reindex by changing the value of:

# Number of issue index updates before automatic index optimization is triggered.
# A value of 0 indicates never trigger.
jira.index.max.reindexes=4000

in the jira-application.properties. Either set it to zero and manually optimise the index (or just leave it to the midnight optimise JIRA performs automatically), or set it to a number larger than the total number of issues. Failure to do this will cause index optimisations during your run, potentially leading to more index time-outs which prevent issues from being indexed - somewhat defeating the purpose.

Lastly, do not swallow the IndexException. If it is caught it means there are serious problems and all your code will do from then on is encounter more. Re-throw it as a RuntimeException.

I beg to differ with you. JRA-7842 suggests creating a new and separate index. What I suggest is using the same index but update it - one issue at a time. This should be safe and essential the same as if a user maually went and updated all issues, one at a time. It should thus be easy to implement and not break anything.

The reason I am asking you about this (instead of just making a plugin that does this and starting to use it) is that is there something I am possibly forgetting? Why have you not implemented this?

Here's a sample on how it could be implemented (I tested it within a service - but optimally it should be a web-item with a progress bar):

IssuesIterable issuesIterable = new DatabaseIssuesIterable(new DefaultOfBizDelegator(CoreFactory.getGenericDelegator()),issueFactory);

for (IssueIterator iterator = issuesIterable.iterator(); iterator.hasNext();) {
    AbstractIssue issue = (AbstractIssue) iterator.next();
    try {
        issueIndexManager.reIndex(issue);
    } catch (IndexException e) {
        e.printStackTrace();
    }
}

Simple but slow to run. However the slowliness should not be a problem as Jira is not blocked when you perform the reindexing this way. What am I missing?

Hi,

This issue is a duplicate of JRA-7842 and therefore I will resolve it.

Please vote for JRA-7842 and add yourself as a watcher to it.

For more information on the way new feature and improvement requests are scheduled, please see:
http://confluence.atlassian.com/display/DEV/Implementation+of+New+Features+and+Improvements

Cheers,
Anton