Details
-
Bug
-
Resolution: Fixed
-
Medium
-
2.10.0
Description
Symptom: for a specific repository JIRA issues in commit messages are no longer indexed.
In Stash 2.10 when the indexing of commits takes longer than 10 minutes (which is very unusual) a timeout occurs and the repository that is being indexed is now "locked" for indexing until Stash is restarted. Plugins like Badgr can exacerbate the time taken to index a single repository, and should be disabled until Stash can be upgraded to a version without this bug.
To tell if this has happened to your index, look for the following in your $STASH_HOME/log/atlassian-stash.log file.
2013-12-08 22:28:00,202 INFO [Indexer:thread-2] user 166x2877x0 jbz125 10.33.68.50 SSH - git-receive-pack '/project1/repo1.git' c.a.s.i.i.ChangesetIndexingServiceImpl [PROJECT/repo[227]] Indexing SCM process has timed out, wrapping up the process
And this is not immediately followed by:
2013-12-17 21:04:31,770 DEBUG [Indexer:thread-2] user 166x2877x0 jbz125 10.33.68.50 SSH - git-receive-pack '/project1/repo1.git' c.a.s.i.i.ChangesetIndexingServiceImpl [~PROJECT/repo[227]] Indexed 1234 changesets
Running a jstack command on the Stash Java process, or generating a Support Zip and looking at the thread-dump folder will contain a thread that is locked up:
[441] Indexer:thread-1: WAITING (waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@291220f2) sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) java.util.concurrent.LinkedBlockingQueue.put(LinkedBlockingQueue.java:349) com.atlassian.stash.internal.idx.BlockingChangesetQueueCallback.onEnd(BlockingChangesetQueueCallback.java:58) com.atlassian.stash.content.AbstractChangesetCallback.onEnd(AbstractChangesetCallback.java:40) com.atlassian.stash.internal.scm.git.command.revlist.CallbackRevListOutputHandler.onComplete(CallbackRevListOutputHandler.java:43) com.atlassian.stash.scm.SummarizingProcessHandler$AbstractStreamHandler.onComplete(SummarizingProcessHandler.java:141) com.atlassian.stash.scm.SummarizingProcessHandler.fireComplete(SummarizingProcessHandler.java:112) com.atlassian.stash.scm.SummarizingProcessHandler.complete(SummarizingProcessHandler.java:62) com.atlassian.utils.process.ExternalProcessImpl.wrapUpProcess(ExternalProcessImpl.java:659) com.atlassian.utils.process.ExternalProcessImpl.finish(ExternalProcessImpl.java:189) com.atlassian.stash.scm.BaseCommand$CommandFuture.cancel(BaseCommand.java:215) com.atlassian.stash.internal.idx.ChangesetIndexingServiceImpl$IndexOperation.wrapUpCommand(ChangesetIndexingServiceImpl.java:489) com.atlassian.stash.internal.idx.ChangesetIndexingServiceImpl$IndexOperation.indexChangesets(ChangesetIndexingServiceImpl.java:447) com.atlassian.stash.internal.idx.ChangesetIndexingServiceImpl$IndexOperation.index(ChangesetIndexingServiceImpl.java:366) com.atlassian.stash.internal.idx.ChangesetIndexingServiceImpl$IndexOperation.perform(ChangesetIndexingServiceImpl.java:273) com.atlassian.stash.internal.idx.ChangesetIndexingServiceImpl.indexRepository(ChangesetIndexingServiceImpl.java:146)
To fix the indexing unfortunately the Stash instance will need to be restarted.