Uploaded image for project: 'Jira Service Management Data Center'
  1. Jira Service Management Data Center
  2. JSDSERVER-6470

Canned response error when trying to define/select a canned response

      Issue Summary

      Environment

      Seems to happen after upgrade to 4.0

      Steps to Reproduce

      Cannot reliably reproduce

      Expected Results

      Canned responses can be selected/edited etc.

      Actual Results

      The below error is thrown in the UI:

      We can't add canned response data to the search index right now. You could try again or contact your administrator
      

      The below exception is thrown in the log file:

      2019-05-13 12:49:33,047 http-nio-8080-exec-519 WARN  769x169315x1  /rest/servicedesk/canned-responses/1/search/PROJECT [cannedresponses.internal.index.CannedResponseIndexManagerImpl] There was an error while searching for Canned Response : CannedResponseSearchParams{query=Optional.empty, userKey=Optional.empty, date=Optional.empty, cannedResponseSearchSubstitutionParams=Optional[CannedResponseSearchSubstitutionParams{issue=EXT-2111}], includeDisabled=false, serviceDesk=ServiceDeskImpl{serviceDeskId=3, projectId=10700, projectName=IRT Support, accessConfig=AccessConfig{publicSignUp=false, openAccess=false}, createdByUserKey=some(hont), createdDate=some(2018-10-23 07:48:04.688), createdWithEmptyProject=some(true), createdAtVersion=some(3.15.3-REL-0003), legacyCommentTransitionDisabled=true}, limitedPagedRequest=LimitedPagedRequestImpl{start=0, limit=5, maxLimit=500}}
      com.atlassian.servicedesk.plugins.cannedresponses.internal.lucene.search.CannedResponseSearcherImpl$CannedResponseSearchException: Error while performing Canned Response search
              at com.atlassian.servicedesk.plugins.cannedresponses.internal.lucene.search.CannedResponseSearcherImpl.lambda$search$2(CannedResponseSearcherImpl.java:63)
              at io.atlassian.fugue.Option$None.getOrThrow(Option.java:409)
              at com.atlassian.servicedesk.plugins.cannedresponses.internal.lucene.search.CannedResponseSearcherImpl.search(CannedResponseSearcherImpl.java:63)
              at com.atlassian.servicedesk.plugins.cannedresponses.internal.index.CannedResponseIndexManagerImpl.lambda$search$3(CannedResponseIndexManagerImpl.java:204)
              at io.atlassian.fugue.Option.map(Option.java:259)
              at com.atlassian.servicedesk.plugins.cannedresponses.internal.index.CannedResponseIndexManagerImpl.search(CannedResponseIndexManagerImpl.java:204)
              at com.atlassian.servicedesk.plugins.cannedresponses.internal.search.CannedResponseSearchManager.searchCannedResponses(CannedResponseSearchManager.java:65)
              at com.atlassian.servicedesk.plugins.cannedresponses.internal.search.CannedResponseSearchService.lambda$searchCannedResponses$7(CannedResponseSearchService.java:129)
              at com.atlassian.pocketknife.step.EitherStep4.lambda$null$0(EitherStep4.java:29)
              at io.atlassian.fugue.Either$RightProjection.flatMap(Either.java:937)
              at io.atlassian.fugue.Either.flatMap(Either.java:231)
              at com.atlassian.pocketknife.step.EitherStep4.lambda$null$1(EitherStep4.java:29)
              at io.atlassian.fugue.Either$RightProjection.flatMap(Either.java:937)
              at io.atlassian.fugue.Either.flatMap(Either.java:231)
              at com.atlassian.pocketknife.step.EitherStep4.lambda$null$2(EitherStep4.java:28)
              at io.atlassian.fugue.Either$RightProjection.flatMap(Either.java:937)
              at io.atlassian.fugue.Either.flatMap(Either.java:231)
              at com.atlassian.pocketknife.step.EitherStep4.lambda$then$3(EitherStep4.java:27)
              at io.atlassian.fugue.Either$RightProjection.flatMap(Either.java:937)
              at io.atlassian.fugue.Either.flatMap(Either.java:231)
              at com.atlassian.pocketknife.step.EitherStep4.then(EitherStep4.java:26)
              at com.atlassian.servicedesk.plugins.cannedresponses.internal.search.CannedResponseSearchService.searchCannedResponses(CannedResponseSearchService.java:124)
              at com.atlassian.servicedesk.plugins.cannedresponses.internal.rest.resources.CannedResponseSearchResource.lambda$searchCannedResponses$1(CannedResponseSearchResource.java:76)
              at io.atlassian.fugue.Either$RightProjection.flatMap(Either.java:937)
              at io.atlassian.fugue.Either.flatMap(Either.java:231)
              at com.atlassian.pocketknife.step.EitherStep1.then(EitherStep1.java:18)
              at com.atlassian.servicedesk.plugins.cannedresponses.internal.rest.resources.CannedResponseSearchResource.searchCannedResponses(CannedResponseSearchResource.java:76)
      
      
      2019-05-13 12:49:33,056 http-nio-8080-exec-544 WARN  769x169316x1   /rest/servicedesk/canned-responses/1/search/PROJECT [internal.lucene.search.CannedResponseManagedIndexSearcherFactory] Unexpected exception during execution of canned response index searcher
      com.atlassian.jira.util.RuntimeIOException: org.apache.lucene.store.LockObtainFailedException: Lock held by this virtual machine: /var/atlassian/jira/caches/indexesV1/plugins/servicedeskcannedresponses/write.lock
              at com.atlassian.jira.index.WriterWrapper$1.get(WriterWrapper.java:66)
              at com.atlassian.jira.index.WriterWrapper$1.get(WriterWrapper.java:57)
              at com.atlassian.jira.index.WriterWrapper.<init>(WriterWrapper.java:50)
              at com.atlassian.jira.index.WriterWrapper.<init>(WriterWrapper.java:57)
              at com.atlassian.jira.index.DefaultIndexEngine$DefaultWriterFactory.apply(DefaultIndexEngine.java:224)
              at com.atlassian.jira.index.DefaultIndexEngine$DefaultWriterFactory.apply(DefaultIndexEngine.java:220)
              at com.atlassian.jira.index.DefaultIndexEngine$WriterReference.doCreate(DefaultIndexEngine.java:205)
              at com.atlassian.jira.index.DefaultIndexEngine$WriterReference.doCreate(DefaultIndexEngine.java:182)
              at com.atlassian.jira.index.DefaultIndexEngine$ReferenceHolder$2.get(DefaultIndexEngine.java:267)
              at com.atlassian.jira.concurrent.ResettableLazyReference.getOrCreateUnderLock(ResettableLazyReference.java:97)
              at com.atlassian.jira.concurrent.ResettableLazyReference.getOrCreate(ResettableLazyReference.java:89)
              at com.atlassian.jira.index.DefaultIndexEngine$ReferenceHolder.apply(DefaultIndexEngine.java:264)
              at com.atlassian.jira.index.DefaultIndexEngine$SearcherFactoryImpl.useNRT(DefaultIndexEngine.java:375)
              at com.atlassian.jira.index.DefaultIndexEngine$SearcherFactoryImpl.openIndexReader(DefaultIndexEngine.java:351)
              at com.atlassian.jira.index.DefaultIndexEngine$SearcherFactoryImpl.get(DefaultIndexEngine.java:339)
              at com.atlassian.jira.index.DefaultIndexEngine$SearcherFactoryImpl.get(DefaultIndexEngine.java:297)
              at com.atlassian.jira.index.DefaultIndexEngine$SearcherReference.doCreate(DefaultIndexEngine.java:163)
              at com.atlassian.jira.index.DefaultIndexEngine$SearcherReference.doCreate(DefaultIndexEngine.java:153)
              at com.atlassian.jira.index.DefaultIndexEngine$ReferenceHolder$2.get(DefaultIndexEngine.java:267)
              at com.atlassian.jira.concurrent.ResettableLazyReference.getOrCreateUnderLock(ResettableLazyReference.java:97)
              at com.atlassian.jira.concurrent.ResettableLazyReference.getOrCreate(ResettableLazyReference.java:89)
              at com.atlassian.jira.index.DefaultIndexEngine$ReferenceHolder.apply(DefaultIndexEngine.java:264)
              at com.atlassian.jira.index.DefaultIndexEngine.getSearcher(DefaultIndexEngine.java:120)
              at com.atlassian.jira.index.DefaultManager.openSearcher(DefaultManager.java:36)
              at io.atlassian.fugue.Option.map(Option.java:259)
              at com.atlassian.servicedesk.plugins.cannedresponses.internal.lucene.index.CannedResponseIndexFactory.openSearcher(CannedResponseIndexFactory.java:239)
              at com.atlassian.servicedesk.plugins.cannedresponses.internal.lucene.index.CannedResponseIndexerImpl.lambda$getSearcher$9(CannedResponseIndexerImpl.java:187)
              at com.atlassian.servicedesk.plugins.cannedresponses.internal.lucene.search.CannedResponseManagedIndexSearcherFactory.executeIndexSearcher(CannedResponseManagedIndexSearcherFactory.java:40)
              at com.atlassian.servicedesk.plugins.cannedresponses.internal.lucene.search.CannedResponseSearcherImpl.search(CannedResponseSearcherImpl.java:59)
              at com.atlassian.servicedesk.plugins.cannedresponses.internal.index.CannedResponseIndexManagerImpl.lambda$search$3(CannedResponseIndexManagerImpl.java:204)
              at io.atlassian.fugue.Option.map(Option.java:259)
              at com.atlassian.servicedesk.plugins.cannedresponses.internal.index.CannedResponseIndexManagerImpl.search(CannedResponseIndexManagerImpl.java:204)
              at com.atlassian.servicedesk.plugins.cannedresponses.internal.search.CannedResponseSearchManager.searchCannedResponses(CannedResponseSearchManager.java:65)
              at com.atlassian.servicedesk.plugins.cannedresponses.internal.search.CannedResponseSearchService.lambda$searchCannedResponses$7(CannedResponseSearchService.java:129)
              at com.atlassian.pocketknife.step.EitherStep4.lambda$null$0(EitherStep4.java:29)
              at io.atlassian.fugue.Either$RightProjection.flatMap(Either.java:937)
              at io.atlassian.fugue.Either.flatMap(Either.java:231)
              at com.atlassian.pocketknife.step.EitherStep4.lambda$null$1(EitherStep4.java:29)
              at io.atlassian.fugue.Either$RightProjection.flatMap(Either.java:937)
              at io.atlassian.fugue.Either.flatMap(Either.java:231)
              at com.atlassian.pocketknife.step.EitherStep4.lambda$null$2(EitherStep4.java:28)
              at io.atlassian.fugue.Either$RightProjection.flatMap(Either.java:937)
              at io.atlassian.fugue.Either.flatMap(Either.java:231)
              at com.atlassian.pocketknife.step.EitherStep4.lambda$then$3(EitherStep4.java:27)
              at io.atlassian.fugue.Either$RightProjection.flatMap(Either.java:937)
              at io.atlassian.fugue.Either.flatMap(Either.java:231)
              at com.atlassian.pocketknife.step.EitherStep4.then(EitherStep4.java:26)
              at com.atlassian.servicedesk.plugins.cannedresponses.internal.search.CannedResponseSearchService.searchCannedResponses(CannedResponseSearchService.java:124)
              at com.atlassian.servicedesk.plugins.cannedresponses.internal.rest.resources.CannedResponseSearchResource.lambda$searchCannedResponses$1(CannedResponseSearchResource.java:76)
              at io.atlassian.fugue.Either$RightProjection.flatMap(Either.java:937)
              at io.atlassian.fugue.Either.flatMap(Either.java:231)
              at com.atlassian.pocketknife.step.EitherStep1.then(EitherStep1.java:18)
              at com.atlassian.servicedesk.plugins.cannedresponses.internal.rest.resources.CannedResponseSearchResource.searchCannedResponses(CannedResponseSearchResource.java:76)
              at sun.reflect.GeneratedMethodAccessor2278.invoke(Unknown Source)
      
      

      Workaround

      The workaround that has been working so far is the following:

      • Stop Jira application
      • Delete the file <JIRA_HOME>/caches/indexesV1/plugins/servicedeskcannedresponses/write.lock
      • Start Jira. Upon startup, the canned response index should be rebuilt.

            [JSDSERVER-6470] Canned response error when trying to define/select a canned response

            Same here. Upgraded last night from 8.0.2/4.0.2 to 8.4.2/4.4.2 and we are now getting this error:
             

            2019-10-31 09:58:56,415 http-nio-8080-exec-12 WARN username 598x32591x1 13jj6ut 192.168.xx.xx,192.168.xx.xx /rest/servicedesk/canned-responses/1/search/IT [c.a.s.p.c.i.lucene.search.CannedResponseManagedIndexSearcherFactory] Unexpected exception during execution of canned response index searcher
            com.atlassian.jira.util.RuntimeIOException: org.apache.lucene.store.LockObtainFailedException: Lock held by this virtual machine: /opt/atlassian/home/jira/caches/indexesV1/plugins/servicedeskcannedresponses/write.lock
            

            Roberto Osegueda added a comment - Same here. Upgraded last night from 8.0.2/4.0.2 to 8.4.2/4.4.2 and we are now getting this error:   2019-10-31 09:58:56,415 http-nio-8080-exec-12 WARN username 598x32591x1 13jj6ut 192.168.xx.xx,192.168.xx.xx / rest /servicedesk/canned-responses/1/search/IT [c.a.s.p.c.i.lucene.search.CannedResponseManagedIndexSearcherFactory] Unexpected exception during execution of canned response index searcher com.atlassian.jira.util.RuntimeIOException: org.apache.lucene.store.LockObtainFailedException: Lock held by this virtual machine: /opt/atlassian/home/jira/caches/indexesV1/plugins/servicedeskcannedresponses/write.lock

            On JIRA Service Desk 4.4.2 (installed in a test environment) we  we ​​now get the message "Error,
            We can't add canned response data to the search index right now. You could try again or contact your administrator.".

            We do not get this message on version JIRA Service Desk 4.0.2 (production)

             

            Marco Brundel added a comment - On JIRA Service Desk 4.4.2 (installed in a test environment) we  we ​​now get the message "Error, We can't add canned response data to the search index right now. You could try again or contact your administrator.". We do not get this message on version JIRA Service Desk 4.0.2 (production)  

            This issue is caused by Service Desk not properly closing a Lucene search index.

            The upgraded version of Lucene that has been shipping with Jira>=8 (JSD>=4) uses a new API where the close() call is now mandatory. Not closing the index will make all subsequent attempts to access the index fail.

            This issue is now fixed and will be included in the next JSD release.

            JSD versions <4 are not affected.

            Markus Reil (Inactive) added a comment - This issue is caused by Service Desk not properly closing a Lucene search index. The upgraded version of Lucene that has been shipping with Jira>=8 (JSD>=4) uses a new API where the close() call is now mandatory. Not closing the index will make all subsequent attempts to access the index fail. This issue is now fixed and will be included in the next JSD release. JSD versions <4 are not affected.

              mreil1 Markus Reil (Inactive)
              mreil1 Markus Reil (Inactive)
              Affected customers:
              2 This affects my team
              Watchers:
              11 Start watching this issue

                Created:
                Updated:
                Resolved: