Uploaded image for project: 'Jira Data Center'
  1. Jira Data Center
  2. JRASERVER-63584

JIRA encounters problems when database transactions are rolled back

    XMLWordPrintable

Details

    Description

      Summary

      Invalid bug: Exceptions from this event dispatcher cannot affect the transaction. The exception is handled and logged.

      Old Summary

      In certain conditions database transactions can be rolled back after committing changes to the Lucene index, for example:

      2016-11-21 10:47:10,665 http-nio-8080-exec-63 ERROR user session jsession ip,jirabaseurl/secure/CommentAssignIssue.jspa [c.a.event.internal.AsynchronousAbleEventDispatcher] There was an exception thrown trying to dispatch event [com.atlassian.jira.event.issue.IssueEvent@77b57e33[issue=issue-key,comment=<null>,worklog=<null>,changelog=[GenericEntity:ChangeGroup][issue,755181][author,user][created,2016-11-21 10:47:10.659][id,4687444],eventTypeId=2,sendMail=false,params={eventsource=action, baseurl=https://jirabaseurl},subtasksUpdated=true]] from the invoker [SingleParameterMethodListenerInvoker{method=public void com.atlassian.jira.event.listeners.mention.MentionEventListener.onIssueEvent(com.atlassian.jira.event.issue.IssueEvent), listener=com.atlassian.jira.event.listeners.mention.MentionEventListener@1a191982}]
      java.lang.RuntimeException: java.lang.NullPointerException
      	at com.atlassian.event.internal.SingleParameterMethodListenerInvoker.invoke(SingleParameterMethodListenerInvoker.java:50)
      	at com.atlassian.event.internal.AsynchronousAbleEventDispatcher$1$1.run(AsynchronousAbleEventDispatcher.java:48)
      	at com.google.common.util.concurrent.MoreExecutors$DirectExecutorService.execute(MoreExecutors.java:299)
      	at com.atlassian.event.internal.AsynchronousAbleEventDispatcher.dispatch(AsynchronousAbleEventDispatcher.java:107)
      	at com.atlassian.event.internal.EventPublisherImpl.invokeListeners(EventPublisherImpl.java:160)
      	at com.atlassian.event.internal.EventPublisherImpl.publish(EventPublisherImpl.java:79)
      	at com.atlassian.jira.event.issue.DefaultIssueEventManager.publishEvent(DefaultIssueEventManager.java:120)
      	at com.atlassian.jira.event.issue.DefaultIssueEventManager.publishAsRedundant(DefaultIssueEventManager.java:132)
      	at com.atlassian.jira.event.issue.DefaultIssueEventManager.dispatchRedundantEvent(DefaultIssueEventManager.java:103)
      	at com.atlassian.jira.issue.util.DefaultIssueUpdater.storeModifiedFields(DefaultIssueUpdater.java:95)
      	at com.atlassian.jira.issue.util.DefaultIssueUpdater.doUpdate(DefaultIssueUpdater.java:50)
      	at com.atlassian.jira.issue.managers.DefaultIssueManager.doUpdate(DefaultIssueManager.java:721)
      	at com.atlassian.jira.issue.managers.DefaultIssueManager.updateIssue(DefaultIssueManager.java:706)
      	at com.atlassian.jira.issue.managers.DefaultIssueManager.updateIssue(DefaultIssueManager.java:686)
      	at com.iamhuy.jira.plugin.workflow.AssignFromCustomfieldFunction.execute(AssignFromCustomfieldFunction.java:36)
      	at com.opensymphony.workflow.AbstractWorkflow.executeFunction(AbstractWorkflow.java:1050)
      	at com.opensymphony.workflow.AbstractWorkflow.transitionWorkflow(AbstractWorkflow.java:1446)
      	at com.opensymphony.workflow.AbstractWorkflow.doAction(AbstractWorkflow.java:564)
      	at com.atlassian.jira.workflow.OSWorkflowManager.doWorkflowActionInsideTxn(OSWorkflowManager.java:825)
      	at com.atlassian.jira.workflow.OSWorkflowManager.doWorkflowAction(OSWorkflowManager.java:786)
      	at com.atlassian.jira.bc.issue.DefaultIssueService.transition(DefaultIssueService.java:430)
      	at com.atlassian.jira.web.action.issue.CommentAssignIssue.doExecute(CommentAssignIssue.java:164)
      	... 1 filtered
      	at com.atlassian.jira.action.JiraActionSupport.execute(JiraActionSupport.java:63)
      	... 7 filtered
      	at javax.servlet.http.HttpServlet.service(HttpServlet.java:725)
      	... 50 filtered
      	at com.atlassian.greenhopper.jira.filters.ClassicBoardRouter.doFilter(ClassicBoardRouter.java:59)
      	... 50 filtered
      	at com.almworks.jira.structure.services.IssueLinksInterceptor.doHttpFilter(IssueLinksInterceptor.java:98)
      	at com.almworks.jira.structure.web.AbstractStructureServletFilter.doFilter(AbstractStructureServletFilter.java:34)
      	... 104 filtered
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
      	at java.lang.Thread.run(Thread.java:745)
      Caused by: java.lang.NullPointerException
      	at com.atlassian.jira.notification.DefaultNotificationSchemeManager.getRecipients(DefaultNotificationSchemeManager.java:335)
      	at com.atlassian.jira.notification.DefaultNotificationSchemeManager.getRecipients(DefaultNotificationSchemeManager.java:368)
      	at com.atlassian.jira.event.listeners.mention.MentionEventListener.onIssueEvent(MentionEventListener.java:39)
      	at sun.reflect.GeneratedMethodAccessor1752.invoke(Unknown Source)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at com.atlassian.event.internal.SingleParameterMethodListenerInvoker.invoke(SingleParameterMethodListenerInvoker.java:36)
      	... 242 more
      

      Early on in the call stack there is:
      at com.atlassian.jira.workflow.OSWorkflowManager.doWorkflowActionInsideTxn(OSWorkflowManager.java:825)

      In this particular case we'll have field values that will now be desynced and also indexes different on each node.

      Workaround

      In case of JIRA Datacenter, please use following workaround

      1. Run health-check (or empty JQL) and get status for each node
      2. Identify nodes(s) with maximin difference, these nodes need Lucene index to be copied from another node:
        1. Remove node from LB
        2. Copy index from another nodes to current node

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              jcurry Jeff Curry
              Votes:
              4 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: