Uploaded image for project: 'Jira Server and Data Center'
  1. Jira Server and Data Center
  2. JRASERVER-40942

When issue link is being created or deleted in a transaction, concurrent access to IssueLinkManager may lead to corrupt data in the links cache

    XMLWordPrintable

    Details

    • Introduced in Version:
      6.02
    • Regular Expression:
      com.atlassian.jira.security.DefaultPermissionManager.doIssuePermissionCheck\(DefaultPermissionManager.java:\d+
    • Occurrence Factor:
      50%
    • Development Effort:
      L

      Description

      Problem description:

      On an active JIRA, concurrent read access to links (such as when an issue is being displayed) and write access to links (such as when a new link or sub-task is created or deleted) may result in errors:

      • when a link/subtask is created, and this problem happens, the link/subtask will not be displayed at the parent issue (even after creation succeeds)
      • when a subtask (or linked issue) is deleted, the parent task “breaks”, the issue page displays an error and there’s a NullPointerException in the server log.

      This is a race condition, so exact reproduction sequence is hard to define. However, I managed to reproduce the problem with 100% certainty using debugger and running JIRA on Postres.

      Investigation:

      DefaultIssueLinkManager has a global per-instance cache of the inward/outward links. It is not transactional, and shared between threads that run transactional changes on the database, and threads that do reading. As a result, the following may happen:

      1. Thread 1: change transaction starts
      2. Thread 1: DefaultIssueLinkManager performs a write operation on a link in the DB
      3. Thread 1: cleans the cache for affected issues
      4. Thread 2: requests getInwardLinks or getOutwardLinks from DefaultIssueLinkManager for the affected issue
      5. Thread 2: because Thread 1’s transaction is not committed yet, old state is read and saved in the cache
      6. Thread 1: commits the transaction
      7. Thread 1 or any other thread: calls getInwardLinks or getOutwardLinks - but gets corrupt cached value, which relates to the state before the transaction has committed.

      How to reproduce using debugger:

      1. Start JIRA instance with Postgres - I couldn’t reproduce on HSQL
      2. Create issue X and sub-task S
      3. Prepare a class with a piece of code that would a) start a new thread, b) in the thread, request getOutwardLinks() for a specified issue ID, c) wait for the thread to finish.
      4. Place a breakpoint in DefaultIssueDeleteHelper.deleteIssue(), on the line removeIssueLinks(user, issue);
      5. In JIRA, delete sub-task S — catch the breakpoint in debugger.
      6. Step over “removeIssueLinks”. Inside, the links will be removed, and cache will be cleared several times.
      7. Before continuing, use debugger’s Evaluate feature to call the prepared method that retrieves outward links in another thread - call it on the parent issue X.
      8. Then you can verify that IssueLinkManager’s cache will contain an invalid entry for issue X with the link that has just been deleted.
      9. In JIRA web interface, if you open the parent issue’s page, there will be errors, and there will be an NPE in the log.

      Note that this is only an example - the same problem can happen in most any place where links cache is being invalidated.

      The NPE:

      [INFO] [talledLocalContainer] 2014-11-19 03:50:24,925 http-bio-2990-exec-19 ERROR admin 230x15313x1 890nfl fe80:0:0:0:cae0:ebff:fe18:7e59%4 /secure/AjaxIssueAction!default.jspa [atlassian.plugin.web.DefaultWebInterfaceManager] Could not evaluate condition 'com.atlassian.plugin.web.conditions.AndCompositeCondition@477a8ce9' for descriptor: com.atlassian.jira.jira-view-issue-plugin:view-subtasks (null)
      [INFO] [talledLocalContainer] java.lang.NullPointerException
      [INFO] [talledLocalContainer] 	at com.atlassian.jira.security.DefaultPermissionManager.doIssuePermissionCheck(DefaultPermissionManager.java:283)
      [INFO] [talledLocalContainer] 	at com.atlassian.jira.security.DefaultPermissionManager.hasPermission(DefaultPermissionManager.java:195)
      [INFO] [talledLocalContainer] 	at com.atlassian.jira.security.WorkflowBasedPermissionManager.hasPermission(WorkflowBasedPermissionManager.java:97)
      [INFO] [talledLocalContainer] 	at com.atlassian.jira.security.DefaultPermissionManager.hasPermission(DefaultPermissionManager.java:190)
      [INFO] [talledLocalContainer] 	at com.atlassian.jira.security.WorkflowBasedPermissionManager.hasPermission(WorkflowBasedPermissionManager.java:90)
      [INFO] [talledLocalContainer] 	at sun.reflect.GeneratedMethodAccessor542.invoke(Unknown Source)
      [INFO] [talledLocalContainer] 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      [INFO] [talledLocalContainer] 	at java.lang.reflect.Method.invoke(Method.java:606)
      [INFO] [talledLocalContainer] 	at com.atlassian.util.profiling.object.ObjectProfiler.profiledInvoke(ObjectProfiler.java:83)
      [INFO] [talledLocalContainer] 	at com.atlassian.jira.config.component.SwitchingInvocationHandler.invoke(SwitchingInvocationHandler.java:28)
      [INFO] [talledLocalContainer] 	at com.sun.proxy.$Proxy9.hasPermission(Unknown Source)
      [INFO] [talledLocalContainer] 	at com.atlassian.jira.config.DefaultSubTaskManager.getSubTaskBean(DefaultSubTaskManager.java:391)
      [INFO] [talledLocalContainer] 	at sun.reflect.GeneratedMethodAccessor689.invoke(Unknown Source)
      [INFO] [talledLocalContainer] 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      [INFO] [talledLocalContainer] 	at java.lang.reflect.Method.invoke(Method.java:606)
      [INFO] [talledLocalContainer] 	at com.atlassian.plugin.osgi.hostcomponents.impl.DefaultComponentRegistrar$ContextClassLoaderSettingInvocationHandler.invoke(DefaultComponentRegistrar.java:129)
      [INFO] [talledLocalContainer] 	at com.sun.proxy.$Proxy63.getSubTaskBean(Unknown Source)
      [INFO] [talledLocalContainer] 	at sun.reflect.GeneratedMethodAccessor689.invoke(Unknown Source)
      [INFO] [talledLocalContainer] 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      [INFO] [talledLocalContainer] 	at java.lang.reflect.Method.invoke(Method.java:606)
      [INFO] [talledLocalContainer] 	at com.atlassian.plugin.osgi.bridge.external.HostComponentFactoryBean$DynamicServiceInvocationHandler.invoke(HostComponentFactoryBean.java:154)
      [INFO] [talledLocalContainer] 	at com.sun.proxy.$Proxy63.getSubTaskBean(Unknown Source)
      [INFO] [talledLocalContainer] 	at com.atlassian.jira.plugin.viewissue.HasSubTaskCondition.getSubTaskBean(HasSubTaskCondition.java:57)
      [INFO] [talledLocalContainer] 	at com.atlassian.jira.plugin.viewissue.HasSubTaskCondition.shouldDisplay(HasSubTaskCondition.java:41)
      [INFO] [talledLocalContainer] 	at com.atlassian.plugin.web.conditions.OrCompositeCondition.shouldDisplay(OrCompositeCondition.java:14)
      [INFO] [talledLocalContainer] 	at com.atlassian.plugin.web.conditions.AndCompositeCondition.shouldDisplay(AndCompositeCondition.java:14)
      [INFO] [talledLocalContainer] 	at com.atlassian.plugin.web.conditions.AndCompositeCondition.shouldDisplay(AndCompositeCondition.java:14)
      [INFO] [talledLocalContainer] 	at com.atlassian.plugin.web.DefaultWebInterfaceManager.filterFragmentsByCondition(DefaultWebInterfaceManager.java:153)
      [INFO] [talledLocalContainer] 	at com.atlassian.plugin.web.DefaultWebInterfaceManager.getDisplayableWebPanelDescriptors(DefaultWebInterfaceManager.java:116)
      

      Workaround:
      If you are seeing this error on issues, please refer to https://confluence.atlassian.com/x/mhDtKQ.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                ohernandez@atlassian.com Oswaldo Hernandez (Inactive)
                Reporter:
                sereda Igor Sereda [ALM Works]
              • Votes:
                35 Vote for this issue
                Watchers:
                52 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: