Details
-
Bug
-
Resolution: Fixed
-
High
-
6.2.5, 6.3.4, 6.3.14, 6.4.6
-
PostgreSQL (likely other DBs that support isolation)
-
6.02
-
Description
Problem description:
On an active JIRA, concurrent read access to links (such as when an issue is being displayed) and write access to links (such as when a new link or sub-task is created or deleted) may result in errors:
- when a link/subtask is created, and this problem happens, the link/subtask will not be displayed at the parent issue (even after creation succeeds)
- when a subtask (or linked issue) is deleted, the parent task “breaks”, the issue page displays an error and there’s a NullPointerException in the server log.
This is a race condition, so exact reproduction sequence is hard to define. However, I managed to reproduce the problem with 100% certainty using debugger and running JIRA on Postres.
Investigation:
DefaultIssueLinkManager has a global per-instance cache of the inward/outward links. It is not transactional, and shared between threads that run transactional changes on the database, and threads that do reading. As a result, the following may happen:
- Thread 1: change transaction starts
- Thread 1: DefaultIssueLinkManager performs a write operation on a link in the DB
- Thread 1: cleans the cache for affected issues
- Thread 2: requests getInwardLinks or getOutwardLinks from DefaultIssueLinkManager for the affected issue
- Thread 2: because Thread 1’s transaction is not committed yet, old state is read and saved in the cache
- Thread 1: commits the transaction
- Thread 1 or any other thread: calls getInwardLinks or getOutwardLinks - but gets corrupt cached value, which relates to the state before the transaction has committed.
How to reproduce using debugger:
- Start JIRA instance with Postgres - I couldn’t reproduce on HSQL
- Create issue X and sub-task S
- Prepare a class with a piece of code that would a) start a new thread, b) in the thread, request getOutwardLinks() for a specified issue ID, c) wait for the thread to finish.
- Place a breakpoint in DefaultIssueDeleteHelper.deleteIssue(), on the line removeIssueLinks(user, issue);
- In JIRA, delete sub-task S — catch the breakpoint in debugger.
- Step over “removeIssueLinks”. Inside, the links will be removed, and cache will be cleared several times.
- Before continuing, use debugger’s Evaluate feature to call the prepared method that retrieves outward links in another thread - call it on the parent issue X.
- Then you can verify that IssueLinkManager’s cache will contain an invalid entry for issue X with the link that has just been deleted.
- In JIRA web interface, if you open the parent issue’s page, there will be errors, and there will be an NPE in the log.
Note that this is only an example - the same problem can happen in most any place where links cache is being invalidated.
The NPE:
[INFO] [talledLocalContainer] 2014-11-19 03:50:24,925 http-bio-2990-exec-19 ERROR admin 230x15313x1 890nfl fe80:0:0:0:cae0:ebff:fe18:7e59%4 /secure/AjaxIssueAction!default.jspa [atlassian.plugin.web.DefaultWebInterfaceManager] Could not evaluate condition 'com.atlassian.plugin.web.conditions.AndCompositeCondition@477a8ce9' for descriptor: com.atlassian.jira.jira-view-issue-plugin:view-subtasks (null) [INFO] [talledLocalContainer] java.lang.NullPointerException [INFO] [talledLocalContainer] at com.atlassian.jira.security.DefaultPermissionManager.doIssuePermissionCheck(DefaultPermissionManager.java:283) [INFO] [talledLocalContainer] at com.atlassian.jira.security.DefaultPermissionManager.hasPermission(DefaultPermissionManager.java:195) [INFO] [talledLocalContainer] at com.atlassian.jira.security.WorkflowBasedPermissionManager.hasPermission(WorkflowBasedPermissionManager.java:97) [INFO] [talledLocalContainer] at com.atlassian.jira.security.DefaultPermissionManager.hasPermission(DefaultPermissionManager.java:190) [INFO] [talledLocalContainer] at com.atlassian.jira.security.WorkflowBasedPermissionManager.hasPermission(WorkflowBasedPermissionManager.java:90) [INFO] [talledLocalContainer] at sun.reflect.GeneratedMethodAccessor542.invoke(Unknown Source) [INFO] [talledLocalContainer] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [INFO] [talledLocalContainer] at java.lang.reflect.Method.invoke(Method.java:606) [INFO] [talledLocalContainer] at com.atlassian.util.profiling.object.ObjectProfiler.profiledInvoke(ObjectProfiler.java:83) [INFO] [talledLocalContainer] at com.atlassian.jira.config.component.SwitchingInvocationHandler.invoke(SwitchingInvocationHandler.java:28) [INFO] [talledLocalContainer] at com.sun.proxy.$Proxy9.hasPermission(Unknown Source) [INFO] [talledLocalContainer] at com.atlassian.jira.config.DefaultSubTaskManager.getSubTaskBean(DefaultSubTaskManager.java:391) [INFO] [talledLocalContainer] at sun.reflect.GeneratedMethodAccessor689.invoke(Unknown Source) [INFO] [talledLocalContainer] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [INFO] [talledLocalContainer] at java.lang.reflect.Method.invoke(Method.java:606) [INFO] [talledLocalContainer] at com.atlassian.plugin.osgi.hostcomponents.impl.DefaultComponentRegistrar$ContextClassLoaderSettingInvocationHandler.invoke(DefaultComponentRegistrar.java:129) [INFO] [talledLocalContainer] at com.sun.proxy.$Proxy63.getSubTaskBean(Unknown Source) [INFO] [talledLocalContainer] at sun.reflect.GeneratedMethodAccessor689.invoke(Unknown Source) [INFO] [talledLocalContainer] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [INFO] [talledLocalContainer] at java.lang.reflect.Method.invoke(Method.java:606) [INFO] [talledLocalContainer] at com.atlassian.plugin.osgi.bridge.external.HostComponentFactoryBean$DynamicServiceInvocationHandler.invoke(HostComponentFactoryBean.java:154) [INFO] [talledLocalContainer] at com.sun.proxy.$Proxy63.getSubTaskBean(Unknown Source) [INFO] [talledLocalContainer] at com.atlassian.jira.plugin.viewissue.HasSubTaskCondition.getSubTaskBean(HasSubTaskCondition.java:57) [INFO] [talledLocalContainer] at com.atlassian.jira.plugin.viewissue.HasSubTaskCondition.shouldDisplay(HasSubTaskCondition.java:41) [INFO] [talledLocalContainer] at com.atlassian.plugin.web.conditions.OrCompositeCondition.shouldDisplay(OrCompositeCondition.java:14) [INFO] [talledLocalContainer] at com.atlassian.plugin.web.conditions.AndCompositeCondition.shouldDisplay(AndCompositeCondition.java:14) [INFO] [talledLocalContainer] at com.atlassian.plugin.web.conditions.AndCompositeCondition.shouldDisplay(AndCompositeCondition.java:14) [INFO] [talledLocalContainer] at com.atlassian.plugin.web.DefaultWebInterfaceManager.filterFragmentsByCondition(DefaultWebInterfaceManager.java:153) [INFO] [talledLocalContainer] at com.atlassian.plugin.web.DefaultWebInterfaceManager.getDisplayableWebPanelDescriptors(DefaultWebInterfaceManager.java:116)
Workaround:
If you are seeing this error on issues, please refer to https://confluence.atlassian.com/x/mhDtKQ.
Attachments
Issue Links
- causes
-
JSEV-670 Loading...
- is duplicated by
-
- mentioned in
-
- was cloned as
-