Uploaded image for project: 'Jira Data Center'
  1. Jira Data Center
  2. JRASERVER-62500

Multiple concurrent login attempts may throw a 500 error/UserNotFoundException due to a race condition

    XMLWordPrintable

Details

    Description

      Summary

      Multiple concurrent login attempts may throw a UserNotFoundException due to a race condition. User will get 500 error as a response.
      JIRA doesn't expect to have high concurrency for authentication requests. So initially problem is caused by very extensive authentications and better way to handle that is to reuse user session. See docs: jira-rest-api-example-cookie-based-authentication

      Steps to Reproduce

      1. Run a script which performs multiple REST login attempts every second

      Expected Results

      All login attempts will be successful.

      Actual Results

      The login attempt will occasionally fail due to race condition.
      The below exception is thrown in the atlassian-jira.log file:

      2016-06-06 08:21:47,545 http-bio-443-exec-349 ERROR      [o.a.c.c.C.[.[localhost].[/].[default]] Servlet.service() for servlet [default] in context with path [] threw exception
      com.atlassian.crowd.exception.runtime.UserNotFoundException: User <username> does not exist
      	at com.atlassian.crowd.embedded.core.CrowdServiceImpl.setUserAttribute(CrowdServiceImpl.java:413)
      	at com.atlassian.crowd.embedded.core.CrowdServiceImpl.setUserAttribute(CrowdServiceImpl.java:392)
      	at com.atlassian.crowd.embedded.core.DelegatingCrowdService.setUserAttribute(DelegatingCrowdService.java:121)
      	at com.atlassian.crowd.embedded.core.FilteredCrowdServiceImpl.setUserAttribute(FilteredCrowdServiceImpl.java:51)
      	at com.atlassian.jira.security.login.LoginStoreImpl.setLong(LoginStoreImpl.java:113)
      	at com.atlassian.jira.security.login.LoginStoreImpl.recordLoginAttempt(LoginStoreImpl.java:47)
      	at com.atlassian.jira.security.login.RecoveryLoginStore.recordLoginAttempt(RecoveryLoginStore.java:63)
      	at com.atlassian.jira.security.login.LoginManagerImpl.recordLoginAttempt(LoginManagerImpl.java:314)
      	at com.atlassian.jira.security.login.LoginManagerImpl.onLoginAttempt(LoginManagerImpl.java:215)
      	at com.atlassian.jira.security.login.JiraElevatedSecurityGuard.onSuccessfulLoginAttempt(JiraElevatedSecurityGuard.java:30)
      	... 13 filtered
      	at com.atlassian.plugins.rest.module.servlet.RestSeraphFilter.doFilter(RestSeraphFilter.java:37)
      	... 73 filtered
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
      	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
      	at java.lang.Thread.run(Unknown Source)
      

      Notes

      • Subsequent attempt at login will succeed
      • Doesn't seem to affect Internal Directory
      • Problem relates to new code introduced in 7.0

      Workaround

      Add the following JIRA property into <JIRA_HOME>/jira-config.properties (create files if it's empty):

      # JRA-62500 disable full user cache
      jira.fullUserCache=false
      
      • JIRA will need to be restarted for this change to take affect.

      That will switch to lazy user cache which doesn't have this race condition.
      That being said, enabling this will heavily impact REST performance and scenarios when JIRA is used as crowd server.

      1. Every call for userCount will hit database instead of cache.
      2. Another thing is that users will not be loaded to cache at once. It may slow down some queries before it's fully loaded. (They will be loaded when needed)

      Attachments

        1. image-2017-04-11-15-12-37-458.png
          28 kB
        2. NPE.log
          21 kB

        Issue Links

          Activity

            People

              izinoviev Ilya Zinoviev (Inactive)
              ywoo Yit Wei
              Votes:
              32 Vote for this issue
              Watchers:
              60 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: