Uploaded image for project: 'Jira Data Center'
  1. Jira Data Center
  2. JRASERVER-68724

Trigger Directory Syncs using the REST API

XMLWordPrintable

    • 13
    • 1
    • We collect Jira feedback from various sources, and we evaluate what we've collected when planning our product roadmap. To understand how this piece of feedback will be reviewed, see our Implementation of New Features Policy.

      It is common to have an LDAP directory that does not support incremental syncs and is moderately sized. These directories don't have the option to lower the sync interval to pick up group membership changes for known users, so admins currently have to trigger syncs from the UI if the sync interval is not an acceptable time to wait for the change to reach Jira. If the REST API had a means of triggering a sync remotely, it would give admins an easy (and scriptable) way to pick up changes to a directory.

      Workaround

      The Bash script below runs a directory sync of all enabled directories, be sure to update your credentials.
      The script and its contents are not supported by Atlassian and are only provided on a best-effort basis. Use at your own risk. 

      #!/bin/bash

set -x

# we need the base URL of the app (ie: https://jira.domain.com)
BASEURL=$1
[ -z "$BASEURL" ] && echo "usage: $0 BASEURL" && exit 1

# build the URL needed to grab the xtoken
URL="$BASEURL/plugins/servlet/embedded-crowd/directories/list"

# store the cookie as a temp file that is deleted as we finish
COOKIE=/tmp/GetRest.$$
RESP=/tmp/GetToken.$$

# login and save the cookie
curl -k -c $COOKIE -s -X POST \
 --data 'os_username=your_admin_username'  --data 'os_password=your_admin_password' --data "formname=loginform" --data "login='Log In'" \
 "$BASEURL/dologin.action" -o /dev/null

# get the list of sync opertaions by getting the URL with the cookie
curl -k -b $COOKIE -s -X GET "$URL" > $RESP

# here is an example of the sync operation
# href="/plugins/servlet/embedded-crowd/directories/sync?directoryId=10501&atl_token=d5265f974829752eaa45a94011452b1b05f3f730">

# look for all of the available sync operations and call each one-by-one
for i in `grep /sync $RESP | grep atl_token= | grep -v "{id}" | sed 's/^.*href="//; s/".$//'` ; do
        curl -k -b $COOKIE -s -X GET "$BASEURL$i"
done

# delete the temp files
rm -f $COOKIE $TOKEN

            [JRASERVER-68724] Trigger Directory Syncs using the REST API

            Vladislav Vaičiūnas added a comment -

            It works, but for some reasons it do only incremental sync, even if I remove "import static com.atlassian.crowd.manager.directory.SynchronisationMode.INCREMENTAL" line.

            I'm getting this in Jira logs:

            2022-02-17 09:02:53,181+0200 Caesium-1-2 INFO ServiceRunner     [c.a.crowd.directory.DbCachingRemoteDirectory] INCREMENTAL synchronisation for directory [ 10000 ] starting
            2022-02-17 09:02:53,184+0200 Caesium-1-2 INFO ServiceRunner     [c.a.crowd.directory.DbCachingRemoteDirectory] Attempting INCREMENTAL synchronisation for directory [ 10000 ]
            2022-02-17 09:02:53,198+0200 Caesium-1-2 INFO ServiceRunner     [c.a.crowd.directory.DbCachingRemoteDirectory] INCREMENTAL synchronisation complete for directory [ 10000 ] in [ 17ms ]

            Vladislav Vaičiūnas added a comment - It works, but for some reasons it do only incremental sync, even if I remove "import static com.atlassian.crowd.manager.directory.SynchronisationMode.INCREMENTAL" line. I'm getting this in Jira logs: 2022-02-17 09:02:53,181+0200 Caesium-1-2 INFO ServiceRunner     [c.a.crowd.directory.DbCachingRemoteDirectory] INCREMENTAL synchronisation for directory [ 10000 ] starting 2022-02-17 09:02:53,184+0200 Caesium-1-2 INFO ServiceRunner     [c.a.crowd.directory.DbCachingRemoteDirectory] Attempting INCREMENTAL synchronisation for directory [ 10000 ] 2022-02-17 09:02:53,198+0200 Caesium-1-2 INFO ServiceRunner     [c.a.crowd.directory.DbCachingRemoteDirectory] INCREMENTAL synchronisation complete for directory [ 10000 ] in [ 17ms ]

            Wu Hao added a comment -

            Hello Vladislav Vaičiūnas,

            If you want to use incremental, you can modify "FULL" to "INCREMENTAL".

            It is groovy script which could be put in scriptrunner, or somewhere.

            Hugo Wu

            Wu Hao added a comment - Hello Vladislav Vaičiūnas, If you want to use incremental, you can modify "FULL" to "INCREMENTAL". It is groovy script which could be put in scriptrunner, or somewhere. Hugo Wu

            Vladislav Vaičiūnas added a comment -

            Hello Wu,

            This script allows to force FULL, not incremental sync, am I right?

            And what is the language and how it should be run?

            Vladislav Vaičiūnas added a comment - Hello Wu, This script allows to force FULL, not incremental sync, am I right? And what is the language and how it should be run?

            Wu Hao added a comment - 
            import com.atlassian.crowd.manager.directory.DirectoryManager
import com.atlassian.jira.component.ComponentAccessor
import static com.atlassian.crowd.manager.directory.SynchronisationMode.FULL
import static com.atlassian.crowd.manager.directory.SynchronisationMode.INCREMENTAL

def directoryManager = ComponentAccessor.getComponent(DirectoryManager)

def did= directoryManager.findDirectoryByName("xxx").getId()

if (directoryManager.isSynchronisable(did) && !directoryManager.isSynchronising(did))
{
    directoryManager.synchroniseCache(did, FULL)
}
else
{
    log.error ("Could not start sync xxx")
}

            Wu Hao added a comment - import com.atlassian.crowd.manager.directory.DirectoryManager import com.atlassian.jira.component.ComponentAccessor import static com.atlassian.crowd.manager.directory.SynchronisationMode.FULL import static com.atlassian.crowd.manager.directory.SynchronisationMode.INCREMENTAL def directoryManager = ComponentAccessor.getComponent(DirectoryManager) def did= directoryManager.findDirectoryByName( "xxx" ).getId() if (directoryManager.isSynchronisable(did) && !directoryManager.isSynchronising(did)) { directoryManager.synchroniseCache(did, FULL) } else { log.error ( "Could not start sync xxx" ) }

            Andreas Hochsteger added a comment - - edited

            This is an improved version that is tested with Jira 8.5.3 and contains the following changes and fixes:

            • Username and password is parameterized using environment variables
            • Username and password parameters are url-encoded in curl commands (fixes problems with special characters)
            • Fixed to cleanup of /tmp/GetToken.* files
            • Disable "set -x" to prevent output of clear text admin passwords

            Example Usage:

            USERNAME=myusername PASSWORD=mypassword ./jira-directory-sync.sh https://jira.domain.com

            Contents of file jira-directory-sync.sh:

            #!/bin/bash

# Source: https://jira.atlassian.com/browse/JRASERVER-68724

#set -x

# USERNAME and PASSWORD are expected to be set as environment variables.
# The base URL of the app is given on the command line (ie: https://jira.domain.com)
BASEURL=$1
[ -z "$BASEURL" ] && echo "usage: USERNAME=myusername PASSWORD=mypassword $0 BASEURL" && exit 1

# Required URLs and paths:
WEB_START_URL="${BASEURL}/login.jsp"
WEB_LOGIN_URL="${BASEURL}/login.jsp"
WEB_SUDO_URL="${BASEURL}/secure/admin/WebSudoAuthenticate.jspa"
WEB_DIRLIST_PATH="/plugins/servlet/embedded-crowd/directories/list"

# store the cookie as a temp file that is deleted as we finish
COOKIE=/tmp/GetRest.$$
RESP=/tmp/GetToken.$$

# Render the login page to get some cookies
curl -k -c $COOKIE -s "${WEB_START_URL}" -o /dev/null

# login and save the cookie
curl -k -b $COOKIE -c $COOKIE -s -X POST \
 --data-urlencode "os_username=${USERNAME}" \
 --data-urlencode "os_password=${PASSWORD}" \
 --data "login='Log In'" \
 --data "os_destination=${WEB_DIRLIST_PATH}" \
 --data "user_role=SYSADMIN" \
 --data "atl_token=" \
 "${WEB_LOGIN_URL}" -o /dev/null

# Do web sudo
curl -k -b $COOKIE -c $COOKIE -s -X POST \
 --data-urlencode "webSudoPassword=${PASSWORD}" \
 --data "webSudoDestination=${WEB_DIRLIST_PATH}" \
 --data 'webSudoIsPost=false' \
 "${WEB_SUDO_URL}" -o /dev/null

# get the list of sync opertaions by getting the URL with the cookie
curl -k -b $COOKIE -c $COOKIE -s -X GET "${BASEURL}${WEB_DIRLIST_PATH}" > $RESP

# look for all of the available sync operations and call each one-by-one
for sync_path in $(grep /sync $RESP | grep atl_token= | grep -v "{id}" | sed 's/^.*href="//; s/".$//') ; do
        curl -k -b $COOKIE -s -X GET "${BASEURL}${sync_path}"
done

# delete the temp files
rm -f $COOKIE $RESP

            To trigger the directory sync for Confluence Server and Bitbucket Server have a look similar scripts in the comments of these issues:

            Andreas Hochsteger added a comment - - edited This is an improved version that is tested with Jira 8.5.3 and contains the following changes and fixes: Username and password is parameterized using environment variables Username and password parameters are url-encoded in curl commands (fixes problems with special characters) Fixed to cleanup of /tmp/GetToken.* files Disable "set -x" to prevent output of clear text admin passwords Example Usage: USERNAME=myusername PASSWORD=mypassword ./jira-directory-sync.sh https://jira.domain.com Contents of file jira-directory-sync.sh: #!/bin/bash # Source: https://jira.atlassian.com/browse/JRASERVER-68724 #set -x # USERNAME and PASSWORD are expected to be set as environment variables. # The base URL of the app is given on the command line (ie: https://jira.domain.com) BASEURL= $1 [ -z " $BASEURL " ] && echo "usage: USERNAME=myusername PASSWORD=mypassword $0 BASEURL" && exit 1 # Required URLs and paths: WEB_START_URL= "${BASEURL}/login.jsp" WEB_LOGIN_URL= "${BASEURL}/login.jsp" WEB_SUDO_URL= "${BASEURL}/secure/admin/WebSudoAuthenticate.jspa" WEB_DIRLIST_PATH= "/plugins/servlet/embedded-crowd/directories/list" # store the cookie as a temp file that is deleted as we finish COOKIE=/tmp/GetRest.$$ RESP=/tmp/GetToken.$$ # Render the login page to get some cookies curl -k -c $COOKIE -s "${WEB_START_URL}" -o /dev/null # login and save the cookie curl -k -b $COOKIE -c $COOKIE -s -X POST \ --data-urlencode "os_username=${USERNAME}" \ --data-urlencode "os_password=${PASSWORD}" \ --data "login= 'Log In' " \ --data "os_destination=${WEB_DIRLIST_PATH}" \ --data "user_role=SYSADMIN" \ --data "atl_token=" \ "${WEB_LOGIN_URL}" -o /dev/null # Do web sudo curl -k -b $COOKIE -c $COOKIE -s -X POST \ --data-urlencode "webSudoPassword=${PASSWORD}" \ --data "webSudoDestination=${WEB_DIRLIST_PATH}" \ --data 'webSudoIsPost=false' \ "${WEB_SUDO_URL}" -o /dev/null # get the list of sync opertaions by getting the URL with the cookie curl -k -b $COOKIE -c $COOKIE -s -X GET "${BASEURL}${WEB_DIRLIST_PATH}" > $RESP # look for all of the available sync operations and call each one-by-one for sync_path in $(grep /sync $RESP | grep atl_token= | grep -v "{id}" | sed 's/^.*href= "//; s/" .$//' ) ; do curl -k -b $COOKIE -s -X GET "${BASEURL}${sync_path}" done # delete the temp files rm -f $COOKIE $RESP To trigger the directory sync for Confluence Server and Bitbucket Server have a look similar scripts in the comments of these issues: https://jira.atlassian.com/browse/CONFSERVER-26737?focusedCommentId=2395933&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-2395933 https://jira.atlassian.com/browse/BSERV-5533?focusedCommentId=2395934&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-2395934

            Joel Pearson added a comment -

            Here is a version that works with Jira 8.7.1 with web sudo enabled:

             

            #!/bin/bash

set -x

# we need the base URL of the app (ie: https://jira.domain.com)
BASEURL=$1
[ -z "$BASEURL" ] && echo "usage: $0 BASEURL" && exit 1

# build the URL needed to grab the xtoken
URL="$BASEURL/plugins/servlet/embedded-crowd/directories/list"

# Web sudo URL
WEB_SUDO="${BASEURL}/secure/admin/WebSudoAuthenticate.jspa"

# store the cookie as a temp file that is deleted as we finish
COOKIE=/tmp/GetRest.$$
RESP=/tmp/GetToken.$$

# Render the login page to get some cookies
curl -k -c $COOKIE -s "$BASEURL/login.jsp" -o /dev/null

# login and save the cookie
curl -k -b $COOKIE -c $COOKIE -s -X POST \
 --data 'os_username=your_admin_username'  --data 'os_password=your_admin_password' --data "login='Log In'" \
 --data "os_destination=/plugins/servlet/embedded-crowd/directories/list" --data "user_role=SYSADMIN" --data "atl_token=" \
 "$BASEURL/login.jsp" -o /dev/null

# Do web sudo
curl -k -b $COOKIE -c $COOKIE -s -X POST \
 --data 'webSudoPassword=your_admin_password' --data 'webSudoDestination=/plugins/servlet/embedded-crowd/directories/list' 
 --data 'webSudoIsPost=false' "$WEB_SUDO" -o /dev/null

# get the list of sync opertaions by getting the URL with the cookie
curl -k -b $COOKIE -c $COOKIE -s -X GET "$URL" > $RESP

# here is an example of the sync operation
# href="/plugins/servlet/embedded-crowd/directories/sync?directoryId=10501&atl_token=d5265f974829752eaa45a94011452b1b05f3f730">

# look for all of the available sync operations and call each one-by-one
for i in `grep /sync $RESP | grep atl_token= | grep -v "{id}" | sed 's/^.*href="//; s/".$//'` ; do
        curl -k -b $COOKIE -s -X GET "$BASEURL$i"
done

# delete the temp files
rm -f $COOKIE $TOKEN

            Joel Pearson added a comment - Here is a version that works with Jira 8.7.1 with web sudo enabled:   #!/bin/bash set -x # we need the base URL of the app (ie: https://jira.domain.com) BASEURL= $1 [ -z " $BASEURL " ] && echo "usage: $0 BASEURL" && exit 1 # build the URL needed to grab the xtoken URL= " $BASEURL /plugins/servlet/embedded-crowd/directories/list" # Web sudo URL WEB_SUDO= "${BASEURL}/secure/admin/WebSudoAuthenticate.jspa" # store the cookie as a temp file that is deleted as we finish COOKIE=/tmp/GetRest.$$ RESP=/tmp/GetToken.$$ # Render the login page to get some cookies curl -k -c $COOKIE -s " $BASEURL /login.jsp" -o /dev/null # login and save the cookie curl -k -b $COOKIE -c $COOKIE -s -X POST \ --data 'os_username=your_admin_username' --data 'os_password=your_admin_password' --data "login= 'Log In' " \ --data "os_destination=/plugins/servlet/embedded-crowd/directories/list" --data "user_role=SYSADMIN" --data "atl_token=" \ " $BASEURL /login.jsp" -o /dev/null # Do web sudo curl -k -b $COOKIE -c $COOKIE -s -X POST \ --data 'webSudoPassword=your_admin_password' --data 'webSudoDestination=/plugins/servlet/embedded-crowd/directories/list' --data 'webSudoIsPost=false' "$WEB_SUDO" -o /dev/null # get the list of sync opertaions by getting the URL with the cookie curl -k -b $COOKIE -c $COOKIE -s -X GET " $URL " > $RESP # here is an example of the sync operation # href= "/plugins/servlet/embedded-crowd/directories/sync?directoryId=10501&atl_token=d5265f974829752eaa45a94011452b1b05f3f730" > # look for all of the available sync operations and call each one-by-one for i in `grep /sync $RESP | grep atl_token= | grep -v "{id}" | sed 's/^.*href= "//; s/" .$//' ` ; do curl -k -b $COOKIE -s -X GET " $BASEURL $i " done # delete the temp files rm -f $COOKIE $TOKEN

            Jeffrey Goh added a comment -

            The grep and sed command simply do not return any value to me at all.

            Jeffrey Goh added a comment - The grep and sed command simply do not return any value to me at all.

              Unassigned Unassigned
              smitra2@atlassian.com Suddha
              Votes:
              56 Vote for this issue
              Watchers:
              42 Start watching this issue

                Created:
                Updated: