Git hooks are unexpectedly deleted on NFS, resulting in hooks not being executed

Issue Summary

When Bitbucket Data Center/Server 7.14+ starts, it unpacks Git hooks (pre-receive, post-receive, etc.). After hooks are unpacked, the system does a check for stale files that might be left over in the directory and prunes them.

When the shared home is on NFS, it's possible for hook files to become visible before their attributes, like their modification timestamp, have been written. When that happens, it results in the system deleting the hooks it just unpacked. Since the hooks are removed, git processes like git http-backend and git receive-pack that are supposed to trigger hooks don't, leading to a range of knock-on issues.

Note: This behavior has not been observed on non-NFS filesystems, where file and attribute creation are atomic. Additionally, earlier versions of Bitbucket Data Center/Server did not unpack hooks the same way, or to the same location, and are not susceptible to this issue.

Steps to Reproduce

1. Start Bitbucket Data Center/Server 7.14+
2. After startup completes, check $BITBUCKET_HOME/shared/config/git/hooks and observe it is empty

Expected Results

Some number of hook scripts, like pre-receive and post-receive, are present in $BITBUCKET_HOME/shared/config/git/hooks.

Actual Results

$BITBUCKET_HOME/shared/config/git/hooks is empty, and hooks are not run. There are no exceptions or other details logged, even with debug logging enabled.

Workaround

Two workarounds that have been confirmed to work are:

• Manually writing the hooks to disk after each restart
• Mounting local disk directory at $BITBUCKET_HOME/shared/config/git/hooks to avoid NFS raciness