It seems that Mercurial refuses to acknowledge Mercurial versions >= 5. Or there is some bug where it refuses to even try to run `hg` from the system path. I can sort of prove this based on the Process Log output on windows. The below are the process log calls when I open the options window and press the "System" button for mercurial. The last `hg --version` call doesn't run against the one on path, but the one it has in it's install folder.
This is an issue, as the version of Mercurial that ships with Sourcetree for Windows is 4.8.1, which is missing many newer features, and is not compatible with many repositories now.