I put in a support ticket and received a workaround that works IF you are using Bitbucket with Bamboo. I don't know whether this works with any other Git server. The workaround is to use only relative SSH URLs in your .gitmodules file. This only works if your main repository is on the same server as the submodule repositories you are trying to authenticate against. If you have external submodules, you'll have to clone them into the same Bitbucket instance as your main repository in order to use this workaround.

Also, note that you still need to manually copy the public SSH key from the "Advanced" tab of the main Repository configuration, and add it as a "Read" Access Key in the repository permissions of each of the submodules in Bitbucket.

[submodule "submodule"]
	path = submodules/submodule
	url = ../submodule.git
[submodule "importing"]
	path = submodules/importing
	url = ../../e2/importing.git

I can't make it work using YAML specs.

I set id_rsa ssh key with chmod 600 on the agent.

I also configured the .ssh/config to use this key.

Please, fix this bug.

wow this issue open more than my total years of experience 

That's nonsense. It's a bug. Please fix it.

@jcbsv Thank you for your help, it work!

But i have to make a minor modification in order to make it work - replaced > to />

@echo off

if exist "%USERPROFILE%" (
if not exist "%USERPROFILE%\.ssh" mkdir "%USERPROFILE%\.ssh"
)

call :data1  /> "%USERPROFILE%\.ssh\id_rsa"

exit /b

:data1
echo -----BEGIN RSA PRIVATE KEY-----
echo ...
echo -----END RSA PRIVATE KEY-----

exit /b

@jingzheng.tan, you can add a script task to your build plan (before the source code checkout task) that uploads the ssh key to the agent, see this blog post for details: https://blog.nephics.com/2018/08/24/multi-repository-build-plans-with-atlassian-bamboo/

I encountered the same problem and i google but yet to find a solid work-around. Does anyone have luck to find one?

This is not a suggestion, btw, this is a BUG that should get fixed.

Big problem here right now, but seemingly not for Atlassian

I no longer have this issue, it seemed to be specific version of git and/or os that caused this for me.

We didn't see this issue at all when using mac osx (sierra+)

We did see the issue when using Centos 7.6 however.

I think you just need git v2 or higher, which is not available in default centos repos but git2u solves this 

hth

Just ran into this problem today.  This is a major obstacle as we try to introduce the company to using Bamboo for CI builds.

This is definitely a big problem for me now. Its interesting that you have been sitting on this since 2012. If there is a way to help this get prioritize, please let me know. 

Waiting for this too

7 years! Cool!

We are waiting for this too, also

We are waiting for this too, this issue and workarounds keeps us having to waste time on many projects.

It would be great if this can be added

It would be great if this feature could be added, having to maintain ssh keys on agents is an inconvenient workaround.

This reason, among others is a large part of why we have migrated away from Bamboo. We now use Gitlab and Gitlab CI for much better Docker and git support.

We are getting close to 2018 and I'm having the same problems outlined by others above. Please consider putting this on your road map as this is a core git feature.

It's December 2016, and I'd be curious to learn if this is still on the roadmap.

A team member recently looked at Git submodules with our DevOps, and they made the call not to use them due to the following reasons.

Specifically, Bamboo does not support either of the two options that would make submodules work:

• Customizing Git checkouts, like providing the `--recursive` flag to `git clone`
• Using the Bamboo job's built-in authentication when checking out submodules

Instead, Bamboo requires separate Git authentication for submodules. This involves either using HTTPS for submodules and providing the credentials through the job's environment variables, or configuring separate SSH keys on each build agent. Using HTTPS would render local builds unusable, requiring credentials every time. Adding SSH keys to every Bamboo agent is unmaintainable.

Why does the bamboo server need submodule keys? The connections aren't going through it at all (that's the problem). It doesn't even need to know about the submodules in the first place does it?

Getting this fixed would mean that the key needs to be on the server.

I agree though, that needing to put the keys on the agents is a concern though (both logistically and from a security perspective).

Thats exactly our problem as well, a security concern!!

Not having this feature doesn't just limit the usage of submodules with Bamboo and makes it harder to configure and manage by the admins and the Bamboo users, but it is also a security concern, because of two reasons:

• You need to setup a private key on Bamboo and all it's agents
• Bamboo's SSH private key needs to be able to access all submodule repo's which you usually don't want to have if completely different teams use Bamboo.

Are there any news on this? "The end of the year" 2015 has already passed and still there is no fix to this bug...

As we have a lot of different customers we need to be able to separate the customer's repository credentials from each other. We have waited for more than 2 years for a bugfix and now it becomes urgent for us. The first teams are already switching to Jenkins (using the SSH-Agent plugin). Please could you reconsider the priority of this issue and provide an update?

Thanks!

Hi,

I just wanted to provide an update on this request. At this stage it's unlikely that we'll address it before the end of the year.

Thanks everyone for your feedback, please know that we read it and take it into account when prioritising our roadmap but due to other factors we are unable to put this development on top of the backlog.

Thanks,

Sten Pittet
Bamboo Product Manager

i usually add a script task before checkout copying the key. I embedd the key in the script itself and just copy the script as id_rsa file.

like so:

@echo off
:: set HOME to?

if exist "%HOME%" (
  if not exist "%HOME%\.ssh" mkdir "%HOME%\.ssh"
  copy /Y "%~0" "%HOME%\.ssh\id_rsa"
)

set HOME=%USERPROFILE%
if exist "%HOME%" (
  if not exist "%HOME%\.ssh" mkdir "%HOME%\.ssh"
  copy /Y "%~0" "%HOME%\.ssh\id_rsa"
)

exit 0

-----BEGIN RSA PRIVATE KEY-----
...
-----END RSA PRIVATE KEY-----  

Kind of annoying this doesn't happen. I don't want to hand out SSH keys for my servers by hand.

Submodules support doesn't work easily with SSH? Really? This is not even working for local agents on Windows, so I have no idea what workaround to use here. Quite shocked that this is still an issue 2 and a half years later, without well documented workarounds, for a fairly important part of Git.

Giving each Bamboo Agent an SSH key with global Git access is also a security risk; Users with plan creation rights can gain access to any code repository because the agents already know how to authenticate. The SSH private key / repository username and password set in the plan should be used for submodule fetch as well.

Anyone come up with an equivalent solution to the scripts linked above but for Windows remote agents?

I've been fighting with relative submodules and Bamboo; it turns out Bamboo doesn't support submodules created from relative urls at all, and then this issue - Bamboo Agent needs credentials passed again for git operations - came back up. I re-read my words and it almost sounds like I think a workaround of embedding an extra access key in the build script is a viable solution. It's not; I can understand how a script task may not have (easy) access to the credentials the built-in "Source Code Checkout" task has, but when the Source Code Checkout task can't fetch relative submodules, then I am in the same boat as this issue - having to pre-populate all the build servers with local ssh credentials. Please fix, Atlassian.

Kelly, can you provide more details on this work around, or link to explanation.

I meant to update this after Summit. One of the Bamboo leads showed off how they do it - the text of the key is passed as a saved argument/variable in Bamboo, and copied to the appropriate spot each build. So simple and obvious, sheesh.

script (with private bits omitted:
cat << 'EOF' > ~/.ssh/id_rsa
----BEGIN RSA PRIVATE KEY----
...
----END RSA PRIVATE KEY----
EOF
chmod 700 ~/.ssh/id_rsa

echo 'server ssh-rsa ...' >> ~/.ssh/known_hosts

git submodule update --init

I got this to work by switching off submodules and adding a script step to a) create ~/.ssh/id_rsa and ~/.ssh/known_hosts, and then b) run "git submodule update --init". A better solution would be to update the URLs for the submodules to also use the proxy if they user/host/port is the same as the base repo (or maybe no matter what).

Looks like I'm running into the same bug using bitbucket, you'd have thought this integration would work out of the box.

Here's the error

Cloning into 'MyDir'...
fatal: Authentication failed
Clone of 'https://***@bitbucket.org/***/***.git' into submodule path 'MyDir' failed
	at com.atlassian.bamboo.executor.RetryingTaskExecutor.rerun(RetryingTaskExecutor.java:132)
	at com.atlassian.bamboo.executor.RetryingTaskExecutor.runTask(RetryingTaskExecutor.java:88)
	at com.atlassian.bamboo.executor.RetryingTaskExecutor.retry(RetryingTaskExecutor.java:191)
	at com.atlassian.bamboo.executor.RetryingTaskExecutor.retry(RetryingTaskExecutor.java:176)
	at com.atlassian.bamboo.plugins.vcs.task.VcsCheckoutTask.execute(VcsCheckoutTask.java:150)
	at com.atlassian.bamboo.task.TaskExecutorImpl.executeTasks(TaskExecutorImpl.java:188)
	at com.atlassian.bamboo.task.TaskExecutorImpl.executePreparationTasks(TaskExecutorImpl.java:69)
	at com.atlassian.bamboo.build.pipeline.tasks.PrepareBuildTask.call(PrepareBuildTask.java:69)
	at com.atlassian.bamboo.v2.build.agent.DefaultBuildAgent.build(DefaultBuildAgent.java:206)
	at com.atlassian.bamboo.v2.build.agent.BuildAgentControllerImpl.waitAndPerformBuild(BuildAgentControllerImpl.java:103)
	at com.atlassian.bamboo.v2.build.agent.DefaultBuildAgent$1.run(DefaultBuildAgent.java:111)
	at com.atlassian.bamboo.build.pipeline.concurrent.NamedThreadFactory$2.run(NamedThreadFactory.java:52)
	at java.lang.Thread.run(Thread.java:662)

Would be nice if it is possible to use username/password authentication for submodules via https. Either the same values as for the parent repository or a possibility to specify these values in the user interface specifically for a submodule.

what about a "git clone --recursive remoterepositoryurl.git destinationfolder" ?

currently I had to circumvent ur feature doing 2 checkouts, one inside the other

@mfahd those solutions only work for Linux, custom instance scripts are broken in Windows, as are mounting EBS volumes. The only option available to us Windows users is to create a custom image which is unsupported by Atlassian.

For Bamboo OnDemand users, the current workarounds are:

1. use the instance setup script to put the keys onto the image
2. create a custom EBS
3. create a custom image (which ends up as not supported by Atlassian).

rafi, for triggering the build we rely on git ls-remote REMOTE_REPO + checking the revision hash for the polled branch/ref. Later we retrieve the list of changesets since last build using more-less git log lastRev..targetRev on a locally cloned repo -> src

I'm not sure if that answers your question. Is it helpful?

@James On what artifacts does your change detection rely on? Strictly physical files in repository?

m.snelling though when we run change detection on the server we do not detect changes on submodules so SSH connections are never established.

Agreed. Submodules are a standard way of working with Git repositories, especially when incorporating open-source libraries into your project. Having the Git client on the agent use the same set of SSH keys would be a good start, however this won't work if you're forced to access your root repository via HTTPS like you are when using Bamboo OnDemand, Atlassian have told me they don't support using Git over SSH in this service due to security concerns.
So ultimately we still need the option of specifying SSH keys for submodules even though the root repository is accessed by Bamboo over HTTPS.

Seriously. If my plan's repository has a private SSH set, why won't submodules cloning use the same private key? Please fix this so that source checkout with "Use submodules" will use same authentication as its set in the actual plan repository.

Added my vote here. This one seems like a pretty fundamental flaw in "submodule support." An interesting "lets meet half way" solution might be for it to look up to see if the submodule is available as a configured shared repo in bamboo and pull that rather than using the git submodule init / update process that it's using now.

BSP-6565