Working with Git Submodules in CodePipeline

We’ve recently faced an issue when deploying a repository containing submodules while using Code Pipeline. First, let’s understand how Code Pipeline pulls data from Github.

Let’s say we have a very simple deployment process like this:

1- CodePipeline pulls a specific branch from a repository
2- CodeBuild receives the files and the build/compiling process happens with a buildspec.yml file.

OK, so ideally what we would do is simply run git submodules init and git submodules update --recursive in our buildspec.yml file and it should be all set, right?

Not really. When CodePipeline pulls the files from your repository and passes to CodeBuild, it actually doesn’t include the .git directory, that being said, we can’t perform any Git actions because it is no longer is a Git repository.

After spending some time researching we came across this Stack Overflow answer https://stackoverflow.com/a/54318204/4241974, which helped to solve our problem.

Solution

We set up a “Machine User” account in Github https://developer.github.com/v3/guides/managing-deploy-keys/#machine-users.

This “Machine User” is just another Github account that contains an SSH key. The SSH key will be…