Go Modules with Private Git Repositories

Tim Jones
Tim Jones
May 20, 2019 · 3 min read
Image for post
Image for post

Modules have greatly improved the dependency management process in Go. If you are new to Go Modules, and you would like to read more about how to get started, check out the official documentation.

If you’re looking for IDE recommendations for Go, this is a good comparison:

Once everything is configured correctly, it’s pretty straightforward to include specific versions of Go packages from public repositories. One of my typical starting points might look something like this:

What if we want to extend this behavior to import Go packages from private repositories? It’s pretty simple, really: make sure your Go installation has access to the private Git repositories. But how do we do that?

Private Repositories

Under the hood, Go is using Git to pull the specified versions of your dependencies. So, the git configuration for wherever Go is running (eg. a Docker container or your laptop) has to have the appropriate credentials to access any private repositories.

Fortunately for us, there’s a git command for that. The commands below will put an entry into your .gitconfig file that tells git to use the credential-formatted URL whenever it needs to access the standard URL. For these we’re using a personal access token instead of a password, because this will store the entries in plain text. Some good discussions about this, and alternatives, can be found on this Stack Overflow question.

Important Note:

Your auth token must be url-encoded!

The following gists are formatted on separate lines to attempt to avoid side-scrolling:

BitBucket

privatebitbucket should be replaced with the URL for your private BitBucket server, if applicable

GitHub

GitLab

privategitlab should be replaced with the URL for your private GitLab server, if applicable

This is great for local development, but what about my CI/CD pipeline?

I’m glad you asked! Here is an example of a Dockerfile that allows for the injection of credentials at build time:

Go Dockerfile

I like to roll with docker compose, so here is an example compose file that I would use to run this Dockerfile:

docker-compose.yml

Of course, Jenkins or Travis or whoever can also provide build arguments while building the Docker image, so that Go Modules can do its work without getting blocked by pesky authentication.

An Alternative — SSH

Another way to set this up would be to use your SSH key to connect, and set your .gitconfig like the following to ensure that SSH is used each time:

I, personally, found this setup to be slightly more difficult to debug when I ran into an issue, so I prefer to use the auth token URL.

Another Alternative — netrc

[added in response to the insightful comment by https://medium.com/@ysamlan]

I originally omitted this alternative, which was pointed out in this comment about using netrc with go modules:

That’s it!

Thanks for reading, I hope this helps on your Go Modules journey. Please let me know if I can make this post better or more complete.

cloud native: the gathering

A gathering place for all things cloud native…

Tim Jones

Written by

Tim Jones

I like to build things and put them in clouds. I try to write about non-intuitive solutions to problems I run into along the way.

cloud native: the gathering

A gathering place for all things cloud native, microservices, api development

Tim Jones

Written by

Tim Jones

I like to build things and put them in clouds. I try to write about non-intuitive solutions to problems I run into along the way.

cloud native: the gathering

A gathering place for all things cloud native, microservices, api development

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store