Vendoring Packages with GV and the Go 1.5 Vendor Experiment

We’ve been gearing up for a new product over at Forest Giant, and we can’t wait to tell you more about it! As with most projects we work on, this one has presented us with the opportunity to work with a new technology, the Go programming language.

Over the past few weeks, we have been diving into this simple yet powerful programming language. We followed along with A Tour of Go, referenced the material in Go by Example, watched a number of helpful videos from Gophercon, and joined in on some discussions in the Gopher Academy Slack Group. The language is a lot of fun to work with, and the community is outstanding. With so many tools at our disposal, our team is feeling empowered as we move forward into the production phase of development.

That said, there was one problem in particular that has been nagging us a bit. While one project may require a specific version of a given package, another may require an alternate version of the same package. Running the go get command will simply download a package and place it within your GOPATH/src directory. As we’ve learned all too quickly, issuing another go get command for the same package in the future might break your older projects. So how do we keep multiple versions of the same package separate during development in order to ensure reproducible builds?

So how do we keep multiple versions of the same package separate during development in order to ensure reproducible builds?

As it turns out, we encountered this problem at just the right time. Go 1.5 was recently released, and in this version of the language the toolchain supports a new experimental vendoring feature. To enable this feature, simply set GO15VENDOREXPERIMENT=1 in your environment. Since the feature is experimental, its unknown whether it will be supported in future updates to the language, but for now this is working great for our needs.

Once enabled, this feature allows you to include local copies of external dependencies in your project structure under a directory named vendor. When it’s time to build, the compiler first looks in the local vendor folder to see if a version of the package is present. If the package is not found in this folder, the compiler falls back to looking in your GOPATH. This allows you to include specific versions of a package with your solution, while still allowing you to reference a package from multiple projects using your GOPATH.

As we started using this new experimental flag, it became evident that we wanted a tool to help us set up these vendor folders in our project structures. After browsing around to see what solutions currently supported the vendor experiment, we decided to try our hands at creating our own lightweight vendoring solution called gv. If you’re interested, you can check out the source for this project on our GitHub.

As for how it works, the tool essentially performs some cleanup steps surrounding a call to go get. Before running the command, it sets your GOPATH to the present working directory. Once that is done, it issues the corresponding go get command, which generates a src directory with some newly downloaded packages. After the command is run, it merges the packages found in the src directory into a local vendor directory, then removes the src directory. That’s all we needed, so that’s all it does for now.

That’s all we needed, so that’s all it does for now.

The tool is extremely young, and we’re sure there are things we haven’t thought out fully, but that’s half the fun of development. As new use cases arise, we plan to continue updating it to suit our needs, so check back with us and see what we are up to. In the meantime, if this sounds like something you could benefit from, we’d love for you to download it and give it a spin.

GV tool on GitHub: https://github.com/forestgiant/gv