Go Environment Setup, Minus the Insanity

9 min readJan 16, 2018

So you’re trying out the Go programming language. Good for you! It’s a neat language with a unique and, in my opinion, refreshing approach to clean design, concurrency natives, and more. That’s not what this article is about, though.

package X: cannot download, $GOPATH must not be set to $GOROOT. For more details see: ‘go help gopath’

The help text kind of helps, but only somewhat. Why is this stuff necessary? Well…

The one and only gopher in this post. Cats will be used forthwith.

GOPATH and Coding by Convention

When designing any kind of system for other programmers to use, one has to decide what implied conventions the users should adhere to, and what the users should be able to configure (or generally control) themselves. Some, like gcc/g++, dump all the configuration in the developer’s lap. Between understanding the compiler, linker, dependency location and configuration, setup of a build system using GNU Make (or other similar tools), just building some software becomes a full-time job. Others, such as Python, are more “batteries included”: running stuff is trivial, installing dependencies is as simple as a single command, etc. The same applies for more abstract tools (consider Flask vs Ruby on Rails).

As a set of build tools, Go errs heavily towards the “convention over configuration” mindset. If you don’t follow the not-especially-flexible convention, things get ugly.

Go’s first convention: environment variables. The way these variables are defined varies by platform, but there are always two most important variables:

GOROOT — this should point to the directory at which Go itself is installed; you should never have to mess with this
GOPATH — this should point to your Go “workspace”, a special area where Go code, executables, and other files reside

The workspace is expected to be a directory containing three directories: bin, pkg, and src. bin will contain any executable binaries we compile using go install, pkg will have any non-executable binaries that get created, and src is for all source code (project, dependencies… everything not bundled with Go itself). In particular, src expects its subdirectories to follow the same structure that imports follow. For example, if your code used import "github.com/foo/bar" then Go will interpret that as a request to look at $GOPATH/src/github.com/foo/bar/*.go.

Note: I will be using Unix-style variables and commands (e.g. $GOPATH) but this information is valid for all platforms running Go.

Step by Step Go Setup

To see all this in action (and to demonstrate other aspects of setting up a Go environment), let’s look at a pre-built sample application: https://github.com/fsufitch/sample-go-app. It takes some input from the command line, and starts a web server serving a cat picture and an uptime counter.

These steps are written as I set all this up on a bare-bones AWS EC2 Ubuntu instance — as vanilla Linux as you can get.

Step 1: Install Go

To make sure we’re using the latest Go, we’ll be installing it manually. Don’t worry, it’s not intimidating at all. Just grab the appropriate archive from here, and unpack it: https://golang.org/dl/. In our AWS server’s case:

$ curl https://dl.google.com/go/go1.9.2.linux-amd64.tar.gz | tar xz

This gave us a go/ folder containing everything we need. We can even check if it works:

$ go/bin/go version
go version go1.9.2 linux/amd64

Looks good. We’ll get tired of typing go/bin/go though, so let’s add it to our PATH:

$ echo "export PATH=~/go/bin:$PATH" >> ~/.bashrc

After restarting the terminal (to apply the new .bashrc), we should be able to use the go command line easily.

Step 2: Set up the workspace

As mentioned before, we need a dedicated “workspace” where all our source code will reside. Let’s call it something simple, and add it to our environment. We’ll add the executables in our Go workspace to our path as well.

$ mkdir go-workspace
$ echo "export GOPATH=~/go-workspace" >> ~/.bashrc
$ echo "export PATH=~/go-workspace/bin:$PATH" >> ~/.bashrc

We can double check we got it right by asking Go for its value (after restarting the terminal again):

$ go env GOPATH
/home/ubuntu/go-workspace

Looking good!

Step 3: Install our package

We’re jumping right into it! Go can install stuff right from Github… and it even uses Git to get the files. Given the sample app’s URL (https://github.com/fsufitch/sample-go-app), we can just run this (including -v to see what it’s actually doing):

$ go get -v github.com/fsufitch/sample-go-app
github.com/fsufitch/sample-go-app (download)
github.com/gorilla/mux (download)
github.com/urfave/cli (download)
github.com/gorilla/mux
github.com/fsufitch/sample-go-app/server
github.com/urfave/cli
github.com/fsufitch/sample-go-app

Looks like it downloaded three packages, and installed four (since one repository contained two). We can even see our files ended up in the right place:

$ ls go-workspace/src/github.com/fsufitch/sample-go-app/
glide.lock  glide.yaml  LICENSE  main.go  README.md  server  static

Step 4: Run it!

Go helpfully put the compiled binary for the sample app in the binof the workspace. Since that’s on our PATH, we can simply run it (using the static resource directory provided with our repository):

$ sample-go-app go-workspace/src/github.com/fsufitch/sample-go-app/static/
2018/01/16 21:46:50 Sending port and static path to server module (8080, go-workspace/src/github.com/fsufitch/sample-go-app/static/)
2018/01/16 21:46:50 Creating router using static path: go-workspace/src/github.com/fsufitch/sample-go-app/static/
2018/01/16 21:46:50 Creating uptime handler; unix now: 1516139210
2018/01/16 21:46:50 Serving on :8080...

It works!

You can even check out the two endpoints of the server:

So… we’re done, right? Not quite.

Did you notice how, when we installed the package, it installed its two dependencies? Go is smart: it traverses all the source it’s compiling, and automatically downloads anything needed. It’s not smart enough, though: it will only download the latest available code. While this may be fine for a small project that does not demand utmost reliability/security, it won’t be fine for those that do. Those require some control over exactly which versions of dependencies are used — something that is not a built-in feature of Go.

Let’s clean up our workspace and start over, this time with dependency versioning in mind.

rm -rf $GOPATH/src

Step by Step Go Setup, With Glide

The sample app is set up to use Glide, a simple Go dependency versioning tool. Glide leverages a non-obvious Go feature: if a project contains a directory called vendor/ (more convention-over-configuration), then its contents are used for resolving dependencies over anything found in $GOPATH/src. Glide reads a couple special files, glide.yaml and glide.lock, and puts all the appropriate dependencies at their correct versions in the vendor/ directory. Let’s get on it.

Step 1: Install Glide itself

That’s easy. Their site gives us a simple (but sketchy-looking) command to do it all at once:

$ curl https://glide.sh/get | sh

Step 2: Check out the code

This time, let’s set up the code manually as opposed to using go get.

$ mkdir -p $GOPATH/src/github.com/fsufitch
$ cd $GOPATH/src/github.com/fsufitch
$ git clone https://github.com/fsufitch/sample-go-app.git
$ cd sample-go-app

Step 3: Use Glide to install our dependencies

Glide can be invoked in a few ways:

glide init creates an initial glide.yamland gets us ready to use it
glide get <x> is like go get; it installs a package, and in this case, also records it in the glide.yamland glide.lock
glide update takes the sometimes non-specific version numbers in glide.yaml and turns them into actual version hashes stored in glide.lock; it also installs all the stuff it goes through
glide install installs the specific package versions remembered by the glide.lock file

In our case, we already have a proper glide.lock so the last option is what we need.

$ glide install
[INFO]  Downloading dependencies. Please wait...
[INFO]  --> Fetching github.com/gorilla/context
[INFO]  --> Fetching github.com/gorilla/mux
[INFO]  Setting references.
[INFO]  --> Setting version for github.com/gorilla/context to 08b5f424b9271eedf6f9f0ce86cb9396ed337a42.
[INFO]  --> Setting version for github.com/gorilla/mux to 53c1911da2b537f792e7cafcb446b05ffe33b996.
[INFO]  Exporting resolved dependencies...
[INFO]  --> Exporting github.com/gorilla/context
[INFO]  --> Exporting github.com/gorilla/mux
[INFO]  Replacing existing vendor dependencies

Note: if your app needs to run without downloading absolutely anything other than its source code, you can safely add the vendor/ directory to your version control. Those running on isolated cloud environments, take note!

Step 4: Install and run the app

Now that we have the vendor-ed dependencies in place, we can build and install our code with go install, which skips the dependency-retrieval part of the build process. We can then run it just as we did before:

$ go install github.com/fsufitch/sample-go-app
$ sample-go-app $GOPATH/src/github.com/fsufitch/sample-go-app/static/
2018/01/16 22:30:51 Sending port and static path to server module (8080, /home/ubuntu/go-workspace/src/github.com/fsufitch/sample-go-app/static/)
2018/01/16 22:30:51 Creating router using static path: /home/ubuntu/go-workspace/src/github.com/fsufitch/sample-go-app/static/
2018/01/16 22:30:51 Creating uptime handler; unix now: 1516141851
2018/01/16 22:30:51 Serving on :8080...

That’s it! Really! You can even see the final result happily live here and here (unless it goes offline and I don’t bother fixing it).

Frequently Asked Questions

Can you have multiple Go workspaces?

Yes, and you can switch between them by manipulating GOPATH. There are some neat environment management scripts that do just that. You can even have multiple values in your GOPATH, with Go cascading through them, looking for your imports.

Why does your app need the path to the static files? They’re right there!

They’re “right there” at build time, but not at run time. Remember: Go creates a completely independent executable when it compiles a program. It is free to be placed anywhere, and has no knowledge of where its source files are (if they’re even on the same computer).

Also no, static files cannot be “baked in” to a Go binary. The Go compiler consumes one type of data: Go code. Thus, the only way to “include” a static file in your binary would be to convert it to Go. There are some ways to do that, but they are in most cases unnecessarily messy, in my opinion.

Why do I have to put my code several directories deep in Go’s weird structure instead of in my main code directory?

Convention. Over. Configuration. When code can be placed anywhere willy-nilly, it becomes more difficult to build. Unnecessarily difficult, according to the Go designers.

Why not just add a symbolic link to your code directory so you can find your code quickly?

I did <X> differently but it still works and I like it better, so your post is wrong.

That’s not a question. Also, there are other ways to do this for sure, but in my experience, this is the simplest and most idiomatic way I could find. Please do let me know if you have any feedback on how I could do better, though.

Why does Go not do something normal, like what C, Java, Python, or Malbolge do?

If Go were the popular one, then Python would seem not-normal instead. Go’s approach is just that: another approach. There is, of course, a more philosophical discussion surrounding Go being so “opinionated” in all areas of its design, but that is something that is better answered by its FAQ or the book written by the language’s designers.

That’s all I’ve got. Thanks for reading!

Thanks go to my co-workers, who are serving as the guinea pigs… er… gophers for this exercise in programming.