Golang: Proof of Concept to Production — What we learnt

We had the need for a highly concurrent, networked service to talk to a 3rd party provider. Go seemed to meet our needs here, so we built a proof of concept that turned into a production service. The following is a list of things that would have been valuable to integrate into the POC earlier. Most of them are around the supporting code that makes writing business logic easier.

context.Context

“Package context defines the Context type, which carries deadlines, cancellation signals, and other request-scoped values across API boundaries and between processes.” — https://golang.org/pkg/context/

Request scoped variables could be: request IDs, user IDs, user authentication information, etc. Information like this is often deprioritized in favour of business logic, especially when trying to find out if the language primitives and patterns meet your needs.

“[Google] requires that [their] Go programmers pass a Context parameter as the first argument to every function on the call path between incoming and outgoing requests.” — https://blog.golang.org/context

Passing a context type to all functions might seem strange. Or overkill when hacking together a set functions that prove a concept. But it has a bunch of novel uses, especially around deadlines and cancellations. Which are well documented on the golang blog.

Instead of shoehorning context.Context into your functions later, start with a stub. The context package includes a TODO() stub, that is recognised by static analysis tools. context.TODO() returns a non-nil, empty Context that can used in place of the proper implementation. So pass this around instead of dealing with the real implementation from the start.

Use embedded interfaces

Go uses composition instead of inheritance. While it is the obvious choice for struct types to reduce duplication, composition can also be used with interface types (called embedding). Embedding interface types within one another can help to reduce duplication of method declarations, but it can also help to make it clearer what the responsibilities of that type are within a function.

Given the mock interface ReadWriter which composes the mock Reader and Writer interfaces:

If we have a type that implements the ReadWriter interface, and we pass that type to the ReadOnly(reader Reader) function, the argument reader will only have access to the Read(value string) method from the Reader interface. This helps define what the responsibility is of that type within the ReadOnly function

Runnable example in the playground.

This is useful when you are trying to create a mock implementation of an interface that has many methods. If that interface is composed of other interfaces, and each interface only defines the minimum method set for a single function / use. Then you only need to implement that minimum method set, and not the entire interface!

N.B. you can still assert the underlying interface type, but don’t!

readwriter := reader.(ReadWriter)

The standard library io package makes heavy use of this embedded interface pattern. Examples: ReadWriter,ReadCloser,ReadSeeker, etc.

Write with tests in mind

Testing is not quite the same in golang as in other languages. Java you have instrumentation. Python you have mocking and monkey patching. In go you have interfaces and function types. These are language primitives, not external dependencies in a ‘get out of jail free card’ kind of way. You have to exploit dependency injection and write code in a way that allows you to inject your mocks from your tests. When structuring the code think more about how you are going to make it testable. This doesn’t mean follow a TDD style system, but consider how the tests will mock the required parts. One way is to exploit the composition of interfaces, as discussed above, and pass the minimum method set required for that function. Then when mocking, you don’t need to mock every method of the interface. You only need to mock the methods that are required within that function. Reducing the number of times this appears in your code:

Use the std lib, for everything else there’s gorilla

The go standard library has got your back on most things. It includes an impressively large number of features that you might need, particularly around networking. For everything networky that you can’t find in the standard lib, there’s gorilla https://github/com/gorilla

They have a great collection of libraries, our favourite? gorilla/mux, which makes routing and url dispatching a breeze!

Structured logging and logrus

Ahhh, the age old argument of readability vs. parsability.

Good for computers:

{“level”: “ERROR”, “msg”: “everything is on fire!!!”}

Good for humans:

ERROR: everything is on fire

When building a POC the tendency is to lean towards more human readable, which makes sense. But when that service gets to production and you want to start pulling metrics out of your log messages. Or you want to use the ELK stack, structured logging is superior. And no one wants to find and replace every log in the entire code base.

sirupsen/logrus to the rescue.

“Logrus is a structured logger for Go (golang), completely API compatible with the standard library logger.” — https://github.com/sirupsen/logrus

Logrus lets you do things like swap the logger formatting at runtime, as simply as: log.SetFormatter(&log.JSONFormatter) or log.SetFormatter(&log.TextFormatter{})

Logrus also allows you to add extra fields to a log message:

logger := log.WithFields(log.Fields{“user-id”: “dc2d37b0-faac-4f83–864a-9cf713725c1d”})

And then using that: logger.Info("foo bar"), with the json formatter you get the log line:

{“user-id”: “dc2d37b0-faac-4f83–864a-9cf713725c1d”, “msg”: “foo bar”}

Adding this extra context to your log messages and reusing the logger is a common pattern.

The other approach is that taken by logging in golang/appengine. This approach uses the appengine implementation of context to provide the extra fields / information to the log function.

func Infof(ctx context.Context, format string, args ...interface{})

What we found

• The standard library has everything, ever! (apart from the things that are missing — then github.com/gorrilla).
• Pay attention to how you’re going to test your code.
• Exploit struct and interface composition whenever possible. It will make everything easier.
• Logurs FTW!