Microservices in Go
I recently did a talk at the Melbourne Golang meetup about how to develop microservices and frameworks. In this post I’ll share my knowledge with you (plus it’s a good reminder for me).
These are the frameworks I’ll compare:
Introduction to the frameworks
Go Micro
One of the most popular frameworks in my opinion. There are lots of blog posts and simple examples available. You can follow microhq on Medium or @MicroHQ to get the latest updates in Go Micro.
Ok, what is Go Micro? It is a pluggable RPC framework for writing microservices in Go. Out of the box, you will receive:
- Service discovery - applications automatically registered with service discovery system.
- Load balancing - client side load balancing used to balance requests between instances of a service.
- Synchronous communication - request/response transport layer provided.
- Asynchronous communication - built in publish/subscribe capabilities.
- Message encoding - encoding/decoding based on Content-Type header of a message.
- RPC Client/Server packages - leverage the above features and expose interface to build microservices.
Go Micro architecture can be described as a three layers stack.
The top layers consist of a client-server model and a service abstraction. The server is the building block for writing a service. And the client provides an interface to make requests to services.
The bottom layer consists of plugins of the following types:
- Broker - provides an interface to a message broker for asynchronous pub/sub communication.
- Codec - used for encoding/decoding messages. Supported formats include json, bson, protobuf, msgpack, etc.
- Registry - provides a service discovery mechanism (default is Consul).
- Selector - load balancing abstraction which builds on the registry. It allows services to be “selected” using such algorithms as random, roundrobin, leastconn, etc.
- Transport - interface for synchronous request/response communication between services.
Go Micro also provides such feature as Sidecar. This allows you to work with services written in languages other than Go. Sidecar provides service registration, gRPC encoding/decoding, and HTTP handlers. It’s available in many languages.
Go Kit
Go Kit is a programming toolkit for building microservices in Go. Unlike Go Micro, it is a library which is designed to be imported to a binary package.
Go Kit follows simple rules such as:
- No global state
- Declarative composition
- Explicit dependencies
- Interfaces as contracts
- Domain driven design
In Go Kit you can find packages for:
- Authentication - basic and JWT.
- Transport - HTTP, Nats, gRPC, and others.
- Logging - generic interface for structured logging in services.
- Metrics - CloudWatch, Statsd, Graphite, and others.
- Tracing - Zipkin and Opentracing.
- Service discovery - Consul, Etcd, Eureka, and others.
- Circuitbreaker - Hystrix implementation in Go.
One of the best descriptions you can find of the Go Kit, is in Peter Bourgon’s article and presentation slides:
Also, in the “Go + microservices” slides you will find an example of a service architecture built with Go Kit. For a quick reference, here’s a diagram of the service architecture.
Gizmo
Gizmo is a microservice toolkit from the New York Times. It provides packages to put together server and pubsub daemons. It exposes the following packages:
- server - offers two server implementations: SimpleServer (over HTTP), RPCServer (over gRPC).
- server/kit - experimental package based on Go Kit.
- config - contains functions to configuration from JSON files, JSON blobs in Consul k/v, or environment variables.
- pubsub - provides generic interfaces for publishing and consuming data from the queues.
- pubsub/pubsubtest - contains test implementations of the publisher and subscriber interfaces.
- web - exposes functions for parsing types from request queries and payloads.
Pubsub package provides interfaces to work with the following queues:
- pubsub/aws - for Amazon SNS/SQS.
- pubsub/gcp - for Google Pubsub.
- pubsub/kafka - for Kafka topics.
- pubsub/http - for publishing via HTTP.
So, in my opinion, Gizmo sits somewhere between Go Micro and Go Kit. It’s not a complete “blackbox” like Go Micro. At the same time, it’s not as raw as Go Kit. It provides higher-level building components such as config and pubsub packages.
Kite
Kite is a framework for developing microservices in Go. It exposes RPC Client and Server packages. Created services are automatically registered with a service discovery system Kontrol. Kontrol is written in Kite and it’s a Kite service itself. It means that the Kite microservices work well within its own environment. If you need to connect Kite microservice to another service discovery system it will require customisation. That was the main reason I deleted Kite from the list and decided not to review this framework.
Comparing the frameworks
I’ll compare the frameworks using four categories:
- Objective comparison - GitHub statistics
- Documentation and examples
- Users and community
- Code quality.
GitHub statistics
Documentation and examples
Well, simply speaking, none of the frameworks provide solid documentation. Usually, the only formal documentation is the readme on the front page of the repo.
For Go Micro lots of information and announcements are available at micro.mu, microhq, and social media @MicroHQ.
In case of Go Kit the best documentation you will find is in Peter Bourgon’s blog. And one of the best examples I found was in the ru-rocker blog.
With Gizmo, the source code provides the best documentation and examples.
To sum up, if you come from the NodeJS world and expect to see ExpressJS-like tutorials you will be disappointed. On the other hand, it’s a great opportunity to create your own tutorial.
Users and community
Go Kit is the most popular microservice framework, based on the GitHub statistics - over 10k stars at the time of this publication. It has a generous number of contributors (122) and over 1000 forks. Finally, Go Kit is supported by DigitalOcean.
With over 3600 stars, 27 contributors and 385 forks - Go Micro takes second place. One of the biggest sponsors of Go Micro is Sixt.
And that leaves third place for Gizmo. Over 2200 stars, 31 contributors, and 137 forks. Supported and created by The New York Times.
Code quality
Go Kit takes first place in the code quality category. It has almost 80% of code coverage and an excellent Go report rating. Gizmo has an excellent Go report rating too. But it’s code coverage is only 46%. Go Micro do not provide coverage information but it does have a great Go report rating.
Coding microservices
Alright, enough theory. To better understand the frameworks, I’ve created three simple microservices.
These are the services which implement one business function - greeting. When a user passes a ‘name’ parameter to a service, then the service sends a greeting response. Also, all services meet the following requirements:
- Service should self register with the Service Discovery system.
- Service should have health check endpoint.
- Service should support at least HTTP and gRPC transports.
For those of you who prefer to read the source code. You can stop here and read the source code in the repository.
Go Micro greeter
The first thing you need to do to create a service with Go Micro is to define the protobuf description. Going forward, the same protobuf definition was used in all three services. I created the following service description:
The interface consists of one method - ‘Greeting’. There’s one parameter in a request - ‘name’, and one parameter in a response - ‘greeting’.
Then I used modified protoc to generate service interfaces from the protobuf. The generator was forked by Go Micro and modified to support some of the framework’s features. I wired this all together in the ‘greeter’ service. At that point, the service was starting and registering with the service discovery system. It only supported gRPC transport protocol:
To support HTTP transport I had to add the other module. It mapped HTTP request to protobuf defined request. And called gRPC service. Then it mapped service response to HTTP response and replied it to a user.
Quite simple and straightforward. Many things were handled by Go Micro behind the scenes - such as registration in the service discovery system. On the other hand, it’s hard to create a pure HTTP service.
Go Kit greeter
After I finished with Go Micro I moved on to the Go Kit service implementation. I spent lots of time reading through the code examples provided in the Go Kit repository. Understanding the concept of the endpoint took me a lot of time. The next time-consuming puzzle was service discovery registrar code. I could not implement it until I found a good example.
Finally, I created four packages for:
- Service logic implementation.
- Transport agnostic Service endpoints.
- Transport specific endpoints (gRPC, HTTP)
- Service discovery registrar.
As you can see, the code does not have any dependencies. It just implements logic. The next code snippet shows endpoints definition:
After service and endpoints were defined, I started exposing endpoints via different transport protocols. I started from HTTP transport:
Before I started gRPC endpoints implementation I did not need protobuf definition. I copied the Go Micro service protobuf. But in the case of Go Kit, I used the default service generator to create service interfaces.
Finally, I implemented service discovery registrar:
After all building blocks were prepared, I wired them together in service starter:
As you may have noticed, I used logging middleware in a couple of places. It allowed me to decouple logging logic from the main service/endpoints workflow.
Gizmo greeter
I created the Gizmo service in a similar way to Go Kit. I defined four packages for service, endpoints, transport, and service discovery registrar.
Service implementation and service discovery system registrar shared the same code with the Go Kit service. But endpoints definition and transport implementation had to be done according to Gizmo features.
As you can see, the code snippet is similar to Go Kit. The main difference is in the interface type which should be returned:
The noticeable difference between Go Kit and Gizmo was in transport implementation. Gizmo provides several Service types which you can utilise. All I had to do was to map HTTP paths to endpoints definition. The low-level HTTP request/response processing was handled by Gizmo.
Conclusion
Go Micro was the fastest way to launch the microservice system. There are lots of features provided by the framework. So you don’t need to reinvent the wheel. But this comfort and speed comes with a sacrifice — flexibility. It’s not as easy as Go Kit to change or update pieces of the system. And it enforces gRPC as first class communication type.
It might take some time for you to become familiar with Go Kit. It requires a good knowledge of Go lang features and experience in software architecture. On the other hand, there are no framework restrictions. All parts can be changed and updated independently.
Gizmo sits in-between Go Micro and Go Kit. It provides some higher level abstractions such as the Service package. But the lack of documentation and examples meant I had to read through the source code to understand how different Service types work. It was easier to work with Gizmo than with Go Kit. But it was not as smooth as Go Micro.
That’s all for today. Thanks for reading. Please check out the microservices code repository for more information. If you have any experience with Go and microservice frameworks please share it in the comments below.
