7 tips on how to write kick-ass high performance Golang microservices

This is a how-to introduction to Go microservice design and development. This article will lead you directly to the best practices of the Go world and show you the gotchas before hand. I will leave all the sources down below.

1. Don’t use Go’s default HTTP client
2. Consider these Frameworks and External Libraries (benchmarks)
3. Don’t be afraid of Makefiles
4. Use Go Modules to save your time setting up the environment
5. Log properly (Logrus)
6. Write unit tests without pain
7. Document quickly (Swagger)

1. Don’t use Go’s default HTTP client

Writing Go programs that talk to services over HTTP is easy and fun. However there is a pitfall that is easy to fall into and can crash your program very quickly: the default HTTP client.

Go’s http package doesn’t specify request timeouts by default, allowing services to hijack your goroutines. Always specify a custom http.Client when connecting to outside services.

When you use http.Get(url), you are using the http.DefaultClient, a package variable that defines the default configuration for a client. The declaration for this is:

var DefaultClient = &Client{}

Among other things, http.Client configures a timeout that short-circuits long-running connections. The default for this value is 0, which is interpreted as “no timeout”. This may be a sensible default for the package, but it is a nasty pitfall and the cause of our application falling over in the above example. As it turns out, Spacely Sprockets’ API outage caused connection attempts to hang (this doesn’t always happen, but it does in our example). They will continue to hang for as long as the malfunctioning server decides to wait. Because API calls were being made to serve user requests, this caused the goroutines serving user requests to hang as well. Once enough users hit the sprockets page, the app fell over, most likely due to resource limits being reached.

The Solution

It is always best to have finer-grained control over the request lifecycle, you can additionally specify a custom net.Transport and net.Dialer. A Transport is a struct used by clients to manage the underlying TCP connection and it’s Dialer is a struct that manages the establishment of the connection. Go’s net package has a default Transport and Dialer as well. Here’s an example of using custom ones:

tr := &http.Transport{
        DialContext: (&net.Dialer{
            Timeout:   n * time.Second,
            KeepAlive: n * time.Second,
        }).DialContext,
        TLSHandshakeTimeout: n * time.Second,
        
        ExpectContinueTimeout: n * time.Second,             
        ResponseHeaderTimeout: n * time.Second,
        MaxIdleConns:    n,
        MaxConnsPerHost: n,
}
cli :=  &http.Client{
        Transport: tr,
        Timeout:   n * time.Second,
}

2.Consider these Frameworks and External Libraries

If you ask a Go developer about what web frameworks or libraries you could use, the typical answer is stick to the standard libraries. Ironically, the top google search result for “golang frameworks” is about why you should not use them.

Building HTTP servers is easiest with a framework.

I found benchmarks against Gin, Echo, Beego, Gorilla Mux, Goji for a single named parameter and below are the results. Gin has the fastest router, followed by close second Echo.

JSON Serialization & Deserialization

Once an API request is hit through the router and passed on to a controller or handler, the next step is to Decode the request JSON or Encode while returning the response.

Go has a really good encoding package which supports multiple formats like json, XML, csv, but a quick look at the alternatives show you tons of libraries. Here is comparison benchmark of Jsoniter, EasyJson against the standard encoding/json package and below are the results.

Below is the result for Decoding JSON.

Now if you have your request decoded, next step could be applying your business logic and may be do some database operations.

While sqlx reduces the typical number of lines you write to build a CRUD, you still end up writing repetitive code a lot of times. Using an ORM could help reduce it and focus on your business logic.

Here is a benchmark of database, database + sqlx, gorm , go-pg for querying and below are the results. Surprisingly, go-pg, an ORM performed faster than the standard package or even sqlx. GORM while very famous in the ecosystem is relatively slow.

Querying 200K records from a postgres DB

These benchmarks will help you choose your set of frameworks.

3. Don’t be afraid of Makefiles

During the development, I was used to repeatedly execute “go build”,“go test” manually. This was a bad habit on which I resign. It is not so painful if you use simple command without any args. But in case of more complex tasks, naturally, it is going to be a pain. There are few options you can consider as a way out. You can use a bash script to do the work for you. Or better, at least for me, you can write a Makefile. The make tool is there for this reason and in the Makefile you can keep all your common tasks together.

My Makefile normally looks something like this:

build: ## Build
go build -o bin/binary_name cmd/main.go

run: ## Run the server
bin/binary_name

help:
@grep -E '^[a-zA-Z_-]+:.*?## .*$$' $(MAKEFILE_LIST) | awk 'BEGIN {FS = ":.*?## "}; {printf "\033[36m%-30s\033[0m %s\n", $$1, $$2}'

.DEFAULT_GOAL := help

You would execute it like this:

> make run
> make build
> make // will list all the commands with the comments

More info about Makefile possibilities: https://sohlich.github.io/post/go_makefile/

4. Use Go Modules

Source: https://blog.golang.org/using-go-modules

For best explanation please refer to the source article about Go Modules.

If possible you should make your project importable to other projects as a module.

This saves a lot of time and you will find where to reuse your code very quickly.

5. Log properly

Recommendation is https://github.com/sirupsen/logrus . While it’s one of the most popular it is also feature rich.

6. Write unit tests without pain

You should not chase that 100% code coverage. Just cover the most critical parts. Standard testing library works just fine.

For testing your http handlers you can refer to "net/http/httptest" library.

7. Document your microservice

For documentation it is very handy to use Swagger.

https://github.com/swaggo/swag has been used and is capable of providing decent documentation with examples.

I hope that this helps you out! Reach out if something needs improvement or you think differently. Thank you for reading, take care!

Related articles

https://medium.com/@nate510/don-t-use-go-s-default-http-client-4804cb19f779

https://hackernoon.com/the-myth-about-golang-frameworks-and-external-libraries-93cb4b7da50f

https://sohlich.github.io/post/go_makefile/

https://blog.golang.org/using-go-modules