Common Go Pitfalls

A few common mistakes and how to diagnose and fix them

Tyler Finethy
Oct 24 · 5 min read
Avoid pitfalls while writing simple, reliable Go code

There’s a few reasons I love Golang:

  • It’s a super small language (it has only 25 reserved keywords)
  • Cross-compilation is a breeze
  • Creating a reliable HTTP(s) server is natively supported

At its core, it’s a boring language, which is probably why awesome projects like Docker and Kubernetes are written in it and companies with high performance and resiliency requirements, like Cloudflare, are using it.

Despite its ease of use, Go really requires attention to detail. If you don’t use the language as it’s intended it can break. It can be hard to diagnose and challenging to fix the mistake.

Here are a few common mistakes I’ve witnessed in production codebases, during code reviews, and made myself. Hopefully, this will make it easier for you to diagnose the same issues as you encounter them.


HTTP Timeouts

This first issue has an entire article written about it but it’s still worth mentioning because the optimal solution can require some thought. It has to do with making outgoing HTTP requests using the default HTTP client.

To illustrate the problem, here’s a basic example of making a request to google.com:

package mainimport (
"io/ioutil"
"log"
"net/http"
)
var (
c = &http.Client{}
)
func main() {
req, err := http.NewRequest("GET", "google.com", nil)
if err != nil {
log.Fatal(err)
}
res, err := c.Do(req)
if err != nil {
log.Fatal(err)
}
defer res.Body.Close()
b, _ := ioutil.ReadAll(res.Body)
...
}

As pointed out in the Don’t use Go’s default HTTP Client article, the default client doesn’t actually have a timeout. This means that code could hang indefinitely depending on the server or until the application is restarted.

So what’s the best way to resolve this issue?

While always defining your HTTP client with a sensible timeout is a good idea, you might also consider attaching a context to your request for a few added benefits:

  • The ability to cancel ongoing requests
  • You can tune the timeout to specific requests

The second benefit is especially important because if you have a few requests that you know are going to take a long time, say over an hour, you don’t want every request to wait an hour before timing out.

In the example above, adding a context would look something like this:

ctx, cancel := context.WithTimeout(context.Background(), time.Minute)
defer cancel()
req = req.WithContext(ctx)res, err := c.Do(req)
...

If the allotted time is exceeded, the call to will result in a error, making it easy to handle or retry. For more information on the context package, check out the documentation.


Database Connections

I’ve had database connection issues crop up in almost every Go project I’ve been on. I think the hard thing for new gophers to wrap their heads around is that the object is a concurrency-safe pool of connections instead of a single database connection. This means that if you forget to return your connections to the pool you can easily exhaust the number of connections and your application can grind to a halt.

For instance, the connection pool contains both and connections which are configured through:

Note that even if you configure the max open connections to 200, the application can still exhaust the number of open connections the database will accept, making a shutdown or restart necessary. You need to check the database settings or coordinate with whoever has the permissions to ensure you’re correctly setting these limits.

If you don’t configure a limit, your application can easily use all the connections the database will accept.

Back to exhausting the connection pool. When querying the database a lot of developers forget to close the object. This leads to hitting the max connections limit and causes deadlock or high latency. Here’s a snippet of code showing this:

package mainimport (
"context"
"database/sql"
"fmt"
"log"
)
var (
ctx context.Context
db *sql.DB
)
func main() {
age := 27
ctx, cancel := context.WithTimeout(context.Background(), time.Minute)
defer cancel()
rows, err := db.QueryContext(ctx, "SELECT name FROM users WHERE age=?", age)
if err != nil {
log.Fatal(err)
}
for rows.Next() {
var name string
if err := rows.Scan(&name); err != nil {
log.Fatal(err)
}
fmt.Println(name)
}
...
}

You’ll notice, just as you can add context to an HTTP request, you can also add a context with a timeout to a database query (or an execution of a prepared statement, ping, etc.) But that’s not the problem.

As mentioned above we need to close the rows object to prevent further enumeration and release the connection back to the connection pool:

rows, err := db.QueryContext(ctx, "SELECT name FROM users WHERE age=?", age)
if err != nil {
log.Fatal(err)
}
defer rows.Close()

This becomes particularly difficult to spot if you’re passing open connections across functions and packages.


Goroutine or Memory Leaks

The last common mistake I’m going to cover here is Goroutine leaks. These can be tricky to detect but are usually caused by user error.

This happens often when using channels. For example:

package main
func main() {
c := make(chan error)
go func() {
for err := range c {
if err != nil {
panic(err)
}
}
}()
c <- someFunc()
...
}

If we don’t close the channel c or if doesn’t return an error, the Goroutine we have initialized will hang until the program terminates.

Instead of enumerating the number of cases that can cause Goleaks, there are two methods I commonly deploy to detect and eliminate them.

The first method is to use a leak detector in your tests, like Uber’s goleak library. In practice this looks like this:

func TestA(t *testing.T) {
defer goleak.VerifyNone(t)
// test logic here.
}

This will verify, after a grace period of 30 seconds to allow for graceful shutdown, that there are no unexpected Goroutines running at the end of a test.

The other method is to use the Go profiler on a running instance of your application and look at the number of active Goroutines. One way to do this is to add the library and click the Goroutine profile.

You can enable it by adding this:

import _ "net/http/pprof"func someFunc() {
go func() {
log.Println(http.ListenAndServe("localhost:6060", nil))
}
}

This will enable on port 6060. For especially bad leaks, you can refresh and see the number of goroutines increase. For more subtle leaks, read through the profile and look for instances of functions sticking around when they shouldn’t. The profile page will look something like this:

goroutine profile: total 39
2 @ 0x43cf10 0x44ca6b 0x980600 0x46b301
# 0x9805ff database/sql.(*DB).connectionCleaner+0x36f /usr/local/go/src/database/sql/sql.go:950

2 @ 0x43cf10 0x44ca6b 0x980b18 0x46b301
# 0x980b17 database/sql.(*DB).connectionOpener+0xe7 /usr/local/go/src/database/sql/sql.go:1052

2 @ 0x43cf10 0x44ca6b 0x980c4b 0x46b301
# 0x980c4a database/sql.(*DB).connectionResetter+0xfa /usr/local/go/src/database/sql/sql.go:1065
...

If your application is idle and you’re seeing a lot of total Goroutine’s that’s a good indication that something is going wrong. After identifying where the leak is, I still recommend using a leak detector in the tests to ensure the issue is resolved.


Conclusion

Hopefully knowing about and seeing some examples of these common mistakes will help you identify and fix them more quickly. Obviously there are a number of other common mistakes, such as:

  • Race conditions
  • Deadlocks
  • Error swallowing

These can be found and fixed through similar techniques, like using the go race detector, writing tests, or using the go profiler.

Better Programming

Advice for programmers.

Tyler Finethy

Written by

Software Engineer at RStudio, Inc with a passion for small companies with big challenges

Better Programming

Advice for programmers.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade