Graceful shutdown in Go http server

Photo by Nikita Kachanovsky on Unsplash

When backend developers writing http servers, the most usual question is:

how am I being able to terminate all the running processes and make sure resources are released properly?

Ideally, we should listen (or you can say intercept) for few signals: SIGINT, SIGTERM. Below is the description of Signals:

The common communication channel between user space program and kernel is given by the system calls. But there is a different channel, that of the signals, used both between user processes and from kernel to user process.

What SIGINT means is Signal Interrupted, send when the user types the INTR character (e.g. Ctrl-C).

The SIGTERM signal is a generic signal used to cause program termination. Unlike SIGKILL, this signal can be blocked, handled, and ignored. It is the normal way to politely ask a program to terminate.

So usually these 2 signals can cover most of the scenarios while an application needs to be shut down, as long as we can capture the signals, we can handle the rest of the steps (e.g. truncate pending jobs in the queue, etc).

Using channel

If you familiar with go concurrency, then you will know there is a thing called “Channel”. And one of its biggest advantage and common usage is the blocking behavior, if somewhere you are listening for output from a channel but in the meanwhile that channel not being given any input, the listening place will be blocked until it receives the output.

Channels are the pipes that connect concurrent goroutines. You can send values into channels from one goroutine and receive those values into another goroutine.

Let’s check the code:

The above code, 1st line in main creates a done channel and it can only accept os.Signal type with 1 capacity, then 2nd line signal.Notify causes the package signal to relay incoming signals to done channel.

Then,<-done is trying to receive output from done channel, but if we started the application, the “Server Stopped” will not be shown, as described earlier, because if the application running normally without any termination signal intercepted and nothing in the channel, it will be blocked.

Now you can try Ctrl-C to terminate it, the string will be print out as expected.

Handling http requests

Now, let’s add the http functionality (with gorilla/mux)

As you can see, we created a http endpoint test with the handler, serves on 8080 port, and after the signal listener being set, we start another go-routine to handle the srv.ListenAndServe()

To be honest, it’s not necessary to wrap another go-routine with srv.ListenAndServe() , this method is blocking and in the go documentation it already described that will use separate go-routine for each incoming request. The only reason I’m using go-routine to have another wrapper is because it’s more easier for me to handle the channel interactions and rest of the shutdown steps, if you are interesting with exploring another way to handling it, here is the example.

Serve accepts incoming HTTP connections on the listener l, creating a new service goroutine for each. The service goroutines read requests and then call handler to reply to them.

Jump to line 48, we see a srv.Shutdown with Context, this method introduced in Go1.8.

Shutdown gracefully shuts down the server without interrupting any active connections. Shutdown works by first closing all open listeners, then closing all idle connections, and then waiting indefinitely for connections to return to idle and then shut down. If the provided context expires before the shutdown is complete, Shutdown returns the context’s error, otherwise it returns any error returned from closing the Server’s underlying Listener(s).

Although Go already had support on it, but it’s still not enough, for 2 parts, internal reason an external reason.

Internal Reason

The quote above came from Go documentation said “and then waiting indefinitely for connections to return to idle and then shut down”, this part you should worry because it means some zombie connections may still be there and use your memory, in order to solve this, the safest way is to setup a timeout threshold using the Context.

ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)

The timer context and the cancel function (the cancel function releases its resources) will be returned, and you can use that context to perform Shutdown(ctx) , inside the Shutdown it check if the timer context Done channel is closed and will not run indefinitely.

External Reason

Besides what we’ve explained so far, the major reason of writing graceful shutdown is because in a large system there will not be only 1 http server, usually combines with multiple other connections like database, message queues, etc. And we need to handle these properly as well by defining the defer func and using cancel() to release the resources.

defer func() {
    // Close database, redis, truncate message queues, etc.
    cancel()
}()

Hope this article can provide you some insights on why we are doing the graceful shutdown. If you have any questions or suggestions, please feel free to contact me!