Aditya pratap singh
Aug 7, 2018 · 3 min read
Photo by Hao Zhang on Unsplash

Timeouts are one of the primitive reliability concepts in Distributed Systems that mitigates the effects of the unavoidable failures of distributed systems, as mentioned in this tweet

The Problem

How to simulate the 504 http.StatusGatewayTimeout response conditionally?

While trying to implement OAuth token validation in zalando/skipper, I had to understand and implement a test to simulate a 504 http.StatusGatewayTimeout using httptest when the server timeouts, but only when the client timeouts because of delay at server. As a beginner to the language, I did what most of us would; create the standard HTTP client and add a timeout as below:

client := http.Client{Timeout: 5 * time.Second}

The above seems very simple and intuitive when wanting to create a client to make an http request. But hidden underneath are a lot of low level details, including client timeout, server timeout and timeout at the load balancers.

Client side timeouts

The http request timeout can be defined on the client side using multiple ways, depending on the timeout frame targeted in the request cycle. The request-response cycle is constituted up of Dialer, TLS Handshake, Request Header, Request Body, Response Header and Response Body timeouts. Depending on the above parts of the request-response, Go provides following ways to create request with timeouts

  • http.client
  • context
  • http.Transport

http.client:

The http.client timeout is the high level implementation of timeout which encompasses the whole request cycle from Dial to Response Body. Implementation wise http.client is a struct type which accepts an optional Timeout property of type time.Duration, which defines limit for when request starts till response body is flushed

client := http.Client{Timeout: 5 * time.Second}

context

The go context package provides useful tools to handle timeout, Deadline and cancellable Requests via WithTimeout, WithDeadline and WithCancel methods. Using WithTimeout, you can add the timeout to the http.Request using req.WithContext method

ctx, cancel := context.WithTimeout(context.Background(), 50*time.Millisecond)
defer cancel()
req, err := http.NewRequest("GET", url, nil)
if err != nil {
t.Error("Request error", err)
}

resp, err := http.DefaultClient.Do(req.WithContext(ctx))

http.Transport:

You can specify the timeout also using the low level implementation of creating a custom http.Transport using DialContext and use it to create the http.client

transport := &http.Transport{
DialContext: (&net.Dialer{
Timeout: timeout,
}).DialContext,
}
client := http.Client{Transport: transport}

The solution

So with the above problem and options at hand, I created an http.request with context.WithTimeout() . But this still failed with below error

client_test.go:40: Response error Get http://127.0.0.1:49597: context deadline exceeded

Server Side Timeouts

The problem with the context.WithTimeout() approach is that it still only simulates client side of the request. And in case the request header or body takes longer than the budgeted timeout, the request fails on the client side itself and not on server side with 504 http.StatusGatewayTimeoutstatus code.

One way of creating a httptest server which times out everytime can be like below.

httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request){
w.WriteHeader(http.StatusGatewayTimeout)
}))

But I wanted it to only timeout based on client timeout value. In order to have the server return a 504 based on client timeout, you can wrap the handler with a handler function http.TimeoutHandler() to timeout the request on server. Below is the working test which applies this scenario

func TestClientTimeout(t *testing.T) {
handlerFunc := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
d := map[string]interface{}{
"id": "12",
"scope": "test-scope",
}

time.Sleep(100 * time.Millisecond) //<- Any value > 20ms
b, err:= json.Marshal(d)
if err != nil {
t.Error(err)
}
io.WriteString(w, string(b))
w.WriteHeader(http.StatusOK)
})

backend := httptest.NewServer(http.TimeoutHandler(handlerFunc, 20*time.Millisecond, "server timeout"))

url := backend.URL
req, err := http.NewRequest("GET", url, nil)
if err != nil {
t.Error("Request error", err)
return
}

resp, err := http.DefaultClient.Do(req)
if err != nil {
t.Error("Response error", err)
return
}

defer resp.Body.Close()
}

The details of the above problem is used in the implementation of tokeninfo_test.go/TestOAuth2TokenTimeout in zalando/skipper

Probably a beginner gopher finds it useful to understand the high level workings of http timeouts! In case you want to check more details of http timeouts in go, this article from Cloudflare is a must read.

Congruence Labs

At Congruence Labs, we develop innovative and creative solutions for our clients from various verticals.

Thanks to Rohit Sharma and Roberto Gritti

Aditya pratap singh

Written by

full-stack developer @ZalandoTech interested in web technologies, JS, Go, Rust and reliable distributed systems

Congruence Labs

At Congruence Labs, we develop innovative and creative solutions for our clients from various verticals.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade