Enhanced Http Client

While working on different projects there was always a need for building the integration layer for different HTTP upstreams. The problematic part is that in distributed systems there could be many transient failures, among them: timeouts, internal service error, network failures.

There are many valuable articles about patterns to be applied for integration layer. You can easily find them by searching with proper keywords in internet: retry policy, circuit breaker.

Here is the golang library that can be used for those who are in need to use these patterns in building their http integration layers: https://github.com/RassulYunussov/ehttpclient

It implements two patterns:

• retry policy. The policy is aimed to cope with transient network/upstream failures including error http status 5xx. The backoff mechanics is linear. The diagram demonstrates the configuration of EnhancedHttpClient with 2 retries.

• circuit breaker. The logic is aimed to cope with long-period upstream failures. As a circuit breaker gobreaker implementation selected.

To use it in your project:

go get github.com/RassulYunussov/ehttpclient

Usage in code is simple. Client can construct EnhancedHttpClient with different features applied:

• WithRetry
• WithCircuitBreaker
• WithRetry + WithCircuitBreaker

client := ehttpclient.Create(200*time.Millisecond, 
                ehttpclient.WithRetry(3,100*time.Millisecond), 
                ehttpclient.WithCircuitBreaker(5,2,time.Second, time.Second))
request, err := http.NewRequest(http.MethodGet, "http://some-resource", nil)
response, err := client.Do(request)
if err!=nil {
  // handle error
  return
}
// and yes, never forget to close ReadCloser
defer response.Body.Close()

Notes:

• The request object should typically have a time bounded context
• The circuit breaker layer takes place before retry policy
• The retry policy is a simple count based algorithm with backoff behaviour. Backoff behaviour takes initial backoffTimeout and with every next attempt multiplies it by an attempt number + jitter. Jitter — is a random value of [ 0, backoffTimeout/2 )