Decoding Brotli in Golang

Brotli is a generic-purpose lossless compression algorithm that compresses data using a combination of a modern variant of the LZ77 algorithm, Huffman coding, and 2nd order context modelling, with a compression ratio comparable to the best currently available general-purpose compression methods. There are a lot of articles out on the internet comparing the advantages of Brotli compression. You can read more about Brotli compression here

We are going to demonstrate how to decode a brotli encoded HTTP response in Go. By default, Go will decode gzip responses if no Accept-Encoding header is set while making the request.

Many CDNs and web servers are using Brotli compression nowadays. However, when we need to make sure the response is encoded (eg. to reduce bandwidth usage), we might end up in a situation to explicitly set Accept-Encoding header as br while making HTTP/HTTPS requests.

In such cases, this article aims to provide the necessary details. At SellerApp, we have come across such use cases.

To support this, brotli package needs to be installed on the machine. For Debian the package name is libbrotli-devand for alpine it is brotli-dev To demonstrate the example I’ve used an nginx docker enabled with brotli compression from https://hub.docker.com/r/fholzer/nginx-brotli .

docker run -p 80:80 fholzer/nginx-brotli will start nginx at port 80.

Go module github.com/google/brotli/go/cbrotli is a required dependency for this.

Let’s get to the code. Assuming the docker is running on the localhost, we can make the request.

req, err := http.NewRequest(http.MethodGet, "http://localhost", nil)

Now let’s set Accept-Encodingheader to br.

req.Header.Set("Accept-Encoding", "br")

After verifying, the response is encoded in br by checking Content-Encoding response header, let’s initiate a new Brotli reader from the HTTP response to decode the response.

reader := cbrotli.NewReader(resp.Body)
defer reader.Close()
respBody, err := ioutil.ReadAll(reader)
if err != nil {
log.Fatal("error decoding br response", err)
}
log.Println("decoded resp \n", string(respBody))

The full code for this can be viewed in this Github repo. The encoded response is logged for illustrational purposes.