How to Utilize io.TeeReader in Golang
Imagine you are given some data in the form of an io.Reader, such as arbitrarily sized files or http requests. Your task is to:
- Send the data somewhere, such as cloud storage. (I will be using stdout.)
- Calculate the SHA256 hash of the data, and send it as well.
Because io.Reader represents an incoming stream of data, we can only read its contents once. A naive way to get around this is by reading the contents into a slice of bytes, any then passing this slice multiple times:
However, this implementation has the massive downside of reading everything into memory. Since this function is supposed to handle arbitrarily sized data this will simply not do; at best, you’ve created a memory hog, at worst you crash your program.
We can improve on this by using the TeeReader method which belongs to the io.Reader interface.
In short, a TeeReader is a the result of “binding” an io.Writer to an io.Reader. Reading from a TeeReader is the same as reading from your bound Reader, with the added side effect of writing all data to your bound Writer.
Go Playground link — https://play.golang.com/p/QicSg-Nrmq3
With this trivial implementation, we are now able hash and send any arbitrarily-sized data with constant memory usage. That’s the power of TeeReader!
