Making HTTP Requests in Golang
We developers make http requests all the time. In this particular post, we’re going to make some http requests using Go. Go is a language I really love and I am going to show you how I make http requests using the net/http
built-in package. We will make use of the httpbin.org site to help us inspect the requests.
Quick Start
The http package offers convenient functions like Get
, Post
, Head
for common http requests. We’re going to see quick code examples for these functions.
Using Get
In this example, we’re calling http.Get
which gives us a response and error value. If there was an error while making this request, the err
variable will be non nil
. We’ll log this error and quit. In case you didn’t know, the log.Fatal
family of functions print the message and then calls os.Exit
to terminate the program.
We then defer the execution of resp.Body.Close()
which will be executed at the end of the function. This step is very very important. When you have a response body (it is not nil
), forgetting to close the response body can cause resource leaks in a long running programs.
We’re then reading the entirety of response body and logging it. The resp.Body
implements the io.Reader
interface allowing us to use ioutil.ReadAll
function. In our case, we do know that the target response is small in size. This gives us the confidence to read the full response in memory. However, since response body is an io.Reader
, you could read the data chunk by chunk and process it as a stream of data. But for this blog post, I will keep my code examples short.
If we run this code, this is what I get as output:
2019/02/03 21:23:00 {
"args": {},
"headers": {
"Accept-Encoding": "gzip",
"Connection": "close",
"Host": "httpbin.org",
"User-Agent": "Go-http-client/1.1"
},
"origin": "<my ip address>",
"url": "https://httpbin.org/get"
}
Using Post
The Post
function is similar. In this example, we’re going to send a JSON payload. We are marshalling a map and will get a []byte
if successful. We handled the error and then called http.Post
. This function takes the url, the content type we’re using (in our case JSON) and an instance of io.Reader
. At our hands, we have a []byte
which doesn’t implement this interface. So, we use bytes.NewBuffer
which gives us a bytes buffer based on our bytes slice. This buffer is both readable and writable. It’s “readable” part satisfies the io.Reader
interface and serves our purpose.
We’re already familiar with the rest of the codes from our previous example.
Posting a Form & Decoding JSON
In this example, the formData
variable is of url.Values
type which is basically map[string][]string
— means it’s a map type, where each key has a value of []string
. For each key, we can have a list of string values.
We can use the http.PostForm
function to submit forms quickly.
Remember, we talked about taking advantage of resp.Body
being an io.Reader
to read as stream? json.NewDecoder
can take an io.Reader
to read the data chunk by chunk. Then we use Decode
to unmarshal the JSON into Go data structure, in this case a map.
We could have used ioutil.ReadAll
like before to first read the data into memory and then call json.Unmarshall
on it. It would have worked pretty well for a small payload.
Custom Requests
In our previous examples, we have used the http.Get
and http.Post
functions which allowed us to quickly make GET and POST requests. But our options were limited, we couldn’t fully configure the requests. For example, what if we want to add a timeout? And what about other http methods?
Now we’re going to see how we can craft our custom requests and complete them using custom http clients.
In this code snippet, we’re creating our own http.Client
and passing a value for the Timeout
option. By creating a new client, we can customize the available options on the client like this.
Next, we create a new request. We pass the io.Reader
as request body. We then set the content type in the request header. Finally, we call client.Do
with our request
. The rest are the same as before.
The http.Get
and http.Post
functions just use a default http client already initialized in the module:
var DefaultClient = &Client{}
File Upload
- First we are opening the file we want to upload. In our case, I have created a file named “name.txt” that just contains my name.
- We create a
bytes.Buffer
to hold the request body we will be passing with ourhttp.Request
later on. - We create a
multipart.Writer
object and pass a pointer to ourbytes.Buffer
object so the multipart writer can write necessary bytes to it. - The multipart writer has convenient methods to create a form file or a form field. It gives us back a writer to which we can write our file content or the field values. We create a file field and copy our file contents to it. Then we create a normal field and write “Value” to it.
- Once we have written our file and normal form field, we call the
Close
method on the multipart writer object. Closing it writes the final, ending boundary to the underlyingbytes.Buffer
object we passed to it. This is necessary, otherwise the request body may remain incomplete. - We create a new post request like we saw before. We passed the
bytes.Buffer
we created as the request body. The body now contains the multi part form data written with the help of themime/multipart
package. - We send the request as before. But we set the content type by calling
multiPartWriter.FormDataContentType()
– which ensures the correct content type and boundary being set.
This is a mostly modified post, derived from my original article here: http://polyglot.ninja/golang-making-http-requests