TDD On A Basic RESTful JSON Quotes API — Part 1
A call to write tests before you write code.
Overview
There are not enough tutorials on TDD. To advocate for this advantageous practice, this post aims to succinctly walk through testing our way to a robust Quotes API.
The content is going to consist mostly of code samples. You can also pull down the code from the following repo on GitHub.
The Initial Test
Let’s kick this thing off with a test. This come as a shock to you, but we’re not going to test the success case just yet. Instead, we want to understand how it reacts when we send a POST with no body.
We use the incredibly useful httptest pacakge from the standard library to give us access to the handler. Likewise, we again reach into the standard library for a client to interact with our test server. (For web servers, you can build really nice software without having to dip outside of the standard library.)
Now we’re just going to pass in the method, the URL, and no body into the client where we then call the server. After which we check the status code and the body.
// main_test.go
package main
import (
"net/http"
"net/http/httptest"
"testing"
)
// TestHandleQuote
func TestHandleQuote(t *testing.T) {
app := newApp() // newApp doesn't exist.
ts := httptest.NewServer(http.HandlerFunc(app.handleQuote)) // no handler yet.
url := ts.URL
req, err := http.NewRequest(http.MethodPost, url, http.NoBody)
client := ts.Client()
resp, err := client.Do(req)
if err != nil {
t.Errorf("client call failed: %v", err)
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusBadRequest {
t.Errorf("expected %v, got %v", http.StatusBadRequest, resp.StatusCode)
}
bs, err := io.ReadAll(resp.Body)
if err != nil {
t.Errorf("couldn't read resp: %v", err)
}
expected := `{"error":"JSON body cannot be empty"}`
if string(bs) != expected {
t.Errorf("got %v; expected %v", string(bs), expected)
}
}
Run the test
go test
# Sample output #
# ➜ quotes git:(first-test) ✗ go test
# github.com/montybeatnik/tutorials/cache [github.com/montybeatnik/tutorials/cache.test]
# ./main_test.go:10:9: undefined: newApp
# FAIL github.com/montybeatnik/tutorials/cache [build failed]
…And, rather unceremoniously, we get a compilation error because newApp does not exist. Let’s write enough code to get a passing test.
The test has not actually failed yet.
The First Bit of Code
// main.go
package main
import "net/http"
type application struct{}
func newApp() *application { return &application{} }
func (app *application) handleQuote(w http.ResponseWriter, r *http.Request) {}
func main() {
}
Run the test
go test
# sample output #
# ➜ cache git:(first-test) ✗ go test
# --- FAIL: TestHandleQuote (0.00s)
# main_test.go:19: expected 400, got 200
# FAIL
# exit status 1
# FAIL github.com/montybeatnik/tutorials/cache 0.305s
Now we have our first failing test.
“Never trust a test you haven’t seen fail.” — Marit van Dijk
Make it pass
We’ll define model struct, an application struct and a handler that is really nothing more than a method on the application struct.
At the moment, we are just reading in the JSON request, checking to see we’re able to unmarshal it.
package main
import (
"encoding/json"
"io"
"log"
"net/http"
)
type Quote struct{}
type application struct{}
func newApp() *application { return &application{} }
func (app *application) handleQuote(w http.ResponseWriter, r *http.Request) {
var quote Quote
defer r.Body.Close()
bs, _ := io.ReadAll(r.Body)
if err := json.Unmarshal(bs, "e); err != nil {
log.Println("json decoding fialed:", err)
w.WriteHeader(http.StatusBadRequest)
w.Write([]byte("{\"error\":\"JSON body cannot be empty\"}"))
return
}
}
func main() {
}
Run the test again
Whoops! Unexpected end of JSON! Wait…our test passes?
Well, that’s what we wanted. We supplied an empty body and we got a bad request; moreover, we also got a meaningful message…the one we expected to get, given the scenario.
go test
# sample output #
# ➜ cache git:(first-test) ✗ go test
# 2023/04/01 11:19:47 json decoding fialed: unexpected end of JSON input
# PASS
# ok github.com/montybeatnik/tutorials/cache 0.250s
Great! We have a passing test. I don’t really like that the logs are showing up in the test output. Let’s inject a logger into our app. When we use our factory function to stand up the app in the test, we can throw the away the log output.
Add a logger
main.go updates
// main.go
// omitting output for brevity
type application struct {
log *log.Logger
}
func newApp(log *log.Logger) *application {
return &application{log: log}
}
func (app *application) handleQuote(w http.ResponseWriter, r *http.Request) {
// omitted code
app.log.Println("json decoding fialed:", err) // now app.log.Println()
main_test.go updates
// main_test.go
func TestHandleQuote(t *testing.T) {
logger := log.New(io.Discard, "", 0) // create logger; throw away output
app := newApp(logger) // inject logger into app
Now the log is gone.
go test
# sample output #
# ➜ cache git:(first-test) ✗ go test
# PASS
# ok github.com/montybeatnik/tutorials/cache 0.272s
We have one passing test, but this only tests a single failure scenario. When someone tries to send a POST with no body, we’re covered. But what about when someone doesn’t send an author or forgets to add the message? What about GET requests with no ID, the wrong ID, or an ID that isn’t an integer?
Consequently, we porbably want to test success cases as well. We are not doing any of these things. We’ve got some work to do.
For these types of tests, where I’m hitting the same endpoint with different values, I like to use table tests and I also like sub tests.
package main
import (
"bytes"
"encoding/json"
"io"
"log"
"net/http"
"net/http/httptest"
"testing"
)
// TestHandleQuote
func TestHandleQuote(t *testing.T) {
// test cases improve readability
testCases := []struct {
name string
method string
body []byte
excpectedStatus int
excpectedBody string
}{
{
name: "post with no body",
method: http.MethodPost,
body: nil,
excpectedStatus: http.StatusBadRequest,
excpectedBody: `{"error":"JSON body cannot be empty"}`,
},
}
logger := log.New(io.Discard, "", 0)
app := newApp(logger)
ts := httptest.NewServer(http.HandlerFunc(app.handleQuote))
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
url := ts.URL
req, err := http.NewRequest(tc.method, url, bytes.NewReader(tc.body))
client := ts.Client()
resp, err := client.Do(req)
if err != nil {
t.Errorf("client call failed: %v", err)
}
defer resp.Body.Close()
if resp.StatusCode != tc.excpectedStatus {
t.Errorf("expected %v, got %v", http.StatusBadRequest, resp.StatusCode)
}
bs, err := io.ReadAll(resp.Body)
if err != nil {
t.Errorf("couldn't read resp: %v", err)
}
if string(bs) != tc.excpectedBody {
t.Errorf("got %v; expected %v", string(bs), tc.excpectedBody)
}
})
}
}
Drats! We’ve introduced an issue. Let’s run our test. Note that while we have added the ability to create additional tests, we are still running only one test.
go test
# sample output #
# ➜ cache git:(more-tests) ✗ go test
# --- FAIL: TestHandleQuote (0.00s)
# --- FAIL: TestHandleQuote/post_with_no_body (0.00s)
# main_test.go:48: expected 400, got 200
# main_test.go:56: got ; expected {"error":"JSON body cannot be empty"}
# FAIL
# exit status 1
# FAIL github.com/montybeatnik/tutorials/cache 0.129s
We have not added any fields to our Quote struct, so we got a 200 response. Let’s add some fields to our struct and validate they are not empty strings.
// updates to main.go
// Quote holds the fields for inisghtful text
// and the creator.
type Quote struct {
Author string
Message string
}
// validate ensures certain input criteria are met.
func (q Quote) validate() error {
if q.Author == "" && q.Message == "" {
return errors.New("JSON body cannot be empty")
}
if q.Author == "" {
return errors.New("author must not be blank")
}
if q.Message == "" {
return errors.New("message must not be blank")
}
return nil
}
// handleQuote deals with incoming requests
func (app *application) handleQuote(w http.ResponseWriter, r *http.Request) {
// code omitted for brevity
if err := quote.validate(); err != nil {
app.log.Println("bad request:", err)
w.WriteHeader(http.StatusBadRequest)
resp := map[string]string{"error": err.Error()}
bs, err := json.Marshal(resp)
if err != nil {
w.WriteHeader(http.StatusInternalServerError)
return
}
w.Write(bs)
return
}
// code omitted for brevity
}
Now let’s see if our updates resolved our issue.
go test
# sample output #
# ➜ cache git:(more-tests) ✗ go test
# PASS
# ok github.com/montybeatnik/tutorials/cache 0.125s
Voila! Now let’s create some more tests. Because writing tests is fun, right?!
// test cases improve readability
testCases := []struct {
name string
method string
route string
body []byte
expectedStatus int
expectedBody string
}{
// POST TESTS
{
name: "post no body",
method: http.MethodPost,
body: nil,
expectedStatus: http.StatusBadRequest,
expectedBody: `{"error":"JSON body cannot be empty"}`,
},
{
name: "post no author",
method: http.MethodPost,
body: []byte(`{"message":"excellent!"}`),
expectedStatus: http.StatusBadRequest,
expectedBody: `{"error":"please provide an author"}`,
},
{
name: "post no message",
method: http.MethodPost,
body: []byte(`{"author":"ted"}`),
expectedStatus: http.StatusBadRequest,
expectedBody: `{"error":"please provide a message"}`,
},
{
name: "post no values",
method: http.MethodPost,
body: []byte(`{"author":"","message":""}`),
expectedStatus: http.StatusBadRequest,
expectedBody: `{"error":"please provide both an author and a message"}`,
},
{
name: "post success",
method: http.MethodPost,
body: []byte(`{"author":"bill","message":"excellent!"}`),
expectedStatus: http.StatusCreated,
expectedBody: `{"message":"succesfully created quote"}`,
},
}
It looks like all of our failure cases are handled, yet our success case has an issue. By the way, if you run go test
without any arguments, you’ll run through all of the tests in the calling or specified directory. You can add verbosity with the -v
flag or to run all tests in the project: go run ./...
.
go test
# sample output #
# ➜ cache git:(more-tests) ✗ go test
# --- FAIL: TestHandleQuote (0.00s)
# --- FAIL: TestHandleQuote/post_with_valid_body (0.00s)
# main_test.go:69: expected 201, got 200
# main_test.go:76: got ; expected {"message":"succesfully created quote"}
# FAIL
# exit status 1
# FAIL github.com/montybeatnik/tutorials/cache 0.123s
Let’s now write the code to make our success case test pass. Because we are not concerned with persistence storage right now, let’s use a map to store our entries. We’ll wire it up to our application.
type application struct {
log *log.Logger
store map[int]Quote
}
func newApp(log *log.Logger) *application {
store := make(map[int]Quote)
return &application{log: log, store: store}
}
func (app *application) handleQuote(w http.ResponseWriter, r *http.Request) {
// omitted code for brevity
// increment the count and add the quote to the store
count++
app.store[count] = quote
w.WriteHeader(http.StatusCreated)
resp := map[string]string{"message": "successfully created quote"}
bs, err := json.Marshal(resp)
if err != nil {
w.WriteHeader(http.StatusInternalServerError)
return
}
w.Write(bs)
}
Now our success case test is passing.
go test
# sample output #
# ➜ cache git:(more-tests) ✗ go test
# PASS
# ok github.com/montybeatnik/tutorials/cache 0.128s
# With verbosity to see all of the test names
go test
# sample output #
# ➜ cache git:(more-tests) ✗ go test -v
# === RUN TestHandleQuote
# === RUN TestHandleQuote/post_with_no_body
# === RUN TestHandleQuote/post_with_no_Message
# === RUN TestHandleQuote/post_with_no_Author
# === RUN TestHandleQuote/post_with_valid_body
# --- PASS: TestHandleQuote (0.00s)
# --- PASS: TestHandleQuote/post_with_no_body (0.00s)
# --- PASS: TestHandleQuote/post_with_no_Message (0.00s)
# --- PASS: TestHandleQuote/post_with_no_Author (0.00s)
# --- PASS: TestHandleQuote/post_with_valid_body (0.00s)
# PASS
# ok github.com/montybeatnik/tutorials/cache 0.126s
# ➜ cache git:(more-tests) ✗
What about getting quotes from the API?
This wouldn’t be a post about TDD if we didn’t start with a test case. Since we’ve already built out the logic in the test body, we now need only to add new test cases. Of course, we’ll begin with a failure scenario.
go test
# sample output #
# --- FAIL: TestHandleQuote (0.00s)
# --- FAIL: TestHandleQuote/get_with_a_non_integer_id (0.00s)
# main_test.go:84: got {"error":"JSON body cannot be empty"}; expected {"error":"id was not an positive integer"}
# FAIL
# exit status 1
# FAIL github.com/montybeatnik/tutorials/cache 0.309s
That’s far from the failure we were expected.
This is a bug/oversight. We need to support only GET and POST methods. There isn’t any logic for that.
func (app *application) handleQuote(w http.ResponseWriter, r *http.Request) {
switch r.Method {
case http.MethodPost:
var quote Quote
defer r.Body.Close()
bs, _ := io.ReadAll(r.Body)
if err := json.Unmarshal(bs, "e); err != nil {
app.log.Println("json decoding fialed:", err)
w.WriteHeader(http.StatusBadRequest)
w.Write([]byte("{\"error\":\"JSON body cannot be empty\"}"))
return
}
if err := quote.validate(); err != nil {
app.log.Println("bad request:", err)
w.WriteHeader(http.StatusBadRequest)
resp := map[string]string{"error": err.Error()}
bs, err := json.Marshal(resp)
if err != nil {
w.WriteHeader(http.StatusInternalServerError)
return
}
w.Write(bs)
return
}
// increment the count and add the quote to the store
count++
app.store[count] = quote
w.WriteHeader(http.StatusCreated)
resp := map[string]string{"message": "successfully created quote"}
bs, err := json.Marshal(resp)
if err != nil {
w.WriteHeader(http.StatusInternalServerError)
return
}
w.Write(bs)
default:
w.WriteHeader(http.StatusMethodNotAllowed)
resp := map[string]string{"error": "allowed methods [POST]"}
bs, err := json.Marshal(resp)
if err != nil {
w.WriteHeader(http.StatusInternalServerError)
return
}
w.Write(bs)
}
}
Baby steps. We now get a clear signal that we don’t have support for a GET. Bummer!
go test
# sample output #
# ➜ cache git:(more-tests) ✗ go test
# --- FAIL: TestHandleQuote (0.00s)
# --- FAIL: TestHandleQuote/get_with_a_non_integer_id (0.00s)
# main_test.go:77: expected 400, got 405
# main_test.go:84: got {"error":"allowed methods [POST]"}; expected {"error":"id was not an positive integer"}
# FAIL
# exit status 1
# FAIL github.com/montybeatnik/tutorials/cache 0.277s
While we’re at it, let’s create some tests to ensure we get proper responses given various unsupported methods
// BAD METHOD TESTS
{
name: "method put",
method: http.MethodPut,
excpectedStatus: http.StatusMethodNotAllowed,
excpectedBody: `{"error":"allowed methods [POST, GET]"}`,
},
{
name: "method patch",
method: http.MethodPatch,
excpectedStatus: http.StatusMethodNotAllowed,
excpectedBody: `{"error":"allowed methods [POST, GET]"}`,
},
We can run only the method tests using the -run argument. Because these are sub-tests, we need to use the following syntax.
go test -run TestHandleQuote/method -v
# sample output #
# ➜ cache git:(more-tests) ✗ go test -run TestHandleQuote/method -v
# === RUN TestHandleQuote
# === RUN TestHandleQuote/method_put
# === RUN TestHandleQuote/method_patch
# --- PASS: TestHandleQuote (0.00s)
# --- PASS: TestHandleQuote/method_put (0.00s)
# --- PASS: TestHandleQuote/method_patch (0.00s)
# PASS
# ok github.com/montybeatnik/tutorials/cache 0.124s
Update our handler’s method switch case to ensure our first GET test passes.
case http.MethodGet:
idStr := strings.Split(r.URL.Path, "/")[1] // grab the id at pos 1
id, err := strconv.Atoi(idStr)
if err != nil {
app.log.Println("bad request:", err)
w.WriteHeader(http.StatusBadRequest)
resp := map[string]string{"error": "id was not a positive integer"}
bs, err := json.Marshal(resp)
if err != nil {
w.WriteHeader(http.StatusInternalServerError)
return
}
w.Write(bs)
}
The test now passes as expected.
go test
# sample output #
# ➜ cache git:(more-tests) ✗ go test -run TestHandleQuote/get -v
# === RUN TestHandleQuote
# === RUN TestHandleQuote/get_with_a_non_integer_id
# --- PASS: TestHandleQuote (0.00s)
# --- PASS: TestHandleQuote/get_with_a_non_integer_id (0.00s)
# PASS
# ok github.com/montybeatnik/tutorials/cache 0.263s
Let’s test out a non-existant entry using a bogus id.
func TestHandleQuote(t *testing.T) {
// test cases improve readability
testCases := []struct {
name string
method string
route string
body []byte
expectedStatus int
expectedBody string
}{
// GET TESTS
{
name: "get invalid id",
method: http.MethodGet,
route: "/one",
expectedStatus: http.StatusBadRequest,
expectedBody: `{"error":"the id must be a positive integer"}`,
},
{
name: "get non-existant id",
method: http.MethodGet,
route: "/42",
expectedStatus: http.StatusNotFound,
expectedBody: `{"message":"couldn't find quote matching that id"}`,
},
}
// omitted for brevity
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
url := fmt.Sprintf("%v%v", ts.URL, tc.route)
req, err := http.NewRequest(tc.method, url, bytes.NewReader(tc.body))
if err != nil {
t.Errorf("couldn't build request: %v", err)
}
}
// omitted for brevity
Update the logic to seed the store with an entry.
var count int
func newApp(log *log.Logger) *application {
store := make(map[int]Quote)
// adding an entry to the store for testing
count++
store[count] = Quote{Author: "Descartes", Message: "I think, therefore, I am."}
return &application{log: log, store: store}
}
run the test
go test -run TestHandleQuote/get -v
# sample output #
# ➜ cache git:(more-tests) ✗ go test -run TestHandleQuote/get -v
# === RUN TestHandleQuote
# === RUN TestHandleQuote/get_with_a_non_integer_id
# === RUN TestHandleQuote/get_with_a_non-existant_id
# main_test.go:99: expected 404, got 200
# main_test.go:106: got ; expected {"message":"no matching record"}
# --- FAIL: TestHandleQuote (0.00s)
# --- PASS: TestHandleQuote/get_with_a_non_integer_id (0.00s)
# --- FAIL: TestHandleQuote/get_with_a_non-existant_id (0.00s)
# FAIL
# exit status 1
# FAIL github.com/montybeatnik/tutorials/cache 0.236s
Let’s implement the logic to make the test pass.
// main.go
// at the bottom of the GET case in the handleQuote method.
// code omitted for brevity.
quote, found := app.store[id]
if !found {
w.WriteHeader(http.StatusNotFound)
resp := map[string]string{"message": "no matching record"}
bs, err := json.Marshal(resp)
if err != nil {
w.WriteHeader(http.StatusInternalServerError)
return
}
w.Write(bs)
return
}
_ = quote
}
Back to a passing test
go test -run TestHandleQuote/get -v
# sample output #
# ➜ cache git:(more-tests) ✗ go test -run HandleQuote/get -v
# === RUN TestHandleQuote
# === RUN TestHandleQuote/get_with_a_non_integer_id
# === RUN TestHandleQuote/get_with_a_non-existant_id
# --- PASS: TestHandleQuote (0.00s)
# --- PASS: TestHandleQuote/get_with_a_non_integer_id (0.00s)
# --- PASS: TestHandleQuote/get_with_a_non-existant_id (0.00s)
# PASS
# ok github.com/montybeatnik/tutorials/cache 0.283s
Final test
{
name: "get success",
method: http.MethodGet,
route: "/1",
expectedStatus: http.StatusOK,
expectedBody: `{"author":"Descartes","message":"I think, therefore, I am"}`,
},
And…it fails
➜ cache git:(more-tests) ✗ go test -run HandleQuote/get_success -v
=== RUN TestHandleQuote
=== RUN TestHandleQuote/get_success
main_test.go:114: got ; expected {"author":"Descartes","message":"I think, therefore, I am"}
--- FAIL: TestHandleQuote (0.00s)
--- FAIL: TestHandleQuote/get_success (0.00s)
FAIL
exit status 1
FAIL github.com/montybeatnik/tutorials/cache 0.324s
Let’s finish this.
quote, found := app.store[id]
if !found {
w.WriteHeader(http.StatusNotFound)
resp := map[string]string{"message": "no matching record"}
bs, err := json.Marshal(resp)
if err != nil {
w.WriteHeader(http.StatusInternalServerError)
return
}
w.Write(bs)
return
}
w.WriteHeader(http.StatusOK)
bs, err := json.Marshal(quote)
if err != nil {
w.WriteHeader(http.StatusInternalServerError)
return
}
fmt.Println("quote: ", string(bs))
w.Write(bs)
The final test passes!
go test -run TestHandleQuote/get -v
# sample output #
# ➜ cache git:(more-tests) ✗ go test -run HandleQuote/get_success -v
# === RUN TestHandleQuote
# === RUN TestHandleQuote/get_success
# quote: {"author":"Descartes","message":"I think, therefore, I am."}
# --- PASS: TestHandleQuote (0.00s)
# --- PASS: TestHandleQuote/get_success (0.00s)
# PASS
# ok github.com/montybeatnik/tutorials/cache 0.292s
All tests pass!
go test -run TestHandleQuote/get -v
# sample output #
# ➜ cache git:(more-tests) ✗ go test -v
# === RUN TestHandleQuote
# === RUN TestHandleQuote/method_put
# === RUN TestHandleQuote/method_patch
# === RUN TestHandleQuote/post_with_no_body
# === RUN TestHandleQuote/post_with_no_Message
# === RUN TestHandleQuote/post_with_no_Author
# === RUN TestHandleQuote/post_with_valid_body
# === RUN TestHandleQuote/get_with_a_non_integer_id
# === RUN TestHandleQuote/get_with_a_non-existant_id
# === RUN TestHandleQuote/get_success
# quote: {"author":"Descartes","message":"I think, therefore, I am."}
# --- PASS: TestHandleQuote (0.00s)
# --- PASS: TestHandleQuote/method_put (0.00s)
# --- PASS: TestHandleQuote/method_patch (0.00s)
# --- PASS: TestHandleQuote/post_with_no_body (0.00s)
# --- PASS: TestHandleQuote/post_with_no_Message (0.00s)
# --- PASS: TestHandleQuote/post_with_no_Author (0.00s)
# --- PASS: TestHandleQuote/post_with_valid_body (0.00s)
# --- PASS: TestHandleQuote/get_with_a_non_integer_id (0.00s)
# --- PASS: TestHandleQuote/get_with_a_non-existant_id (0.00s)
# --- PASS: TestHandleQuote/get_success (0.00s)
# PASS
# ok github.com/montybeatnik/tutorials/cache 0.128s
Conclusion
The tests led us to a better API. We account for several failure scenarios and have the framework in place for the future when someone breaks the logic in our code. When we think in terms of how the code will fail, we’ll write better software. Success is free. Things will fail. People will do things they shoudln’t. When you fail to plan for such things, you plan to fail yourself.
Not that it’s super important, but our test coverage is pretty good, too!
➜ cache git:(tutorial-practice-finished) ✗ go test -cover
PASS
github.com/montybeatnik/tutorials/cache coverage: 73.4% of statements
ok github.com/montybeatnik/tutorials/cache 0.131s
Video walkthroughs:
Part 1: https://youtu.be/DnjpMIkRwwA
Part 2: https://youtu.be/AvUga--zYpE
Thanks for reading! Cheers!!!
Edit: added video links.