Writing Tests First

Josiah Ulfers
Mar 3 · 4 min read

There’s a best way to automate your tests: write them before you write the program. If you’ve heard “test-driven development,” that’s what it means, but if you’ve never seen such a thing before, it’s an unintuitive idea. How should you start?

A basic example

Here, we’ll develop a function that implements the factorial function in a test-driven style. We start with a stub: an empty function. This does nothing, but that’s the point: in effective automated testing, we write the tests first, only after we have a test do we write the code.

def factorial(n):
...

Traditionally, automated tests are a series of examples in the form “given input X, I expect output Y.”

For our factorial function, we’ll have examples like “what is the factorial of 5?” Answer: 120. We don’t start at five though, we start with something much more basic: “what is the factorial of 0?” We’ll write our examples using the simplest method of automated testing: the assert statement:

def factorial(n):
...
assert factorial(0) == 1

After adding our first test, we run the code. It’s important that we still haven’t written any code in our factorial function, because we want our test to fail. It does:

$ python fac.py
Traceback (most recent call last):
File "fac2.py", line 15, in <module>
assert factorial(0) == 1
AssertionError
$

Red-green

When we write test-first, we always write a failing test first. Once we have a failing test, we can write the simplest possible code to make the test pass:

def factorial(n):
return 1
assert factorial(0) == 1

More sophisticated test tools report how many tests run and how many fail. When using plain assert statements, however, we don’t get those details. Instead, we assume no news is good news: if we run our program and see no errors, our test passed:

$ python fac3.py  # no errors, all's well!
$

But all is not well yet. We know that this isn’t correct: our factorial function returns “1” no matter what we give it. Obviously, there should be an “if” condition here, maybe “if n == 0: return 1”, but we don’t write that because we’re writing test-first and we don’t have a test for the if condition. We could just as easily write “if True: return 1” or “if ‘bananas’: return 1” and all our tests would still pass.

We repeat:

  1. Write a failing test
  2. Write the simplest code to make that test pass

This is called the “red-green” cycle because “failing” is “red” and “passing” is “green.” Let’s write a test:

def factorial(n):
return 1
assert factorial(0) == 1
assert factorial(2) == 2

It fails, as we expect:

$ python fac.py
Traceback (most recent call last):
File "fac4.py", line 17, in <module>
assert factorial(2) == 2
AssertionError
$

We “fix” it, again in the simplest way we can.

def factorial(n):
if n == 0: return 1
if n == 2: return 2
assert factorial(0) == 1
assert factorial(2) == 2

And all our tests pass:

$ python fac.py
$

Back to writing another failing test:

def factorial(n):
if n == 0: return 1
if n == 2: return 2
assert factorial(0) == 1
assert factorial(2) == 2
assert factorial(5) == 120

And verifying that it fails:

$ python fac.py
Traceback (most recent call last):
File "fac6.py", line 14, in <module>
assert factorial(5) == 120
AssertionError
$

And now we make it pass again. We could write “if n == 5: return 120”, but is that really the “simplest” way to make our latest test pass? It’s a matter of judgment but we’ll say that the simplest thing really is to write the recursive branch.

def factorial(n):
if n == 0: return 1
return n * factorial(n - 1)
assert factorial(0) == 1
assert factorial(2) == 2
assert factorial(5) == 120

And it passes. We’ve now built an actual implementation of factorial(n) and by methodically testing at each step, we can be confident we have tests that exercise all the code.

$ python fac.py
$

Refactor

Another name for the “red-green cycle” is the “red-green-refactor.” The “red-green” part is about writing a correct function; the “refactor” part means that once we’ve written a correct function, we can edit it to make it easier to read.

There are many different ways to write this same function. Whether one is better than another may be obvious, or it may depend on your personal taste, but having written this in a test-driven style, we can be confident that we have a suite of tests that covers our algorithm, so when we edit the code for clarity, we can check easily enough that it’s still correct:

def factorial(n):
return n * factorial(n-1) if n else 1
assert factorial(0) == 1
assert factorial(2) == 2
assert factorial(5) == 120

It still passes:

$ python fac.py
$

No test suite is perfect: it is possible to introduce breakage that your test suite doesn’t detect, but this systematic way of building a test suite is the best known way to approach the theoretical ideal. Here, we’ve just rearranged the code to implement the same algorithm a slightly different way; if we were to change the algorithm during refactoring — to implement an iterative algorithm, for example — we might need to write new tests to cover the new code.

This is a good start, but we’ve only done the easy part so far. The hard part of most programs is error handling, so we’ll look at negative testing next.

New Light Technologies

New Light Technologies (NLT) provides enterprise information solutions that combine Information Technology (IT) ingenuity and industry intelligence to drive high-performance results for our clients. We help commercial companies and governments at the federal, state, and local

Josiah Ulfers

Written by

https://josiahulfers.com/

New Light Technologies

New Light Technologies (NLT) provides enterprise information solutions that combine Information Technology (IT) ingenuity and industry intelligence to drive high-performance results for our clients. We help commercial companies and governments at the federal, state, and local

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store