Do I need test-driven development?

A pragmatic approach to a polarizing dilemma

Note: This post was originally published on April 26, 2018 on Spring Engineering. It’s now re-published here after ShopRunner acquired Spring, Inc.

I spent the past couple of months as a mentor in the Tech Leaders program. It was a very interesting experience: I had to answer a lot of fundamental questions that I no longer even think about during my daily routine. Actually having to take a stance on some of them taught me a lot. One day I found this question in my email inbox:

“If test-driven development is so good, should I use it everywhere?”

I’m a big fan and proponent of this technique and I use it regularly. However, the short answer to the aforementioned question is no: You don’t have to use it everywhere.

Test-driven development, or TDD, is a technique where you must first write a test that fails before you write new functional code. The idea behind it is that it allows requirements to be turned into very specific test cases. It also ensures that the tests are actually meaningful. An elementary knowledge of TDD principles is required for understanding this article. This tutorial may serve as an introduction to the topic.

So when exactly is it good to use TDD?

Oftentimes, this decision is not immediately clear. In this article, I am going to provide a set of guidelines to help you decide whether or not to leverage this technique in your software development process. These guidelines will be based on these four questions:

1. How complex is your problem?
2. How well do you know your tools?
3. Is your interface well defined?
4. How important is your code?

Question 1. How complex is your problem?

Before we address this issue, let me start with another one — the ultimate question of software engineering:

What exactly is programming all about?

The answer is obviously “42”, but I’ll explain it in detail anyway. You see, programming is all about taking hard problems and turning them into simple solutions.

There are two types of complexity mentioned in the sentence above: problem complexity and solution complexity. The first one is inherent to the problem, and there is nothing we can do to decrease it without redefining the problem itself. It’s a property that describes how hard it is to comprehend something, and how long it takes to explain it. In more technical terms, it tells us how long the specification is.

Because there is more than one way to solve a problem, solution complexity is related, but different. It has to do with things like number of loops, variables and conditionals in the code; also architecture design, probability of having bugs, etc.

Solving a problem with software is like climbing a mountain

Unlike problem complexity, solution complexity is completely under your control. Imagine that you’re trying to climb a mountain: the summit is your problem complexity. You can’t move it anywhere. The climbing path, however, is your choice. When climbing, the easier path you choose, the smaller is the chance of dying a horrible death. When writing software, the simpler your code is, the smaller the probability of having bugs.

Armed with this knowledge, let’s turn to the actual question that should help us find out whether we need TDD. Or any dedicated tests at all.

A complexity-centric view of TDD

It’s tempting to just say that you should write dedicated tests for all of your code. That’s only partially true; it all depends on the problem complexity.

The more complex a given problem is, the more tests it should have. Every requirement adds to this complexity and potentially creates a need for new test cases. Here is where TDD really shines: it’s much easier to create meaningful test cases for a complex piece of code if its solution complexity is actually enforced by the tests, which in turn reflect the problem complexity. With this in mind, you can even think about the TDD process in two parallel ways:

Two ways of interpreting the TDD cycle [Accessibility description: Image on the left depicts a cycle of three states, with transitions named “Break”, “Fix”, and “Refactor”. Image on the right shows the same cycle, with appropriate transitions named “Complicate the problem”, “Complicate the solution”, and “Simplify the solution”]

You are probably familiar with the process described by the left side of this diagram. First you break the tests, then you fix them the quick-and-dirty way, and finally you clean up the code while keeping the tests green.

I prefer to look at it from a different angle, and here comes the right part of our diagram. First, you start with Nothing. Nothing is simple; it’s the simplest possible code: the one that hasn’t been written at all. Despite being simple, Nothing is also useless: it only solves the simplest of all problems — the one that does not exist. So the first step for us is to complicate the “Nothing problem” just a little bit by enforcing our first requirement.

When we complicate the problem by writing the first test, this test should initially be broken, because our code (the solution) is too simple to handle it. We then complicate our code in order to comply with the tests (by turning Nothing into Something).

Then comes the part that is often neglected, but is actually very important: the refactoring. That’s the moment when we pull Occam’s razor out of the toolbox, and focus solely on decreasing the solution complexity. In other words, we simplify the code and make it more elegant and readable. Because while a machine will understand your code regardless of its complexity, the same doesn’t hold in the case of human beings.

If William of Ockham lived today, he would tell you that simplicity is important, and so is refactoring.

Wash, rinse, repeat.

As you can see, TDD is more powerful, the more times you execute this cycle. That’s because each cycle is isolated, introduces a fraction of problem complexity, and allows you to actively trim the solution complexity at any time without sacrificing correctness. However, it also means that the simpler your problem is, the less you benefit from using TDD. That’s because in simple code, there’s little solution complexity to get rid of.

There are even cases where a given piece doesn’t even need dedicated tests at all. These are the simplest parts of your system; most prominently, this rule applies to boilerplate code.

Let me give you an example. We have a simple Java class — let’s call it Employee. (Yes, I’m using Java as an example because there’s a word “boilerplate” in the previous paragraph.) Our class has only a simple constructor and some property getters: getName() and getSalary(). Something so trivial doesn’t need to be developed using TDD. Even more — you don’t have to write a unit test for this class at all. It’s just overkill.

Does it mean that the Employee class will not be tested? Not necessarily. You can still cover it with a test by writing one for some other part of code that actually uses this “dumb” module. Let’s say that you create a Department class, which acts as a container for Employee instances, with a getSumOfSalaries() method. This method is a perfect candidate for a unit test: it needs to loop over employees and return a sum of their salaries, so it has to compute something, and it will have at least one edge case (an empty department). This test can’t possibly work without calling the Employee constructor, and getSumOfSalaries() will have to call the Employee.getSalary() method, so even if we didn’t provide explicit tests for these, they will be executed in a test anyway!

Another example of code that doesn’t have to be developed using TDD, because it usually doesn’t need dedicated unit tests at all, is the startup code of your application. Usually — depending on programming language and technologies used — every application has some amount of setup code that simply unpacks the entire Rube Goldberg machine before it’s set in motion. Maybe you register your API endpoint handlers. Maybe you perform some dependency injection, whether manual or automatic, and bring up your application’s modules to life, one by one.

And again: does it mean that your startup code will not be tested? Just like previously, they still can be tested in a different way. If the startup code sets up your entire application, then a good way to put it under a test is to create an end-to-end test for the entire application. Which you would do anyway, wouldn’t you?

Question 2: How well do you know your tools?

When I’m coding, I usually operate in one of two modes: the exploratory or productive one. They can be distinguished easily by asking the question stated in the title of this section.

If you know all the tools you need — libraries, frameworks, languages, APIs — you are in the productive mode. You can focus on the problem that you’re trying to solve, as opposed to figuring out how to make your tools fit into your solution.

Sometimes, however, you don’t have this comfort. Maybe you don’t really know the library that you are forced to work with. Maybe you need to learn a new language or a programming technique. These situations force you to switch to the exploratory mode: you are like a little child that gets to play with a new toy and tries to figure out what it can do and where it belongs. Does it move? Does it make sounds? What if I bash it against the floor for fifteen minutes, rub it against the doggy, and then put it into my mouth?

TDD is really efficient when you are comfortable with your toys.

However, it doesn’t support the exploratory mode equally well. That’s because in TDD, the cycle starts from enforcing your assumptions on the code you write. If you don’t know the tools behind your belt, you don’t really know what you can do with them in the first place. This is especially visible if you can’t use a given tool directly in tests, and you have to mock it instead. If you don’t know how the mocked component works, you are basically writing against blind assumptions, which rarely ends up well.

That being said, after you explore your possibilities, it’s still good to write tests for the code that you created. If you want to be perfectly sure about your code quality, you may even consider throwing out what you wrote as a prototype, and then start over using TDD.

Question 3: Is your interface well defined?

In the previous section, we discussed what to do if we don’t have clarity about the dependencies of your code. In this one, I’m going to consider the opposite: how the code that depends on your unit influences whether you should use TDD or not.

Let’s first think about how your component could possibly be shaped by the code that will depend on it. This external code probably shouldn’t have much to tell about your component’s implementation details. If it does, then you should take a deep breath. Can you feel that funny smell? Well, that’s your design. Go fix it.

Even if the code that depends on your component shouldn’t dictate its internals, it certainly should have influence on this component’s interface. Why, you say? After all, the authors of the libraries that you’re using on a daily basis didn’t ask you how to build their stuff.

And that’s precisely why it’s so damn hard to get it right.

Knowing how the “outside world” would use your component helps you to design this component’s interface in a way that makes it easy to work with. It’s true regardless if the “outside world” is just someone sitting next to you, or a large community of developers. Did I hear you say that it doesn’t matter, as long as your module’s implementation is fueled by your brilliant mind?

Yes, we’ve all been there. Including me. If you feel like this, you again need to take a deep breath. Do you feel… Oh, well, I think you get the point.

Treating your fellow coders well is like treating your users well: just like a good UX decreases the possibility that users will shoot their feet, well-designed interfaces do the same for your team. So what can we do in order to make sure that our interfaces actually make sense?

Apart from actually talking to other people, the best thing you can do is to try using the component before you even start implementing it. And that, in essence, is how test-driven development works. In other words, the less you are confident about how your interface should look like, the more likely you are to benefit from TDD.

Question 4: How important is your code?

Let’s be honest: good tests don’t come for free. In practice, writing them can easily double the amount of time that you need to spend working your code. This may seem like a waste of time, but it really pays off once your code gets important.

So when exactly does code become important? It’s obvious if you are writing something to be a part of a critical production service: trust me, it is important. On the other end of the spectrum, you may want to create a script that will serve as a Secret Santa generator for your department. It’s probably safe to assume that a tool that will be used once a year, and whose failure could at worst make you an owner of ten ugly Christmas sweaters, doesn’t deserve a comprehensive test suite.

OK, you know what? Maybe it does. [Accessibility description: Image depicts two men in hilarious Christmas sweaters]

Beware: there is a lot of gray area here. It’s very easy to believe that some piece of code is not important, only to rediscover it a couple of years later, mentioned in a post-mortem report from a production outage. That’s because programmers rarely verify whether something is well tested before they use it. And that’s why in practice, it’s good to use the following rule of thumb: if it gets submitted, it is important.

Conclusion

As you can see, there is no silver bullet, and no single correct answer to the question whether you should use test-driven development. In this article, I attempted to help you in making this decision. Once more, here are some signals that should encourage you to use TDD:

1. You are writing complex code. Boilerplate code rarely needs any tests.
2. You know your tools. Don’t bother with tests if you’re just exploring your possibilities.
3. You don’t know how your component’s interface should look like.
4. Your code is important.

Image credits:

• Main image: “Dilemma” by zeevveez (CC BY 2.0)
• Photo of a climber: Alicja Gancarz, used with author’s permission (all rights reserved)
• William of Ockham: Moscarlop (CC BY-SA 3.0), from Wikimedia Commons
• Llama unicorn: Amberbunting [CC BY-SA 3.0], from Wikimedia Commons; by conditions of the license, the derived work may be reused under the same conditions
• Christmas sweaters: Sgt. Quentin Johnson (https://www.dvidshub.net/image/804211) [Public domain], via Wikimedia Commons

Special thanks to:

• Julie Qiu, Ben Anderman, Jason Walker-Yung, Victor Quinn, and Mateusz Małochleb for valuable feedback;
• Alicja Gancarz for being an inspiring mentee.