TDD or not — what do the experts say?
Following up on my previous rant about TDD, I’d like to do a bit of a summary of a debate within the software engineering community that took place after Coplien’s original rant from 2014 “Why most unit testing is waste”:
- David Heinemeier Hansson (the original author of Ruby On Rails) observed peer pressure towards TDD and confesses to not doing TDD all the time: TDD is dead. Long live testing. I agree that in the early days the TDD movement served as a wake-up call to do more automated testing, but it clearly has run its course
- In the wake of Heinemeier Hansson’s post, Kent Beck (creator of Extreme programming) posted a sarcastic article on Facebook titled RIP TDD in which he provides a bucket list of TDD benefits for which he now needs to find “new techniques to help me solve many of my problems during programming”
- Heinemeier Hansson posted another article titled Test-incuded design damage, where he argues that optimizing an architecture for unit testing can make code harder to understand
- Software engineering gurus Martin Fowler and Kent Beck then summoned Heinemeier Hansson to a hangout discussion
- Ian Sommerville (author of the Software Engineering text book) comes to the conclusion that TDD ends up being a useless chore if you try to apply it everywhere: Giving up on test-first development. He is making the case that GUI code is hard to unit test because such tests need to be rewritten quite often
- Uncle Bob Martin (whose writing about clean code and architecture I strongly recommend) followed up with a reply to Sommerville, who in return replied with another post (the discussion continues below the article). Bob argues that Sommerville is doing it wrong if tests need to be rewritten all the time, and suggests that Sommervilles tests must be too tightly coupled to the production code.
- Vitaliy Pisarev (The No1 unit testing best practice: Stop doing it) also observed unit tests making his code actually harder to change because they were brittle, and has found that system tests are more efficient. Coplien shows up in the discussion defending Pisarev with some snarky comments xD
- Henrik Warne posted a refutal of Coplien’s original article called A Response to “Why Most Unit Testing is Waste” (09/2014). Coplien replies in the comments and things get a little heated: He convinces Warne to change some parts of the article as he admits to having misrepresented Coplien’s point of view, but a Japanese blogger had already translated Warne’s post and Coplien demands Warne to contact said blogger.
- Here someone else did a probably better attempt to summarize the debate that mostly happened in 2014.
- In a podcast (starting at 42:20) from 2009 Joel Spoelsky (Joel on software) and Jeff Atwood (Coding Horror) question the useful of units tests:
Joel: There’s a debate over Test Driven Development… should you have unit tests for everything, that kind of stuff…
And that strikes me as being just a little bit too doctrinaire about something that you may not need. Like, the whole idea of agile programming is not to do things before you need them, but to page-fault them in as needed.
But the real problem with unit tests as I’ve discovered is that the type of changes that you tend to make as code evolves tend to break a constant percentage of your unit tests. Sometimes you will make a change to your code that, somehow, breaks 10% of your unit tests. Intentionally. Because you’ve changed the design of something… you’ve moved a menu, and now everything that relied on that menu being there… the menu is now elsewhere. And so all those tests now break. And you have to be able to go in and recreate those tests to reflect the new reality of the code.
So the end result is that, as your project gets bigger and bigger, if you really have a lot of unit tests, the amount of investment you’ll have to make in maintaining those unit tests, keeping them up-to-date and keeping them passing, starts to become disproportional to the amount of benefit that you get out of them.
Jeff: … to me that’s what it’s all about. Anything that gets in the way of you fixing your code or improving your code in a way that your customers can appreciate, is a negative. If that means using Ruby, or having lots of unit tests: whatever’s working for you: do that. But if it’s getting in the way, if it becomes friction, like, “I’d love to have this great new feature but I’d have to have 1000 unit tests,” that’s a negative.
Joel: Yeah. And the worst thing that happens is that you get people that just stop thinking about what they’re doing. “This is the principle, to always write unit tests, so I’m always going to write unit tests,” and then they’re just not thinking about how they’re spending their time, and they wind up wasting a lot of it.
I couldn’t agree more. Writing unit tests for everything means writing and maintaining twice as much code — how that can be worth the time is beyond me — unless you’re exremely sloppy and make that many errors while coding (maybe because you’re not thinking?). And I just remembered that Atwood once wrote some very wise words about avoiding code on codinghorror.com:
the best code is no code at all. Every new line of code you willingly bring into the world is code that has to be debugged, code that has to be read and understood, code that has to be supported. Every time you write new code, you should do so reluctantly, under duress, because you completely exhausted all your other options. Code is only our enemy because there are so many of us programmers writing so damn much of it. If you can’t get away with no code, the next best thing is to start with brevity.
The pro TDD camp however will argue more tests are always better, because it improves the quality of the production code.
The takeaway is that the software engineering community is divided on the question wether TDD is a practice to be applied universally to all programming problems or wether its usefulness is limited to certain unit-testable problem domains. Some people prefer either upfront design, simplicity, exploratory development, or like to weigh benefits against costs, while others seem to have an unshakable faith in TDD as a the one true way and won’t trust a line of code that hasn’t been unit tested.
My theory for this chasm is that people like Uncle Bob, Martin Fowler and Kent Beck maybe live in a different world from us app developers, somewhere between academia and big enterprise, where development cycles are long and code has to be really correct and reliable. Or they are mostly involved in layers of the software that is far away from any user interface. I don’t know. But In my day to day work as app developer however, requirements and UX designs change frequently and clients demand fast delivery. So time efficiency is really important, and from my experience, unit tests for the kind of code I write are not worth the time I spend on them.