Dr. Codelove or: how I learned to stop worrying and love TDD
By Titus Nachbauer
If you want to improve your life as a developer, you often need to learn new ways of doing things. A new discipline or way of working can be a gamechanger. One of these disciplines is Test-Driven Development (TDD). It can speed up your work, while also making it more fun and helping you build better code that is easy to understand for others.
So why aren’t we all doing this? Maybe because we have not seen the benefits for ourselves yet. Or maybe we used to see testing as something for testers? Myself, I just was not aware of the amazing benefits until I had truly tried it out. Therefore, I want to share my personal journey and show you how I went from constant debugging to loving TDD.
How it all began
A few years ago, I was working as a migration consultant. The work involved writing throw-away scripts for one-off data migrations. We wrote a lot of scripts for customers that would crawl a website to extract data. When we thought we were finished, we would run them, go grab lunch and come back half an hour later to see an error message in the logs, because certain pages were different or simply because we had a silly regular expression that was not working. We would debug the error and run the scripts again. This all seemed fine and natural to us, even though we did spend quite a lot of time debugging.
Sometimes the work was a bit more tedious if we would have to work remotely using a remote desktop connection, especially if the client’s system was situated on a different continent. Next, we found that we wanted to start reusing some code. After all, each crawler would have the same basics, with some configuration and logic on top. As a result, we started to create libraries of commonly used functions. This code was very different from the code we would write for a project. Suddenly it mattered that someone else could understand and debug the function you wrote. We tried establishing some coding guidelines. And all was well for a while.
However, as time went by, our libraries grew bigger, the number of projects grew, and we got more consultants who needed to be able to switch projects on short notice. Now we encountered a few problems: some people had their own versions of the libraries and they were not sharing updates. This meant that the code was dependent on their version.
Why Write Better Code if it Works?
Despite our efforts, we were unable to keep libraries in sync, and given the time pressure in most projects, it was understandable that people would not give much about cleaning up their code. We tried creating frameworks and we even had our own domain-specific language, but we never found a way to improve code quality and readability much. We had tried creating frameworks and some rudimentary unit tests, but all to no avail.
Fast-forward a few years and I was the release manager of our own open-source scripting language, written in Java. Since this was shared outside our organization we did, of course, write unit tests and integration tests. And we had a pipeline that would run them before building a release package. My only problem as a release manager was that these tests did not cover all the use cases that the consultants would come up with. When you are developing a scripting language it is going to have a lot of edge cases that will pop up when you least expect them.
What was worse: we might upgrade a Java library and that would break existing projects. We tried to tackle this by doing exploratory testing and by using semantic versioning. This helped a lot, but it slowed down our process. Also, it would leave consultants stuck on a certain version, not being able to use newer features.
Before long there was a serious backlog of features that the consultants really needed, but that we could not deliver. I tried pushing people to write more and better-automated tests, but since these were added after the fact, no one was very interested in this task. It seemed like boring housekeeping to the developers. I tried mandating a certain percentage of coverage using SonarQube, but that only led some developers to deliver unit tests without proper assertions. Sometimes all they did was cover the code, but there was no way of knowing if the tests would properly test what needed testing.
Test-Driven Development to the Rescue
There were a few developers who were different. One was a young guy, 22 at the time, who would always deliver properly tested code. When I asked him how he did that he mumbled something about Test-Driven Development and went on coding. I kept pushing a bit and finally when it was time to develop a new plugin for the software, we sat down together to develop it in a pair programming fashion. He showed me how he would write a test first, run it to see it failed, and then create the code that would satisfy the test. After that, he would rewrite the code to make it more readable and maintainable and run the test a third time to see everything was still fine. This was very elegant and impressive, and I remember being left in awe of this super simple but very effective way of working.
Not only did he deliver code that worked, but it was also clean code because he could refactor it without any stress. After all, he could simply run the tests again and it would still show that the code was working. Also, other developers were able to read his tests and understand how the code worked. It took me a few more years to realize what the biggest advantage of what he was doing was: as a team, we would never have to worry about this part of the code being broken in a release because it was 100% covered by proper unit tests. After all, he had not written any functionality or logic without writing the test first.
Later I learned that what he was doing was the classic loop known as Red, Green, Refactor.
- Red: Start by building a simple test that asserts one thing and makes it fail.
- Green: Next write the production code that makes this test (and all previous tests) pass.
- Refactor: Re-arrange the code so it adheres to coding standards like SOLID and Clean Code. Check that the tests still pass.
It is important to understand that while most of the time only the three steps above are mentioned, there is a fourth step in this loop:
4. Carefully think about your next step, considering any design/architecture that may be needed. This was pointed out by Sandro Mancuso at Devternity 2018.
When your application is simple, you do not need to think very long, but in practice, you may find that sometimes a bit more effort is needed.
This experience made me rethink how to approach development work. I was unable, however, to immediately follow in this developer’s footsteps and apply TDD to my daily work. We could have benefitted greatly from promoting this practice, but it was seen as a low priority. I personally got distracted by learning Docker, AWS, and Azure DevOps and ended up not thinking much about Test-Driven Development for a while
Joining the Coaches
A few years later I joined ABN AMRO and the CICD Enablers team. I quickly noticed that some of my teammates were extremely experienced developers. They all talked about Test-Driven Development as one of the core practices of good programming. They also acknowledged that it can be hard to learn. Now I was extra motivated to start incorporating it more into my own daily work, but I still couldn’t find the time to do it regularly. Also, I missed a simple exercise to get started.
Then the corona pandemic came and changed all our lives. Suddenly we were working from home and not traveling anymore. My personal life also changed greatly with the gift of our little baby daughter. During long feeding sessions and when holding our sleeping baby, I started watching conference talks on YouTube. I finally encountered a great series of talks by Robert Martin, also known as Uncle Bob.
Uncle Bob showed the basics of TDD in a demo and explained those same advantages that I had seen a few years earlier. The main point is the confidence that you can achieve with good test coverage. Being sure that if the tests pass, you can safely release. Uncle Bob was the one who made me realize how important this high level of confidence really is. I had seen it work in practice, and as a DevOps consultant and CICD coach I had been saying this for years. But, seeing this demo was still an eye-opener, because it showed a way forward that could help teams improve their software delivery.
Practice by Doing
In the evenings I started doing so-called code katas: small exercises that you can use to learn things like Test-Driven Development in practice. The idea is to learn through repetition. For me, that didn’t mean repeating the same kata over and over again, but it meant doing a few of them per week. Within a few days, I felt I got a much better understanding of the basics of Test-Driven Development. Not just the concept, but developing in very small steps, adding tests before the production code.
What’s more, I enjoyed the process and found that once you wrap your mind around it, using TDD is faster than just “guessing” whether the code you wrote works. It is faster and more fun than trying to get proper test coverage after the fact. It is faster, because there are fewer steps that could go wrong and, more importantly, if you make a mistake, you always know where it was because you only wrote a small amount of code. If you commit to git every time you have working tests, it also allows you to easily roll back to the last working version.
Reaping the Benefits
As a result, I learned not just how to improve my test coverage and the quality of my tests, but in the process, I also learned how to refactor the code after writing, to make it readable and maintainable. This is a very important part of test-driven development as Uncle Bob explains: everyone writes bad code the first time around because that is how our brains work. We are not done when the tests pass, we are done when the code looks good, and the tests still pass. Thanks to those passing tests the refactoring becomes super easy and it is fun to deliver good-looking code fast.
Additionally, doing the katas taught me to:
- get the tests running faster locally (it is essential that they run in less than 10 seconds)
- get faster at using the IDE by using shortcuts and automated refactoring
- try out coding design patterns safely and quickly
- splitting my work into small commits
I am still practicing, sometimes together with colleagues. It is fun and a lightweight way of greatly improving my speed and accuracy. I still need to think when I write the test: what would be the next small thing to test, what is the easiest way to implement this. But I do not have to go into long debugging sessions anymore.
And that is how I learned to stop worrying about the quality of my code and to start loving Test-Driven Development (TDD). I believe our organization could benefit greatly by adopting this more. We could have smaller changes, better code coverage, and better confidence in new code.
Improving these things would lead to shorter delivery times and better stability, isn’t that what we all want? This change must be driven by developers because it is a personal choice to work like this.
Please ask your questions in the comments below and share your own experiences with TDD.
How to Get Started?
So how to start learning TDD? Best start by reading a book on TDD or simply watching some demos on YouTube (Uncle Bob explains it really well). Next, it is very important to try several katas yourself. Coding Dojo is a good source for katas of various difficulties. Also please visit Emily Bache’s Github for fantastic exercises. Of course Googling “TDD Kata” will also help you.
It is important to realize that TDD is a discipline and, as such, requires a lot of practice. You may still decide it does not work for you, but give it a chance and train yourself before you start using it on your code at work. Many experienced developers have made the leap before you.
What you may find is that there are many situations that lead to questions:
- How do I handle external dependencies (I/O, databases, shared resources)? (Spoiler: you make your logic independently testable)
- How do I test a front end? (Spoiler: you make your front end so simple that you can unit test the behavior)
- When do I really design anything? (Spoiler: you could use DDD and Hexagonal Architecture when needed)
- How do I keep my tests running fast if the application gets bigger? (Spoiler: you only test the logic, which is fast)
- Do I need to write separate unit tests for Getters and Setters? (Spoiler: No)
There are good answers to these questions, which are interesting to go into, once you have made the first steps. Do not let these get in the way of learning to unit test your code before you write it. First, get into the wonderful cadence of Red -> Green -> Refactor -> Think and enjoy some of the benefits.
About the author:
Titus Nachbauer, CI/CD Enabler
Titus is a technology enthusiast and DevOps/CICD expert. He started programming at the age of 5 and has worked as a trainer and coach since 1999. At ABN AMRO he helps teams build software better and faster while complying with the rules and regulations of a large enterprise.
