The Dreyfus model applied to testing
Every now and then, I see some posts on the internet that say how unit testing is useless and actually counterproductive when you want to refactor. In their opinion coarser grained tests (integration and end to end) should be the only type of tests. I also have read posts that talk about how integration tests and end to end tests are hard to write, run and maintain. For this crowd, the test pyramid is sacrosanct.
I empathize with both camps. I have been in teams where the unit tests were hard to read and maintain (long tests with lots of mocks and “assert something was called”). It was genuinely hard to refactor code. I have also been in teams where we had hundreds or even thousands of end to end tests. These tests took a long time to run, were hard to write and even harder to fix.
As software developers, we make tradeoffs everyday. According to wikipedia
A tradeoff is a situational decision that involves diminishing or losing one quality, quantity or property of a set or design in return for gains in other aspects.
We decide if we have to refactor code now or later. We decide if we can use a library or hand roll something. The key factor when you make a tradeoff, is the perceived return on investment of one approach versus another. Some tradeoffs are easy to make. For example, I will choose to refactor if it takes five minutes of my time and makes code readable versus not doing it. Some are harder. Should I spend a couple of days refactoring when it takes an hour to push a hack and add value to my customers quickly ? Then, there are tradeoffs which we don’t make but let “best practices” make it for us. We test drive our code because it is “best practice”. Do we think about, how coupled to the internal implementation our tests are ? We write a lot of end to end tests because we believed someone on the internet who said that you can refactor your code with ease. Do we think about how much these tests affect our build times and how hard they are to change ?
The Dreyfus model of skill acquisition talks of about five stages. Novice, advanced beginner, competent, proficient and expert. I like to visualize this as spectrum versus strict categories. You don’t suddenly change stages one fine day. It is a gradual transition.
If you are new to software development (a novice/advanced beginner on the spectrum), you will dogmatically test drive your code and be religious about unit tests and the test pyramid. Then one day, you will realize how much time you spend on this approach and you will start writing more integration/end to end tests because you are now “troubleshooting”. This is when you are hovering around the “competent” part of the spectrum. On another day, you will realize that there is “no one way that works”. You will think about the design and type of tests (the tradeoff) before you even start programming and you will do it more frequently. This is when you can be considered to be on the “proficient” part of the spectrum. If you are on the expert band of the spectrum, you will intuitively know what kind of test and design works for a problem given the context. You will make tradeoffs subconsciously and swiftly. Sometimes, it will take your conscious brain some time to explain why you made a particular decision.
In an ideal world, it is desirable to have only experts in your team. In the real world though, human life is limited and we need to pass on your knowledge and skills to other people. Therefore, novices advanced beginners and competents will exist forever and so will posts that complain about one approach to testing and praise another.
