Is there a necessary skill set for TDD?
In social sciences they say that the existing differences among the participants to an experiment is as important as the difference due to the treatment.
In software engineering the differences are usually the existing skills and experience, in particular software development skills.
Quoting Steve Freeman, “no tool nor technique can survive inadequately trained developers”. This is evident for a technique like TDD.
So lately I have been interested in studying the people’s skill factor, and set part of my research agenda in understanding what are the necessary skills that can make the best out of TDD.
If you read the Beck’s definition of the TDD process, you’ll realize that there are at least two high-level abilities one needs to have to complete a TDD cycle: she should be able to write unit-tests, and she should be able to develop. Someone might disagree, as I have considered refactoring, the third step of TDD, part of development ability. In hindsight this was a mistake.
Nevertheless, those two abilities should be taken together in the TDD way of developing software, so the third skill I set upon is the ability to follow the TDD process.
I used really sophisticated methods to calculate the development and testing ability, aka Likert questionnaires :-). I truly wish this was published before.
For the process metric we used a really cool Eclipse plugin which I recommend to anyone interested in visualizing their TDD flow.
Me and team collected data about the aforementioned three skills of 30 professional software developers here in Finland, we clustered them and compared the results in terms of the bugginess of the software they produced, as well as their productivity. The sample size was limited, and as one might expect, the results were not statistically significant, but the effect size gives some hope for future work. In fact, it seems that just focusing on these skills can bring around 30% improvement in removing bugs, and 40% in productivity.
Reflecting on the experience of this study I came to realize how much I have underestimated the people factor of TDD, and focused on the process factor. The big realization was that too many studies, at least in the Agile research community do not pay enough attention to people’s skills and experience, and even less on soft traits like emotions, affections and biases. And the lack of tools to measure developers’ skills in a standardized way is a symptom. Regarding specifically TDD, I am coming to realize that the different abilities with refactoring might have an huge impact of TDD, at least on the short term; but then again I wasn’t able to find an accepted tool to measure someone refactoring skill.
On a final note, I was giddy when I found out that journals accepted non significant results, and that the reviewers where engaged in discussing effect sizes, rather than p-values.
Originally published at dfucci.co on October 29, 2015.