Preventing Software Bugs from Ever Occurring
Trying to fight bugs is exhausting and frustrating. But if we study their nature, we can understand which preventative measures will be most effective in saving us from the pain.
As a kid, I remember being frustrated by a fly that was keeping me awake in the night. It was one of those huge monstrous ones that made an electrifying buzzing sound as it flew around the room.
I recall my dad teaching me a little trick. He told me to turn the light off in my room, and switch on the light in the hallway. The fly was attracted to the light in the hallway and flew towards it. He then told me to open the front door, turn the hallway lights off. The fly duly flew towards the brighter street light, allowing me to get back to sleep.
What really happened was that my dad understood the nature of bugs. He knew they were attracted to light. So rather than teaching me to fight the bug, he taught me to control the lights which controlled the bugs.
I learnt a big lesson that day:
If you understand bugs, there is no need to fight them.
Nowadays I use the same technique to prevent all sorts of bugs in the bedroom. When mosquito are around, I turn on the hallway light for 30 minutes before bedtime, and I am bite free in the morning.
As Benjamin Franklin once said:
“An ounce of prevention is worth a pound of cure.”
But What about Bugs in Software?
What if you could understand the nature of software bugs, and therefore understand how to prevent them? Is there a “light switch” trick for software bugs?
Turns out, there are only 3 types of bugs. Yes, you read that correctly, 3! Each has a “light switch” trick, which I like to call the highest impact quality function. In this article I will discuss a model that I use to classify these 3 types and what these quality functions are.
Bug Type 1: Implementation Defect
The specification was correct, but the implementation was defective.
At first glance, this might look like the most common type of bug, but it’s actually quite the opposite! As this study shows, only 15% of defects occur at the implementation stage. However, the study does not consider regression bugs, and in my model of bug classification, I consider regression bugs to be incorrect implementation. Even still, this type of bug is the least common of the three in my experience.
Bugs lurk in large and complex codebases that are full of cobwebs. The industry average is 15–50 bugs per 1000 lines of code! Even the most skilled developers can introduce bugs as they can’t see the impact of their changes.
“Smelly Code” is considered an indicator of weaknesses in the code design that are likely to introduce bugs. Some of the factors that contribute to this are management pressure to go faster, laziness, and above all — ignorance.
Timelines get tight and teams need to release implementations quickly. As a result, testing (manual or automated) is skipped or hurried, which means bugs that spawned at development time can casually stroll into production.
Highest Impact Quality Function
Test Driven Development
This practice ensures that code which works as specified today, will continue to work as specified in the future as others maintain it, making it the most effective prevention to Regressions.
All of the above benefits are known to reduce defects between 40–90%. That’s HUGE! This makes TDD the highest impact quality function to prevent implementation bugs.
What baffles me is this:
How is TDD not a standard-issue weapon for all developers today?!
Should you TDD EVERYTHING? Not really, TDD should not be a dogma. You need to consider the opportunity cost and your team dynamics.
If you’re not already doing it, then start using TDD to clean up your code and the benefits will fall into place over time. Just like with the light switch trick, you have to be patient!
Type 2: Incorrect Specification
The specification was incorrect, therefore the implementation is incorrect.
A Cambridge University study shows that 50% of defects occur in specifications. The same message is resonated in other studies. Mistakes made and discovered later in the cycle can cost 50–100x more than if they are discovered earlier. Despite being the largest source of bugs by far, specifications still go into development unchallenged. What gives?!
I mentioned above that “specifications are the communication of assumed value”. When the specifications don’t contain details that are known to the author, then bugs can occur due to misunderstanding.
Sometimes new specifications are written that conflict with older specifications. When this happens, a bug will magically appear at the development stage (if you’re lucky), or in production.
Highest Impact Quality Function
Behaviour Driven Development
Behaviour Driven Development (BDD) is a team-wide development methodology, with a heavy emphasis on creating a shared understanding between business people and technical people.
BDD starts with deliberate discovery sessions that involves people from all disciplines. Together they flesh out detailed scenarios using exercises such as Example Mapping from Matt Wynne. The output of such exercises leads to a deeper understanding of the domain, and also to creating Specification by Example documents that express the system behaviour. These exercises and artifacts are very effective at addressing the Inadequate Communication problem.
Developers use these specifications to model the domain model using a technique called Modeling by Example, or Event Storming from Alberto Brandolini. In turn, a ubiquitous domain language forms which speeds up development by reducing the translation cost from requirements to code. The ubiquitous domain language is excellent at preventing the Inconsistency of specifications.
What’s awesome is that the very specification document becomes the actual automated test. I encourage you to have a look at our Chimp.js open source library that we wrote at Xolv.io to streamline this process. Chimp.js builds on Julien Biezemans↯’s incredible Cucumber.js project.
What does all of this add up to?
Wait, what?! 2%?! If this methodology is not the highest impact quality function to prevent specification bugs, I don’t know what is!
BDD is quite intuitive and you’re probably doing parts of it already. It doesn’t force you to draw thousands of UML diagrams, and is pretty low friction to adopt.
A word of caution however. BDD is a very misunderstood practice. Many people think it’s just a testing approach but it’s far more than that. I would encourage you to read Aslak Hellesøy’s view of BDD.
If you’re confused on where to start, then just start by having conversations in deliberate discovery sessions. These are the most important aspects of BDD. If you do one thing only from BDD, it should be to have these conversations.
Do this for a few releases and see with your own eyes the number of defects reduce. You might even get some stress relief and enjoy it!
Bug Type 3: Missing Specifications
The specification was missing, therefore the implementation is missing.
Simply put, these are the scenarios you never imagined when designing your product. They are otherwise referred to as the “unknown” type. Their impact is unpredictable and can range from embarrassing to catastrophic.
You’ve probably heard the phrase that that when you assume, you make an ass out of you and me. Assumed value is no different and can come with tricky nuances. When these are not researched, an unknown bug hatches!
Someone realizes the “implementation is wrong” at release time, even though the expected behavior was not specified! Sound familiar? These bugs come into existence due to the lack 0f specifications.
Highest Impact Quality Function
Behaviour Driven Development
Huh? Again? Yep!
The deliberate discovery stage of BDD is a great way to use the wisdom of crowds and identify unknowns. The discovery stage is not unidirectional but rather, it’s iterative. It is intended to create questions as well as answers. Some of these questions require further research to feed back to the team. This cycle of interactions is an excellent way to deliberately discover unknown specifications.
Summing it up
For the foreseeable future, as long as we create software, we will inadvertently create some bugs. But just my old man can share with me a proven technique to save my night’s sleep, so we can learn some great techniques to save us a lifetime of fighting bugs.
The trick is not to think about how to fight them, it is to understand how to never invite them into a fight in the first place.
“The greatest victory is that which requires no battle.”
― Sun Tzu, The Art of War
I will be writing more on other quality functions that you can put in place to increase the quality of your product.
If this theme and my comments resonate with you, please let me know by clicking that heart icon and recommending the article to expand the reach of this message.
Become a Quality Ninja!
If you would like you and/or your team to learn how to implement a quality-first practice, and how this can benefit you, check out our new book “Quality Faster”
This digital knowledge resource combines both theory and practical tutorials to provide everything you need to improve quality (and therefore flow too) across your full stack, and entire team. We start from the very foundations of quality, value and software delivery principles, and apply these to forming your own team process and test strategy. These are then executed through specific code samples for Node.JS, React, Chimp JS, Mocha, Meteor and many more.
Join our mission
We at Xolv.io are out to fix the problem of software bugs.
If, like me, you are passionate about quality, click the button below to follow me here on Medium and here on Twitter, where I will update you on our progress. You can also contact me or comment below if you have any thoughts or ideas.
I look forward to helping you deliver higher quality software, faster.
Other articles I’ve written:
Quality is one of the greatest things in life, but it’s ephemeral and hard to define. In a similar way to how “acts of…medium.com
We have an industry-wide blindspot for the most important metric in software, and now is the time to change that.medium.com