Why is it so common in software engineering to have so many bugs?

Let's start with this article: Enterprise Software is the hardest to develop.

There are several kinds of software defects, which is the actual name of ‘bugs’, and all of them have completely different causes.

First, let start with the technical defects. These ones are common among new and less experienced developers. For example, let say you need an application that sums two integer numbers and the requirements clearly indicate that, then when your developer delivers the software these things happen:

  1. You can enter letters or any kind of characters: decimal numbers, improper numeric format, very large numbers out of the computer boundaries. What happened, in this case, is that the developer didn’t validate the input. This is usually the number one technical defect and the most neglected. There is a saying in software development: All input is evil until proved otherwise. The first line of defense against defects is doing proper data validation. Sanitize: Good for Beer, Good for Data — DZone Big Data
  2. The application crashes, stops working, or shows an error. The best way to explain what is going on is that the developer is leaving loose ends on the code, so that the code’s quality is poor. Part of the software design should include “Error Handling Strategies” so that won’t happen.

Technical defects are the easy ones and should never leak out to the production environment. If these kind of defects are happening in your production environment, you desperately need software testers that prevent that. If you have software testers, they are doing a very poor job.

Usually, technical defects are transient. With a good strategy, training, and followup, they tend to disappear over time or happen less often. Over time, developers should gain experience on how to prevent them and testers on how to detect them before hitting the production environment. If not, your team is only focusing on writing code but not learning from their mistakes nor improving the deliverable code quality.

Second, business rules defects. These are the difficult ones, the ones that make the project fail or take a lot more time than expected to be completed. They are poor quality requirements. Would you think it is fair if somebody asked you for a blue ball and when you bring a blue ball then he says “It is blue, but I need navy blue and that is not navy blue”? Or even worse, they say “you know what, it was actually a red one we needed”. Or “after trying to fit the ball in place we see that we actually needed a cube”.

The problem with business rules is that requirements gathering is not as simple as they seem and they tend to be very neglected (Again with this link: Enterprise Software is the hardest to develop). It takes time from several roles to properly get the requirements from the users and shareholders, create stories or use cases (pick your preferred weapon), develop the software based on those requirements, test that the software meets the requirements, and deploy to production. It’s an expensive thing when the user finds out that the software is well done but it doesn’t do what is needed. Basically, all the work being done is converted to waste and it needs to start over. (It is a lot easier to create a cube from scratch than converting a ball to a cube).

It is a common scenario that teams: 1) don’t have product owners or business analysts in which lies this responsibility, 2) the analysis is done superficially in a high-level view (or as some others call low-resolution requirements) leaving the developers to get the details, 3) the developers during the development phase need to do requirements analysis. All these are the perfect ingredients to make a lot of business defects.

The developers in the development phase don’t have a way to tell if they are working on a defective software. So, having a defect inthe requirements is a silent ticking time bomb. Proper requirements gathering and analysis is crucial. Your best scenario is that during the user acceptance test (UAT), where the user is supposed to review the software against the requirements to verify that everything has been developed, catch the defects before they are going to be deployed.

From each development phase that the software development goes through without being noticed, the more expensive it gets. Formal methodologies (or as everybody knows them, cascades type) put in place proper controls to catch these kinds of defects before they start in the development phase.

I will recommend these two books that address the issue more broadly: Software Requirements (Developer Best Practices) 3, Karl Wiegers, Joy Beatty, eBook — Amazon.com and the book Exploring Requirements: Quality Before Design: Donald C. Gause, Gerald M. Weinberg: 9780932633132: Amazon.com: Books

Part of the project manager/team lead job is to have a separate followup list of any issue arisen in the production environment (not only software defects) for two reasons: 1) to make sure any issue/defect is not lost in the work pile and that they are properly followed up and resolved as soon as possible, 2) to do an issue/defects analysis and create strategies like developers and testers training or implement proper processes.

So, as you can see, it is not a single cause, but rather several ones, that without proper training and management can lead to delivering defective software. They all need to be addressed separately and should never be neglected.

Photo by NESA by Makers on Unsplash