How Automated Testing Can Actually Reduce Development Costs
In this article, we will run through a real-life example where automated testing could have prevented huge losses.
Automated testing is a set of tools that sit alongside your everyday development ecosystem, and perform operations that either simulate or replicate a user’s journey through your program. Often these tools produce error/success reports that can be used to provide a high-level snapshot of the status-quo across your entire application.
Some tools (pickles, cucumber, gherkin) can even bring non-technical team members into the picture, plugging them into the development workflow, and giving them access to up-to-date reports generated by your technical staff’s ongoing work.
How much does automated testing cost?
The question you really should be asking, is: “how much is not having automated testing costing me?” It’s true, there are some up-front costs to writing unit tests as it adds to the workload of the development team. However, the benefits and long-term savings will prove to be worth every penny. In this article, we will run through a real-life example where automated testing could have prevented huge losses.
It doesn’t have to be expensive
One of the biggest qualms in opposition to automated testing is the idea that it’s more code you have to write, on an already tight deadline. The reality is, you can put as little or as much into your test suite as you want.
You will get exactly the amount of benefits from your testing setup that you put into it. Even a little can save a tester hours of time otherwise spent verifying and reverifying tiny little details over and over again.
For example, are you building a web app? Start by writing test that helps you verify if each screen has rendered. This will catch a hundred little dumb bugs right up front. This is what we call low-hanging fruit.
You don’t have to test every tiny little detail, and no one who knows anything about this subject should ever say automated testing can replace your QA team. Instead, it’s an important tool to help your QA team do what they love; find weird edge-cases and eliminate them, improving the experience for as many users as possible. At the very least, you can reap the benefits of automated testing without investing thousands of extra dollars into your project’s budget.
What are the benefits of automated testing?
Stop bugs before they get to production
The main goal and purpose of automated testing is to keep production safe and stable. Automated tests can behave as a gatekeeper, stopping bugs before they get into a production environment. Let’s take a look at a real-life example:
Your application is in production, and the development team is working on a new feature set that consists of 5 requirements, each needing about 4 hours of development work. You pay, on average, $100 / hour for development. All requirements get developed, the developers eye ball it to make sure it works, and deploy it to production.
But what if just one of those requirements, releases a bug into production. Now we start to see the costs sky rocket. The next day, a customer finds the bug and reports it to your company.
Customer Goodwill / Faith in the product or service
At this point, your customer has lost something that you will never be able to get back — faith in the integrity of your product. Every customer will have a different level of good will to give to a product, and will lose a different level when they see minor to severe bugs production.
For our example, let’s assume the customer is upset, but hasn’t lost enough good will to leave the product yet.
At this point, your development team will be under stress to quickly patch fix the bug and get it out to production as fast as possible.
They spend one hour debugging the problem, another to develop a solution to fix the bug, and a half hour deploying it to production.
Software Regression
Great! The bug has been fixed. But there is still a problem. Developers under stressful conditions make mistakes. While they were able to fix the bug, they accidentally introduced a new one, on a different part of the system. This is a common problem in software development called Software Regression.
Unfortunately, the customer found this new bug in production. This time, they no longer have any good will to give. Without any faith in the integrity of the product, the customer decides to exit their contract, and take their money elsewhere.
This is a devastating blow to the product’s income, and you still have to fix the new bug for other customers. The developers fix the system again, this time without introducing new problems.
So now, for a small feature set that was originally supposed to cost $2050, you have spent $2550 total. That’s an additional 25% on production fixes, and you are left with one customer less.
Adding Automated Testing
Sadly, this is not a rare scenario in software development. No doubt you have experienced this at some point on a project. But let’s go through the same example, this time including automated testing from the beginning.
On average, it might take a developer an extra half-hour per requirement to write a unit test. This obviously depends on the complexity of the requirement.
So our feature set of 5 requirements will now look something like this:
It looks like a large expense at first. However, before the developer went to deploy the code with the original bug to production, he was blocked from doing so, and given a clear reason as to where and why the bug exists. No he can easily fix the bug as part of the original development costs, before the customer could even have the chance to see it in production, and saves a code deploy.
But remember, in our example the developer created a new software regression bug by fixing the original bug. Again, automated tests would alert the developer that he just broke another part of the system, and not allow him to release it to production. So he fixes the regression issue as part of the initial development costs, not later under stressful conditions.
The developer can now confidently, and safely deploy this code to production once, knowing that he did not introduce any bugs, and the requirements are met.
So total our initial costs of development were higher at $2300, but there was $0 spent on production bug fixes, or multiple deployments for patch fixes. But most important of all, the customer never saw a single production issue, has full faith in the system, and is here to stay.
Right away we can see the monetary benefits of including automated testing even in just this small example. The savings will be exponentially higher the bigger and more complex a product is. However there are other benefits that can prove to be invaluable.
✔ Customer Good Will is still in-tact.
✔ Regression Issues no longer exist
Developer Confidence
An added benefit of automated testing is that it adds to a healthy environment for your development team. Less production bugs means less phone calls in the middle of the night, less high stress rushed patch fixes, and less worrying if their code will work or not. A team that is not scared to deploy code to production, is a team that can work efficiently and effectively.
QA testers can focus on higher level workflows and functionality
With core business logic tested automatically and regularly, QA testers no longer need to waste time testing the nitty gritty of a product. Instead they can focus on higher level functionality, or test intricate workflows that the development team might not have thought of. This not only will save money on QA time, but might help to find bigger problems before they reach production.
Peace of mind
As a product owner, you can sleep soundly at night knowing that you have a gatekeeper keeping bad code out of production. The chances of you getting a call from an upset customer are drastically reduced.
In addition, automated testing gives you a window into how well the development team is working. Are they often writing code that breaks tests? Are they struggling on a particular part of the system? Being able to see these metrics can help you fine-tune your development team turning them into a well-oiled machine.
Production code is backed by logical proof that requirements are met
At anytime you can look at a list of requirements and see that not only have they been developed and completed, but in real-time, as they are being developed, you can be assured that they work to the specifications given.
Conclusion
The benefits are invaluable
We are living in a time where good software can make or break business. Simply put, we can no longer afford to go without automated testing. The small, simple integration of automated testing into your development workflow benefits, not just the product itself, but the entire development team, QA testers, and product owners. How much is peace of mind worth to you?
Training for your team
We would love to help get your automated testing pipeline set up, and development team trained on best practices today. Send us an email to get started.
