Are You Thinking About QA Costs the Wrong Way?

Why It’s Time to Change Your Perspective on QA Costs, Plus Top Tips to Maximize Your Spend

If you’re a CIO in the digital age, you know at least one thing to be true: QA costs are on the rise.

And a look at the numbers just goes to confirm what you already know: Capgemini predicts that QA and testing will reach a full 40% of IT spend by 2018, up from a mere 18% in 2012.

Inside the Rising QA Costs

With a 122% budget increase in six years, we’re left wondering — what’s behind these rising QA costs? In short, it’s a greater emphasis on quality. We’re now spending more money than ever to ensure quality, as evidenced by Capgemini’s finding that security (80%) and customer experience (78%) rank as the top two priorities for QA and testing executives.

More specifically, four key trends that support a higher quality end result have contributed to thesesoftware testing budget increases:

• Growth of multi-channel business: Ten years ago, we developed software only with the desktop user in mind. Today, that couldn’t be further from the truth. We now have to develop software that works for the web, mobile and any number of emerging platforms, and we need to test each of those platforms and ensure consistency among them. That requires more spend and time.
• Rise of Agile: Organizations using an Agile methodology have reached 95% according to VersionOne, and taking on an Agile methodology requires more frequent testing. Often times, this testing also needs to be completed quickly and on short notice, which can increase the associated costs.
• Trend toward onshoring: Alongside the rise of Agile comes the use of more onshore resources, since this methodology requires developers and testers to collaborate more closely. However, onshore resources tend to be more costly than their offshore counterparts.
• Increased automation: Increased automation is another side effect of the rise of Agile as well as a contributor to rising QA costs. That’s because automated testing requires higher quality resources and more expensive tools to run properly since it’s more technical than manual methods.

When we look at costs in light of these trends, the steady budget increases make much more sense. More importantly, they become much more palatable, especially given the greater push for quality. But while we’re right to spend money on quality, it’s still a significant investment. Fortunately, there are several different methodologies and tools that can help you get the most bang for your buck.

How to Maximize QA Spend

So what’s a CIO to do? How can you maximize the 40%+ plus of your budget allocated to QA spend to ensure the quality you need without falling into a seemingly endless money drain? It all starts by understanding your priorities.

To achieve this goal, I recommend thinking about testing as a triangle, since we typically have three main priorities when it comes to QA and testing: Time to market, cost and quality. Without any budget, time or resource constraints, we’d be able to accomplish all three at once, but the world isn’t perfect and we are constrained by the realities of our business. At best, we can only satisfy two of these priorities at once. And once you identify your top two, properly maximizing QA spend where it counts becomes much easier.

Consider the following tips:

To achieve time savings…

• Automation: Automated testing tools can lead to significant time savings, however you do have to be careful when getting into automation. While automation is generally a time saver, it can sometimes fall on the other side of the fence and add both time and money to projects if you don’t approach it correctly. In order to realize the desired benefits of automation, you first need to understand and plan for the initial cost of building scripts and maintaining automated tools.
• Exploratory Testing: If you want to stick with manual methods, using exploratory testing can save significant time and provide the same level of coverage as test cases. Just look at the data: A 2007 controlled study found that testing with test cases can take almost seven times longer than testing with exploratory testing due to the amount of time needed to write the test and report on the results. It also found that test cases lead to more false defect reports, which can add unnecessary time to the testing process and monopolize developers’ time on unnecessary activities.

To achieve cost savings…

• Open Source: If you’re automating testing, replacing expensive automation tool licenses with open source tools that provide the same benefits can lead to big savings. Once again though, you need to be careful with how you approach open source. You need to understand the true cost and value of specific open source tools and use them selectively, keeping in mind that these tools don’t come with the same level of polish as licensed tools and often times require high levels of customization and maintenance.
• Freelancers: Intelligently using freelancers can save you a lot on people costs. Multitudes of freelance networks where you can go to hire specialized, technical resources on a short term basis now exist. Using freelancers allows you to bring on someone to solve problems as needed rather than staffing year round at a fixed cost with expensive resources who typically only have a backlog for one or two months. This thinking holds especially true when it comes to needs for highly specialized skills.

To ensure quality…

• User Acceptance Testing (UAT) & Beta Testing: Using non-dedicated testing resources through UAT or beta testing is a great way to get more opinions on a product before release without needing to increase the scope of the dedicated testing team. Additionally, end users (beta testing) and business users (UAT) can provide unique and valuable insights about how the software is actually used. These insights are often missing from teesters and developers who have been biased due to their involvement in the software design and development process.
• Test-driven Development: A perfect pairing for DevOps, test-driven development tightens collaboration between testers and developers/business analysts by requiring testing to be defined prior to writing any code. Without this collaboration, it’s easy for developers to interpret requirements one way and testers to interpret them another way, which can lead to testers missing defects or waiting until it’s too late to raise a defect because they don’t understand the functionality properly. As a result, tying testing and development together can improve quality and overall outcomes.

What’s the Right Balance?

There’s no one-size-fits-all for testing budgets, but the model is absolutely changing, and we need to adapt our thinking accordingly.

Furthermore, as QA becomes more and more integrated into overall development, perhaps it’s time to bake QA into the development budget rather than having two separate budgets.

However you slice it, we shouldn’t fear increases in QA costs. Rather, we need to look at it as an opportunity to deliver a better customer experience and remember the return that comes from making such an investment.

What’s your software testing ROI? Check out our free ROI calculator to find out and get tips for how to improve.