Which Test Metrics Agile Teams Should Use to Drive Success
Many modern software development teams follow the principles outlined in the Agile Manifesto. Agile development teams are cross-functional, with an emphasis on delivering working software faster and more often.
Using Agile principles, there is still a need to measure and improve upon testing efforts. However, archaic test metrics that focus on individuals(such as ‘story points per developer per time interval’) won’t cut it — Agile testing metrics arise which can help evaluate and improve how well you test your software.
Using Agile principles, there is still a need to measure and improve upon testing efforts. However, old test metrics that focus on individuals won’t cut it — Agile testing metrics arise which can help evaluate and improve how well you test your software.
This post highlights some of the main Agile testing metrics your teams should use to drive success and release better quality software quicker.
5 Agile Test Metrics
Any good Agile test metric must provide actionable insight that can help improve software testing within Agile teams, meaning the metrics should in some way give feedback on the main objectives of becoming Agile.
Cumulative Flow Diagram
A cumulative flow diagram depicts summary project information on an area graph. This type of chart lets you visualize whether testing is a bottleneck in your projects, and you can easily make a CFD in MS Excel.
Since testing must keep up with development for teams to become Agile, this metric can provide useful insight into what effect testing has on overall development time. The below diagram shows a project in which testing is a bottleneck process, represented by the green vertical area widening over time.
Cyclomatic Complexity
Cyclomatic complexity counts the number of linearly independent paths through a program’s source code, and several testing tools can measure this metric for you. The usefulness of cyclomatic complexity is that it determines the risk of unstable and untestable code which is likely to contain errors. Generally, a cyclomatic complexity value lower than 15 is recommended for testable code.
Escaped Defects
Software development teams must strive to find defects as early as possible in the development cycle when they are far cheaper to fix. Measuring the defects which are found in a production environment after your team releases software is a good bottom line measure of test effectiveness.
Higher than anticipated escaped defects, especially serious ones, warrant an investigation into the test suites used by your Agile teams so you can improve upon the testing efforts. Similarly, escaped defects can serve as a warning signal that the software under development is too complex, making it difficult to test adequately.
Defect Spillover
Agile development is iterative, meaning that a large development project gets broken down into small iterations of fixed lengths of time, often referred to as sprints. Defect spillover measures the defects that don’t get resolved in each iteration, meaning they carry over to future iterations.
The usefulness of this metric is that it measures whether the main goal of your Agile approach is paying off — are you releasing working software with each iteration? Or are teams getting stuck due to defects building up?
Defect Density
Defect density provides a simple measure of quality in program modules. This metric counts the number of defects found per thousand lines of code. By viewing the defect density on software modules, you get insight into the weakest areas of the applications you build. You can then decide whether those areas with high defect densities should be released at all, or made simpler so they have fewer defects. The formula by which you can measure defect density is: Defect count/ size of the release, where size of release can be measured in terms of line of code.
A Note on Continuous Delivery
Continuous delivery (CD) aims to get working software to users more frequently by ensuring all changes made to an application’s code are releasable. With CD, updates are smaller and software is delivered more frequently, helping to achieve one of the main Agile aims.
Adopting CD lets teams adapt their software on-demand to meet user feedback or market demands. Continuous delivery reduces the strain that Agile puts on older manual testing methods which cannot keep pace with modern development.
CD means a huge shift away from manual testing and towards more automation — when you need frequent and faster releases, you need feedback within minutes through automated code checks and tests that run continuously. Manual testing simply takes too much time in the current landscape, and Agile team members need to prepare for automating everything and creating a veritable conveyor belt for continuous releases.
Closing Thoughts
Agile development practices have changed not only the approach to how companies develop software, but how they test that software too.
Underpinning the collaborative approach to software testing in Agile are Agile test metrics, which are only useful if they provide feedback on team progress in achieving the main Agile Manifesto aims.
Even though test metrics are still helpful in Agile, there is a need to further speed up testing so that software releases make it to customers quicker and more often. Continuous delivery further changes the testing landscape by moving away from manual testing and attempting to automate as much as possible.
