What makes your software metrics to fail?
Quantification is the essence of software engineering.
Software engineering is good at quantifying typical aspects of systems and programs. But most likely, engineers forget about some weak points of measuring. We can take as an example quantification of underestimated aspects of systems like usability and security and if we dive even deeper, we can find more sub-disciplines of software and data. However, we need to use the quantification tool in all critical aspects of our systems, not just in traditional sectors. In this article inspired by Tom Gilb’s “Result Planning Limited”, INCOSE, 2008, we will explore the weakest areas of systems engineering measures and answer the question — what makes your software metrics to fail.
Measuring what is easy to measure.
This statement doesn’t need too much explanation. There are always so obvious parameters that are very easy to quantify, that engineers cannot refuse to measure. And it takes their attention from a bigger picture of an issue they supposed to look at. We always should determine what is the most critical to control first, and then find a way to quantify it. The whole world can be written in numbers, and surely there is a way to measure that quantity.
Too late measuring.
Measuring at the end of the project is just too late to learn in time. It’s also very hard to convince people that they have a solvable problem that we discovered through our metrics analysis. And even if we make them believe in it, they will find more and more reasons why “not to touch it”. If we start tracking metrics early, we will have a chance to understand what to measure and what requirements are.
If we apply concrete metrics in the early stage of the project and look for them during the development, we will see which metrics are very useful and brings the most value to our business and which are just supporting metrics and what is worth measuring. As we all know, management looks on software engineers through numbers so early measuring gives us the possibility to give them actual numeric levels of requirements.
Measuring too few or too many parameters.
We should limit our minds to think about the top ten most critical requirements. If we master them, probably we will be ready and have enough resources to start thinking about the next priority measures. Mastering ten critical variables at a demanding level is a magnificent technical management deed. If we fail on 11th it will be forgiven, at this time it’s next on our hit list anyway.
Too low metric level.
The question is — what is ‘too low’ a requirement level? We should consider some variations. We can think about too low metrics level in terms of a particular area and time, specific conditions, and events. Metrics can be too low concerning a future competitor level (it makes system uncompetitive) or too low with our current system level — what makes us competitors. We should always think about the relations of our metrics and benchmark relationships that let requirements numeric values be compared with these relationships.
Main and supporting metrics observation.
As mentioned in previous paragraphs, it’s critical to know the role of metric so it can roll over on a project. As we know all metrics treat systems environments — deadlines, intervals, market segments, task types, and so on. We need to define that environment first and give the metric role to play. One simple example of main and supporting metrics can be a requirement which is the list of conditions that must be true to make the requirement effective true or not. Each item on the list will be a supporting metric and the list itself will be treated as main.
The set of principles described above is considered poor practice in software engineering quantification. We can improve our systems by making use of realistic numbers instead of subjective opinion numbers with the help of metrics. We should use more numbers in describing critical requirements, especially soft and quality requirements. However, we should think about it not only in terms of using numbers but using them more effectively. That means making distinctions among different numeric levels of requirements and specifying the relationships of both requirements and designs to other elements of planning and their environment.
