IT Performance Metrics : The Mean Or the Median
“What gets measured gets improved”
— Peter Drucker in his book The Effective Executive.
Within the IT industry, there’s an ongoing pursuit of enhancing efficiency and discovering avenues for improvement. Key Performance Indicator (KPI) Metrics serve as pivotal tools in crafting strategies toward this end, underscoring the importance of selecting the appropriate metrics.
A prevalent method for gauging performance involves utilizing the ‘Averages’ or the Mean : Mean response time, Mean execution time, Mean time to repair, Mean time to resolve to name a few.
‘Mean’ as we all know it from primary school days is the “sum of all values in the dataset divided by the number of values” .
But is mean the universally suitable metric for all KPIs ? Why not other measures like Median (Highest occurring value when sorted in scending or descending order) ? Most obvious answer would be because of the way we have got used to it and most of the tools capture averages.
Further in this post, let’s compare the Mean and the Median based approach with few examples relevant to an IT service team:
1. Measuring Reaction time : Time for the request to be acknowledged.
Mean clearly seems to be the right choice. It reflects reaction time to various requests.
Now let’s take a look at another metric — the time to complete these requests(Execution Time)
2. Execution Time
Here we see a clear divergence between the Mean and the Median. Median here quite accurately represents the time spent on Execution on most of the requests and that is the number that we are interested in. Mean, on the other hand is impacted by one outlier. There may be several reasons for the outlier, but that’s a separate metric to track and investigate.
So, which metric is the most appropriate choice? There’s no one-size-fits-all solution. However, certain guidelines listed below can aid in determining the relevant metrics to employ:
- Opt for the Mean when dealing with data exhibiting a consistent, predictable range without significant fluctuations.
- Embrace a Median-based approach when the data displays outliers with considerable variance. However, ensure to separately examine these outliers to identify opportunities for improvement.
Feel free to share any alternative performance measurement approaches that have proven effective in your experience.
