You cannot fix what you do not measure

You need data. An organization flies blind without data and regular inspections of data.

When you set a goal — your target state, for example — how do you know you have accomplished it? By writing clear metrics that establish success and failure. Similarly, if you do quarterly planning, you need to ensure that you have written down what your success metrics are. Without writing this down, and having regular inspections of these metrics, you do not know if you are succeeding or have succeeded.

I like to have a single metrics review every other week where all data is reviewed and examined. This is an important forcing function that ensures that all leaders are able to talk to their metrics, and have taken the time to understand it.

So what are the key metrics to be tracking? A lot depends on what is most important to you. Here are the ones that I track

Coding Days

I measure productivity primarily using coding days as the metric. I expect coding days to be 2 or better for most scrum teams. if it is fewer than that, there are possibly problems (too many meetings?) to be resolved. In addition, I find it useful to look at PR requests and PR review comments.

Measuring coding days and related metrics are never part of performance discussions. It is to diagnose problems in the system that prevent engineers from doing the best they could.

The key non-functional attributes

Availability (99.95 is standard for SaaS)

Performance (read my detailed explanation at this blog post) — TP50–1 sec, TP90 -2 sec , TP99–10 s

Scalability shmalability. Serverless means not worrying about this. more on this later

Defects out of SLA: Metrics for defects are: P1 = all hands on deck. Max 24 hours turnaround. P2 = 72 business hours, P3 = 1 sprint , p4 = same quarter. I track how many defects did not meet SLA’s with a focus on P1 and P2 defects

Releases per week — Code in microservices (lambdas) get deployed continuously. I generally expect each team to be release atleast 2–3 times a week