“So Pick Me, Choose Me, Love Me” — A 4 Step Guide To Finding Your R&D Execution Metrics’ True North
Neta Ilany | R&D Lead Program Manager at Skai
What’s the best metric to measure R&D teams’ efficiency? I ❤️ this question! It opens the door to an important and mature discussion in the team. Here’s my POV on how to answer.
I’ll take the “classic” engineer approach here, and say — “it depends”. It depends on how “effective” translates into in your org. It depends on what is currently causing “noise” and disturbing the team. And it depends on what you’ll want to balance, so no aspect is compromised over the other.
WIIFM
Metrics are friends, not foes; they’re first and foremost for the team to consume, and only then for management to review. Part of taking yourself and your team to the next professional level, is establishing the ceremony of looking at these metrics and have them tell the story rather than you. Yes, as data driven as it gets.
Mindset in place, remember that this is a journey. It might be a bumpy ride, but the process is more valuable than reaching the destination. It raises important questions and discussions such as what’s important for the team, what the team feels is blocking them and what they want to improve. So take the time to get to the roots of it.
Finding True North Metrics
Step1: You Need To Crystallize What “Being Effective” Means in Your Org
Efficiency requires focus, and what to focus on should be first defined by the R&D organization at a much higher level.
Examples of some focus points and their reflecting metric:
- How fast do we deliver? → Feature cycle time to production
- How many features do we deliver? → Feature development vs production support
- Quality of deliverables → Escaped Bugs, Time to Resolve (TTR)
- Scope maturity → Unplanned tasks ratio in sprint
- Blocker reduction → measure specific pain points such as test environment creation or support tickets per component
Step 2: Identify Your Pain Points
The art of selecting the right metrics relies on 2 pillars: identifying the pain and being able to measure it with reliable data.
Each R&D org is unique in its culture, but at the end of the day we face common problems, and can use common solutions. Here is our pain matrix. Some pains have more than one root cause, and more than one metric that can point out the trend.
Once you have your pain matrix laid out, it’s easier for the team to start the discussion around which pain they experience, what hurts the most, and what can move the needle in terms of team’s execution.
Each team will probably want to choose a different set of pains. It’s important to use more than one metric to ensure balance, and avoid overcorrection.
At Skai we find it useful to balance the following 3 focus points, so neither one is compromised, and all teams are on top of their throughput.
- Feature ratio vs production support/tech investment
- Feature cycle time
- QA metrics
The impact: our development teams decrease feature cycle time in an AVG of 30% YoY, get more control over tech debt investment and gain better visibility of our R&D execution.
Step 3: Make Sure You Use Data That You Can Trust
Use out of the box/customized reports that are based on strong R&D team habits, so data reflects reality. E.g, if the team updates the task status on a daily basis, but does not report their actual work on tasks — measure cycle time rather than logged time reported.
When people see the value they’re getting by learning from these metrics, they’ll be open to implement new habits that will eventually provide new data.
At Skai, we took all data from our task management system, and built our own BI views on top of it. This enabled us to apply our own logic. Consuming data is easy, and requests for new metrics/knobs are managed by the program team.
Be minded to user experience — a dashboard that is also a tool to investigate the above “pains” can be your way to capture your users’ attention.
Step 4: Adoption & Stickiness
You need to work your way in a Pincer Movement here — both bottom up and top down;
Include the metrics review routine in your team retrospective. This is where team members ventilate how they feel and validate it with the metrics.
From the other end, managers need to show interest and provide professional feedback based on the execution trend.
At Skai, we initiated a monthly group retrospective for team leads with their manager, and a quarterly with the directors. These meetings focus on reviewing the trend and the underlying issues, and discussing what we need to do to improve.
By implementing this framework, the trend is reviewed from the team up.
3 Tips Before You Hit the Road
- Dance like nobody’s watching — think of the right metric for you that reflects your pain, without overthinking what if other people see it.
- Compare You to you — compare yourself to who you were yesterday, not to other teams
- Amplify every small success with the team. We rely on feelings and tangible feedback, like “this feature is amazing”. But feedback backed up by data — that’s golden.
