Why Your Engineering Productivity Metrics Are Failing: Insights from Google’s GSM Framework
Currently, I’m preparing for my first day as an Engineering Lead at my new company. To get ready, I’ve been reading extensively — articles, books, and listening to podcasts about engineering leadership. I’ve also been speaking with friends who have held engineering lead or manager roles for several years. In this blog post, I will share a valuable framework I found in the book “Software Engineering at Google” called GSM (Goals/Signals/Metrics). This framework is instrumental in selecting meaningful metrics with clear goals.
Engineering Productivity Metrics
In my experience as a Software Engineer, I’ve encountered numerous approaches to measuring engineering productivity. Different metrics serve various purposes and are measured in diverse ways. The most common metrics I’ve used include:
- Number of Merged PRs: This metric reflects engineering productivity on a micro-level.
- Number of Deployments/Releases: This slightly higher-level metric represents engineering productivity. Deployments can include new features, bug fixes, code refactoring, and other non-feature-related updates.
- Test Coverage (Backend): In my experience this metric is typically measured on the backend, while the frontend focuses on integration testing (e2e).
- Number of Bugs and Resolved Bugs: This metric ensures code quality and tracks how quickly engineers resolve issues.
These metrics are useful but should not be the sole approach to measuring productivity. I’ve worked under a leader who blindly implemented productivity measurements without considering other factors, resulting in poor decisions that hurt team trust, motivation, and overall product quality.
GSM Framework
Instead of relying solely on traditional metrics, we can use Google’s GSM Framework to guide metric creation. The framework involves defining:
- Goal: A high-level objective without specific implementation details.
- Signal: Information indicating whether the goal is achieved, which usually cannot be measured directly.
- Metric: A measurable proxy for signals.
Following this framework from Goals to Signals to Metrics can prevent many issues. In my past experiences, hastily creating metrics without defining clear goals and signals led to miscommunications, wasted time, and poor decisions. For instance, upper management reviewing reports without context might misinterpret the data, leading to misguided actions.
Streetlight Effect
A funny anecdote from my hometown illustrates this concept:
A man was looking for his lost wallet under a streetlight at night. Another man asked him where he dropped it, and the first man replied, “In the bushes at the edge of the forest.” Confused, the second man asked, “Then why are you looking here?” The first man innocently replied, “Because it’s dark in the bushes, and there’s no light there.”
This situation, known as the “streetlight effect,” highlights the danger of looking only where it’s easy rather than where the answer lies. The GSM framework helps avoid this by ensuring we define goals and signals before metrics.
Metrics Creep and Metrics Bias
Ciera Jaspen, in “Software Engineering at Google,” warns against metrics creep and metrics bias. When metrics are chosen without a principled approach and fail to meet stakeholders’ expectations, stakeholders might suggest new metrics to achieve desired results. This can harm the engineering team over time. The GSM framework prevents these issues by establishing a clear, goal-oriented approach to metrics.
Conclusion
The GSM framework offers a structured approach to selecting meaningful metrics in engineering leadership. By defining goals and signals before metrics, we can avoid common pitfalls and ensure our measurements accurately reflect productivity and quality.
Thank you for reading! If you enjoyed this post, please give it a clap, drop a comment, share it with others, and follow my Medium blog. For more frequent and personal insights, follow my blog on Substack. Your support means a lot!
