Velocity vs Quality? It’s all about the learning
I recently read an article by Julie Zhou on How to Make Things High-Quality in which she describes a common debate amongst almost every product development team:
Julie’s premise is that this is a false choice because we should be shipping high quality experiences every time, and that the real problem is we did not take into account the time required to get to high quality:
“If we knew X would take this long to get to high-quality, would we have opted to do it in the first place?”
I love questions like these because they force teams to question previous assumptions and that typically drives changes in future behavior. But I also think this is the wrong question to ask to solve the problem.
And the reason is because high quality is an output. No different than high velocity. And that’s not what we’re trying to optimize for. We’re trying to optimize for outcomes. Let me explain with an example:
One of the most impactful features we released at Amplitude in 2018 was a feature called Team Spaces. It’s driven an increase in collaboration in our product of over 50% and that has made a significant impact on the growth rate of our North Star metric.
But if you asked me when we broke ground if we should build this feature knowing it will take over 12 weeks to get to high quality, I would have said I’m not sure. Because we hadn’t tested any of our risky assumptions yet.
Instead, the right question to ask was: How long will it take us to get to learning? If you told me it would take over 12 weeks to get to learning, I would have said no way. We estimated it would take three weeks. It ended up taking closer to four. And given our understanding of the problem, that felt like the right level of investment to test our assumptions.
Instead, the right question to ask was: How long will it take us to get to learning?
Why is learning the right question? Because learning implies a level of quality. Which is very different than high quality. Its quality with context. For the first version of Team Spaces, the quality was not great. Everyone would acknowledge that. But it was good enough to get a handful of customers to start using it. And from that we learned a lot. We learned that some of our initial assumptions about the problem were actually wrong, which helped give us conviction on the areas to double down on. Which led to a much higher quality iteration centered on the right use cases. And eventually to a product that we all believe is high quality, but more importantly solves real customer pain.
Shipping a high quality experience is an output. The hard part is shipping the right high quality experience to drive your desired outcome.
