5 Product Discovery Pitfalls Leading to Scrum Failures
Products succeed or fail when they get two things right — “what to build” and “how to build”.
Many falsely believe that Scrum is a delivery framework which helps in the “how to build” domain, but doesn’t talk enough about “what to build”. This is because Scrum is widely misunderstood.
I once worked in a large organization where discovery was exclusively separated from the delivery process. Discovery was mostly conducted by individuals outside the Scrum team, and the Scrum team was solely responsible for delivery. I couldn’t last long there as I failed to break the silos.
Product discovery, the science of “what to build” is so much more than the process of design, backlog refinement and sizing.
Let’s explore what product discovery entails, where we commonly err in the process, and the mindset for effective product discovery. Additionally, let’s examine how Scrum approaches discovery through the lens of empiricism.
Understanding Product Discovery
Product discovery determines ‘what to build.’
Our product backlog floods easily with:
- CEO wants a feature
- VP of Sales promises a customer a feature
- Chief Security Officer wants security compliance
- A “whale” customer demands a feature
- External or environmental pressure to integrate fancy new technology like Blockchain or AI
The team faces a big challenge in deciding what to build as they grapple with the high stakes of selecting the right ones.
To determine what to build, we should work on two areas:
- Problem space: Here, we identify the right problem by pinpointing customer pain points and needs.
- Solution space: Once we understand what customers want, we identify the best solution to address their needs.
Where Do We Go Wrong in Product Discovery?
1. Inadequate Time for the Problem Space
We habitually rush to find solutions without deeply understanding the underlying problems. For example, when customers find it hard to search for items on our platform, our immediate reaction is to brainstorm potential fixes. We rarely ask questions like:
- Why are customers experiencing search difficulties?
- How long have these issues persisted?
- How many customers are affected?
Product Discovery is not planning. It is exploring and understanding. A plan, followed by exploration, holds meaning; otherwise, it is superficial. Also, It’s not always the case that we explore first and then plan. Plans can also emerge through exploration.
2. Discovery Solely for Estimation
For many of us, product discovery revolves around breaking down epics into user stories and estimating them to provide timelines to stakeholders. Our focus tends to be more on the solution and prediction rather than the details of the problem. Often, the requirements already include high-fidelity design files, and coders are asked to forecast how long it will take to build them.
If we need to estimate to assess feasibility, it can be treated as a part of discovery. But it should only be a trivial aspect of discovery. What matters most is whether we understand the problem and if the solution we are planning contributes to the desired outcome.
“Scrum knows only one condition for estimation: do we believe it can be ‘done’ in the Sprint or not?”
— Sjoerd Nijland
3. Discovery Without Past Evidence
Measuring the impact of our last release is far more challenging than measuring output focused metrics like velocity or cycle time. Important things are always difficult to measure. In contrast, our time is often consumed by internal meetings and trivial discussions, leaving little or no time for those crucial assessments.
In this complex world, how can we afford to ignore the lessons from our past endeavors? They offer great opportunities to identify the real pain points and needs of our customers.
Personally, I doubt ideas validated by UX prototypes shown to a few customer representatives.
In my past life as a developer, I worked on a job portal for a startup in Nepal. During a UX walkthrough call with early users, they provided extensive feedback comparing to the most famous job portal in Nepal at that time. I was frustrated then. Today, I realize it’s our fault for seeking opinions that are often influenced by various cognitive biases.
Asking customers if ‘it will solve’ your problem doesn’t validate our ideas. Instead, let customers use the product and inquire if ‘it solved’ their problems. This approach yields more authentic evidence.
4. Discovery for a Quarter or More
Because we have limited ability to measure the impact of our deliverables, we often resort to our strength: Planning. Otherwise, we won’t be able to discover for many months ahead. For issues like missing roadmap timelines, customer churn, revenue decreases, or declines in NPS scores, ‘planning’ often takes the blame. Ironically, the response is more planning, simply because it is what we believe we do best.
Assumption-based roadmapping is not discovery. The more meticulously we plan the solution, the more resistant we become to changing the plan. In today’s dynamic landscape, following a rigid plan won’t lead us to our destination. We must be willing to adapt and respond to changes.
The most serious impediments to using Scrum are habits of waterfall, predictive thinking over the last twenty to thirty years; these have spawned command and control management, belief that demanding something will make it happen, and the willingness of development to cut quality to meet dates. These are inbred habits that we aren’t even aware of anymore
— Ken Schwaber
5. Separation of Responsibilities in Discovery and Delivery
I once witnessed a team where the product manager involved developers in discovery conversations. Despite this, during one of the discovery calls, a senior product manager suddenly entered and abruptly questioned the presence of developers.
This scenario is common in velocity-driven organizations. Transitioning from a feature-factory mindset to a value-driven mindset across the entire organizational structure is the greatest challenge in today’s product world.
The concept of a Product Trio has gained popularity, but organizations often misapply it. Typically, a trio consists of the product manager, the most senior designer, and the most senior software engineer. I have seen situations where the trio is responsible for discovery. What makes it worse is that the trio is separated from the core delivery team so they can handle discovery for multiple delivery teams.
What I Have Seen That Can Help?
Recommendation 1: Define Product Vision
A solid Product Vision establishes a clear destination for your product’s future and why it matters. It gives you a benchmark against which you can measure your endeavors and determine whether you are moving in the right direction.
Let me give you an example of a Product Vision: “Build a digital platform to organize at least 1 million hours of premium pair programming activities among software engineers and industry experts.” It provides a clear destination, customer value, and business value.
We use Product Goal in Scrum, and it isn’t different. Some teams use the Product Goal synonymously with the Product Vision. For some, the Product Vision represents a long-term goal, with the Product Goal acting as a stepping stone towards that ultimate destination.
Recommendation 2: Spend Ample Time on the Problem Space
Do not rush into solutions, invest time in understanding how your customers interact with the product, where they encounter challenges, and what aspects provide value or lack thereof. Direct interview questions usually do not generate useful insights.
For instance, if asked what I want more on LinkedIn, I honestly don’t know a precise answer. When you analyze my user journey on LinkedIn with both quantitative and qualitative data, you will uncover features that I not only love but am also willing to pay for.
Moreover, develop the skills to recognize core needs from surface desires.
My 5-year-old son often insists on making fruit ice cream, yet shows little enthusiasm for eating it. It took time for me to realize that his true need lies in the shared experience of preparing the ingredients, rather than the desire for the ice cream itself.
Recommendation 3: Foster an Autonomous Team
If you want your team to truly own creating value for customers, there’s no substitute for a fully cross-functional product team. They do all the work to discover what to build, actually build, and deliver. They jointly make all the decisions on product discovery and delivery.
While they must collaborate with other departments like sales and marketing, the team should be empowered to make decisions. If someone else makes a decision for them, they cannot learn and improve. It hinders the product’s path to success.
Recommendation 4: Discover Continuously
Customer desires and needs are ever-evolving, with intense market competition. Regularly engage with customers to understand their evolving needs, frustrations, expectations, and aspirations for the product. Every interaction will give you valuable insights.
Before you decide what to build next, assess the impact of past decisions and leverage those learnings. This reflection is an indispensable aspect of product discovery.
Scrum and Product Discovery
Product discovery is a popular term in the Product Management world. Although you won’t find the word ‘discovery’ in the Scrum guide, ‘inspection’ is mentioned many times as a closely related term. Regardless of terminology, Scrum talks a lot about deciding what to build. Let’s explore further:
“Scrum is founded on empiricism and lean thinking. Empiricism asserts that knowledge comes from experience and making decisions based on what is observed.”
— Scrum Guide 2020
Scrum provides a closed loop where we ‘continuously’ decide what to build next based on our past observations. There is a reason why Scrum insists that the Sprint length should be one month or less. Scrum asserts that if you can reliably predict users’ needs beyond a month, the domain is likely simple, making Scrum obsolete.
Sprint Increments make our efforts transparent. Once released, these increments enable us to inspect user journeys, identify their needs and pain points. This observation helps us determine the right thing to tackle next.
During the Sprint Review, we inspect whether the value of the Sprint’s Increment resonates with key stakeholders. Using their feedback, we explore avenues for generating both business and customer value.
Frameworks like Evidence Based Management (EBM) and Objectives and Key Results (OKR) help quantify our observations. It strengthens our decisions.
After exploring the problem space, Sprint Planning becomes the event where we transition to the solution space. Here, we identify the best solutions to address in the sprint.
In a nutshell, I couldn’t agree more with what Willem-Jan Ageling said in one of his articles:
“Scrum is about discovery through delivery.”
Merely assuming that discovery will lead to future value is shortsighted. Similarly, delivering without inspecting past efforts is negligence. Discovery and delivery are intertwined.
What are your stories of Product Discovery? I am eager to listen and learn from your experiences.
