Discovery Sprint

Part 1: The Old Approach

I started working with product teams when I was eighteen. I always state that one of my biggest advantages as a product manager is that I have had the privilege to collaborate and interact with many teams across the globe. This allowed me to gather various insights about their culture and discipline.

The most interesting finding that caught my attention was the fact that the best tech companies had very efficient work patterns when discovering and creating products for their customers. Many of these patterns, in fact, were similar in nature. Meanwhile, businesses that had difficulties growing and staying competitive in the market, were developing their solutions using old-fashioned and cost-ineffective approaches.

In 2016 I moved back to Armenia from Silicon Valley and became the vice president of engineering in one of the largest tech companies in the local market. Previously, I never had an opportunity to collaborate with Armenian businesses, so this was a pivotal event that allowed me to explore the unique technology ecosystem in the country. My entire journey inside the company was a life-changing experience and also one of the reasons why I decided to write this story.

Many things in this enterprise worked differently from what I have seen and experienced in businesses with great product cultures. However, the most demotivating thing was seeing how much organizational resources (talent, time, capital) we were wasting on product initiatives in every quarter that turned out to be complete failures. Eventually, some natural questions that came to my mind were who decided what solutions the company should build, how did that decision-making process happen, and how did they measure success?

The Approach

The company I was talking about, I would call a request and executive-centric business. What does this even mean? Imagine that nearly 90% of all solution ideas and product changes come directly from the customers or top management.

You may ask, “What is wrong with that?” The answer is simple. Although the company had a role responsible for the success of its products, none of the employees who were holding that role were able to control the success of their efforts. Let’s look at the whole process in greater detail.

Ideation
Everything started from the Ideation step. Note that, those terms were not specifically used in that enterprise; I have introduced them to describe the cascading process of organizational work for defining, designing, and delivering solutions.

As I have already mentioned, most of the ideas were either from the executives like the CEO of the business or from the B2B customers, who were aggressively asking for new products and features day after day. And the company’s goal was to implement all those ideas, no matter how promising they were. The end result of this stage was to gather the requests in one place.

Business Analysis
The next step which I like to call Business Analysis involved the CEO and executives, who would gather a meetup of project managers and discuss with them what it takes to support those features or build a product that customers or the management had asked for.

Meetings happened on a weekly basis and would last for a couple of hours. They would usually involve conversations about prioritization and the resources that should be allocated for the execution of the requested solutions. The aim of these gatherings was to create an output-based roadmap, which later on would be distributed into Jira.

The concept behind output-based roadmaps was this — there is a feature, and this feature should be implemented. The success of the team is the release of that feature. Nothing more than that.

For example, if the customer requested integration of a specific game, that project would be put into a roadmap. Then a deadline would be set, and the definition of success would be actually delivering the solution by the agreed date. That’s it.

Documentation
Once everything was set up and decided, the work would be passed on to the product owner. Note that, if the company is using the term product owner to describe the role of its product managers, then there is already something alerting there. Due to the common misunderstanding about agile and its corresponding frameworks, many people conceptualize the role of a product manager as someone who leads the delivery of the product.

Don’t get me wrong. Product delivery is one of the responsibilities of the product manager, but only one-third of their responsibilities. If the business needs product managers who are just there to lead the delivery process, then they are not really product managers, and the company doesn’t have a strong product culture. They are essentially project managers who organize the activities for bringing a solution to completion within the agreed requirements, time, and quality.

In the third step of the process, which I call Documentation, a product owner had to take the general roadmap, pick the ideas relevant to their team, and just document them. Hypothetically, if the integration of a third-party game was part of the roadmap, the product owner responsible for that initiative would write down a user story about the new feature and its acceptance criteria. The end goal of this phase was to have a somewhat defined list of requirements that engineers could follow during software development.

Design
After the documentation was prepared, the designer would go through the given descriptions of the solution and create its interaction and visual designs based on the requirements coming from “unknown sources” and for “unknown reasons.”

Many times, designers would sit in a totally different department, far away from the product owner and engineers and most of their communication was via Jira tickets and Slack. The end goal of this phase was to provide developers with the necessary design mockups so that they could build the user interface.

Development
As soon as the mockups were ready, engineers would start the Development step. They would take the designs and the documentation and begin implementing them according to some agile framework usually Scrum or Kanban or something in between called Scrumban.

Sometimes the delivery process could last several months — several months of consistent implementation without value, usability, or demand testing. And since oftentimes the execution deadlines were pretty strict, the team had to implement the product or feature just to meet those deadlines. The objective of this stage was to generate a shippable solution.

Testing
In the final step, the quality testing engineers would examine the created software. Since most of the time they were sitting far away from the product team, this would make the communication process chaotic.

The end goal of this phase was to guarantee that the developed backlog items meet the minimum quality standards. Once the team reached that state, the solution was delivered to the market.

As you have noticed, the mere implementation of a solution would thus define the team’s and product’s success. And the results were shocking. Over 80% of our product efforts eventually became complete failures. We were basically wasting most of our organizational budget on things that were not solving meaningful customer problems and helping the company to grow. This was a very expensive strategy to run a business.

Originally, I thought that this approach was peculiar to companies that were far from Silicon Valley — companies that haven’t had a chance to experience better product development tactics. However, after returning to the valley, I was even more shocked to see that it was something that a lot of businesses there do as well.

So, this was not regional or cultural. This was not specific to the type of industry or technology. This was just the consequence of the fact that many companies nowadays are not aware of more efficient ways of building products that customers need, value, and love.

What’s Wrong With This Approach?

There are many reasons why this approach is not effective. This is not how great teams work and create technology products. And since I went through the process in cascading order, I will keep the same format when presenting the problems of each step.

But before that, pay attention to the fact that the discussed organizational workflow significantly resembles the framework called Waterfall. Even though the company claimed they followed the principles of agile, the ideology of agile was only used in the Development phase — when engineers were writing the software.

Each step in the process was dependent on one another, and the product teams heavily relied on top management. It is correct to say that executives were deciding not only what to build, but also how to build it. The product teams were there just to serve the business.

Problems in Ideation
In the Ideation stage, when the customer makes feature requests, does it mean the product team should always follow them blindly? Imagine if Instagram implemented all the solutions and changes people are asking for.

A great product is focused on solving a set of meaningful customer problems that make sense to target together. But as we add more and more features, it loses its focus, becoming heavier and less usable. Hence, it is critical to consider why we are building or changing something. Implementing a solution should take time and research. It’s not only about gut feeling or satisfying every single request.

Moreover, it’s not something to be decided by the executives only. Our executives can have a pretty good understanding of the industry and business. Many of them can also be data-driven and continuously learn about their customers and market trends. That is fantastic. However, the experience shows that oftentimes the most successful ideas do not come from top management.

The role of an executive takes a full-time commitment. Understanding customers, their needs, and discovering winning products for them is also like a full-time job. Can we expect a C-level person with a super heavy schedule to have time to learn about all those details to make good decisions about products?

Unfortunately, no. Executives usually sense the big picture very well and have a clear understanding of the problems their business faces. They can communicate about the issues, but many of them don’t hold enough background information and insights to suggest the most optimal solutions. So maybe it’s better to leave product decisions to product teams?

We hire those intelligent and creative product managers, engineers, and designers to tackle the hardest product challenges and create solutions that bring tangible value to the customers and the business, so let’s allow them to do their job.

Problems in Business Analysis
I see problems in the Business Analysis phase too. Since there was no filtration of the requests coming from the customers and executives, the main thing left in this step was to prioritize the initiatives, determining which one to do after what. And the company evaluated each initiative based on its potential and the resources that were needed to implement it.

But can we confidently evaluate how promising an idea is at this stage? More importantly, how can we be even sure if an idea is going to work? What’s our evidence? Well, the unfortunate reality is that at this phase, we don’t have enough knowledge to assess an idea. A product initiative might sound promising to executives and one or two customers, but we can’t indeed conclude that it will be valuable for the market.

When prioritizing the initiatives, the management was also fixing the corresponding deadlines. In some cases, those deadlines were enforced because of the partnership requirements and the fierce competition in the market. However, the unpleasant reality was that they were being set without the estimates from the product teams.

Engineers were the ones responsible for delivering those solutions, but unfortunately, they were not present in these business meetings. They were just receiving the documented requirements with strict deadlines, which sounds unfair because their opinions had never been heard before making those decisions.

While the engineering executives such as the CTO and vice president of engineering were attending the meetings, their estimations were often inaccurate. These people are aware of the high-level technical infrastructure of the company, but in most cases, they are not up-to-date enough to make correct assessments about the development process for each team.

Last but not least, there are significant flaws with output-based roadmaps as well. The concept of output is the new state of the product after shipping a set of features and changes. So, those roadmaps basically describe what is the output of a product after each release. This creates the illusion that the management will be satisfied if the requirements get implemented and on time, connecting the concept of success to the proper delivery of projects before the deadline.

But what happens when solutions get launched but never really used by the customers, bringing no significant impact on the product and business? Do we consider this a win? An obvious answer is NO. This leads to a need for redefining what is product success.

The release is not a success yet. Solving important customer and business problems that impact the growth of the company is what we call success. Hence, instead of concentrating on outputs, the executives should empower each product team to think in terms of outcomes — short to medium-term objectives that a team needs to achieve with their product to help the business grow.

In this model, finishing the integration of a third-party game isn’t enough to say that the team has completed the initiative. They still need to understand whether that integration helped them to get to their desired outcomes or not. For example, if the team wanted to increase user engagement, did that new game allow them to achieve that?

The definition of success for a product team would now become the accomplishment of their desired outcomes. What’s important here is to keep the outcomes measurable enough — so the team can easily track if they have managed to reach them or got closer to them. This outcome-driven mindset is one of the key indicators of a great product culture.

Problems in Other Steps
One of the biggest “innovation killers” inside the company could be the executives who constantly tell product teams what to build and how to build things. The teams this way undergo the risk of losing their charm and value.

In this setting, the product manager just prioritizes the backlog and takes care of the documentation (essentially, backlog management). The product designer creates the designs based on the requirements they are provided with (with no proper communication or brainstorming). And engineers write lines of code and ship the product. Delivery. This is what teams do here.

However, in great companies, product teams are concerned not only with the delivery but most importantly with the discovery of valuable and usable solutions. Product managers do more than just documentation. They lead the team in understanding what they need to build, who they are making it for, and why they are building it.

Similarly, engineers do more than just coding. They help the team reveal the best ways to solve the given problems based on the advantages and limitations of the technology. And product designers do more than just creating visual and interaction designs. They identify the best holistic experience for the customers.

Having so knowledgeable and cross-functional teams, companies should at least listen to their opinions before putting a new initiative on their roadmap. At the end of the day, what will happen when the solution goes live and fails? Whose fault would that be?

It is unfair to expect the product manager and the entire product team to be accountable for the success of their product without empowering them to create that success.

In this old approach, product teams implement the ideas coming from the business and then get blamed just because the management has made wrong decisions.

Do you not face any of the problems mentioned above? You are lucky if you don’t. If I have presented things similar to your current company situation, then continue reading this article. In the second part you will learn about the right approach — a more productive framework and process for revealing great products called discovery sprint.

Part 2: The New Approach

The Two Types Of Product Failures

When building technology solutions, two of the core organizational processes managed by product teams are product discovery and product delivery. Product delivery combines all the activities the team performs to ship the solutions to the market as quickly as possible. It includes activities such as documenting the product requirements, planning the upcoming release, and developing, testing, and deploying software into production.

Product discovery, on the other hand, connects all the activities for understanding the customer and business problems and then revealing the best ways to solve them through experimentation. The ultimate goal is to identify the next value the team is going to create as early as possible.

The issue I have witnessed in many places is that companies tend to overemphasize product delivery and underemphasize product discovery. The unfortunate reality for those businesses was that their well-defined delivery processes were not enough to save them from major disappointments after the launch.

In essence, there are two types of product failures. When we fail in the discovery phase, we waste only a tiny portion of organizational resources on exploration and experimentation. This is when the product manager, designer, and one or a few engineers work together to test whether their solution hypotheses are something that will benefit the customers and the business. If not, the experiment will be considered unsuccessful.

However, when the team fails after the actual release, that is a real problem. It means that the team has worked on an idea, fully executed the plan, and shipped it to the market, eventually achieving no actionable results.

Sadly, this is not only about the efforts of the product team. Marketing professionals utilize their skills to promote the solution. Salespeople try to sell it. The customer support department starts learning the new features to better assist customers. The company utilizes a big chunk of its talent, time, and capital on a prospective waste.

Thus, the right mindset would be treating new ideas as just hypotheses that we still need to test through experiments before committing to any sort of major development work. And before deciding to build anything, teams should tackle the main product risks and gain reasonable evidence that their initiatives have enough potential.

Marty Cagan — one of the top influencers of contemporary product management, has talked about product risks in his book “Inspired: How to Create Tech Products Customers Love”. These risks include:

• Value risk: Does this solution bring actual value to the customers and the business?
• Usability risk: Is this solution usable enough so the customers would understand the value it gives?
• Technical feasibility risk: Can our engineers build this solution with the time, skills, and technology we have?
• Business risk: Is this solution right for our business?

In working with so many product teams, I have noticed that there is also a need to address another important risk during the discovery that helps us understand if there is a strong demand for the solutions we are thinking of building. A solution might sound valuable for a group of customers, but they could represent only a small portion of the current and prospective customer base. I have decided to name this the demand risk.

Problems In Modern-day Product Discovery

The great thing regarding the recent development in product management is that more teams started to talk about product discovery. But unfortunately, some obstacles still hinder them from organizing efficient discovery activities. Let’s discuss these difficulties.

Lack of focus and structure
Oftentimes, teams don’t know what objectives they should achieve and what directions they should follow to satisfy the business needs. They realize that the product must perform well but they don’t have a concrete plan for revealing winning solutions for the market. Even if the goals are provided, finding what needs to be built without a precise set of activities can seem really chaotic to them.

Lack of systematic, repeatable, and sustainable approach
Just a few customer interactions and promising launches do not guarantee long-term success. We live in a dynamic world where customer problems and their levels of severity can change very fast. Meanwhile, competitors can create new trends and increase customers’ expectations of existing products.

To stay up-to-date with all this knowledge and be competitive in the industry, companies must incorporate a systematic, repeatable, and sustainable model for discovering new value propositions and solutions and improving the current ones.

Lack of practical guidance
Even when the teams decide to go with some method, there are limited resources that will allow them to master discovery. There is, undoubtedly, a lot of content about talking to customers, revealing market opportunities, running solution ideation sessions, and creating and testing prototypes. However, most of them don’t connect the dots and miss hands-on tips and personal examples that help people who are new to discovery understand the specifics of it.

The new approach
In my ten years of experience building products, I have learned that product discovery is one of the toughest challenges for tech companies. Having no mentor who would teach me the essentials of continuous discovery, I really feel how hard it can be for people to get started with it.

Hence, to help businesses incorporate that systematic, repeatable, and sustainable approach to discovery which is focused, structured, and customer-centric, I have created a framework and process called discovery sprint. But before we dive deep into this model, let’s define some key terms together:

Product team: a self-organized and cross-functional unit that consists of a product manager, engineers, and a product designer

Customer: a collective term I use in this article to replace different persona type names such as buyers, end users, champions, etc

Desired Outcome: an objective that when achieved, indicates success for the team

Hypothesis: an evidence-based assumption about your customers, product, business, etc

Solution: refers to both a feature and an entire product, or an improvement of those two

Prototype: a solution archetype (design, technical, content, live data, hybrid) built for testing your hypotheses

Experiment: the whole operation and artifacts (prototypes) designed for testing the solution hypotheses (usability testing, concierge testing, wizard of oz, fake door testing, landing page smoke tests, explainer video tests, technical feasbility testing, etc)

From Delivery Sprints To Discovery Sprints

The last two decades were remarkable for optimizing delivery processes. The rise of agile was revolutionary for tech companies aiming to decrease their time-to-market lengths. It also allowed reducing the chances of potential failure. By shipping the solutions as fast as possible, product teams started to gather validated feedback in earlier stages of work.

This was the beginning of a great transformation. Many businesses that used to take years of consistent development without sufficient customer interaction and market validation, began to deliver solutions in quarters, months, then in weeks, and even several times a week. This shaped the genesis of modern product delivery.

Product professionals define the efficiency of their execution model by a term called a delivery cycle. Agile enthusiasts like to call it a sprint or iteration. The shorter your sprints are, the more effective your delivery will be.

Besides the opportunity to gather rapid customer responses, short delivery sprints enable optimal release planning. They also help teams control the risks in execution by dividing the big chunk of work into smaller components. This process of decomposition allows easier creation, testing, and deployment of software.

In addition, short delivery sprints are a perfect way to keep stakeholders happy. Stakeholders don’t like waiting for months to see the updates you promised to them. Although short delivery sprints produce smaller solution increments, they can quickly bring noticeable changes to your product. This can lead to more practical discussions about your strategy and execution. Management cares about progress, and short sprints demonstrate that you are making one.

The improvements in delivery cycle times show that people have been really smart about making their development processes efficient. Seeing all this advancement, a logical question would be how can we incorporate the same iterative, structured, time-boxed, and focused mindset into the discovery to remove the chaos from it? This is when the discovery sprint comes into the picture.

The discovery sprint is an outcome-driven framework and process that allows product teams to effectively reveal the customer and business problems, and formulate, prototype, and test the solution hypotheses that are designed for solving those problems.

It consists of six steps:

1. Define your desired outcomes
2. Discover the relevant problems
3. Run discovery ideation
4. Determine the scope of your experiment
5. Create the minimum viable solution prototype
6. Test your minimum viable solution prototype

We often have hypotheses about our customer needs, opportunities and anomalies inside the product and our business. These views are formed and influenced by qualitative and quantitative sources, our intuition, and domain experience. However, putting the team’s efforts and the company’s resources into the implementation of those ideas without enough validation and evidence is full of risks we can’t afford.

Thus, discovery sprints bring the desired balance between our thoughts and real customer and business problems. Those six steps act as checkpoints to robust our critical thinking and help us to ensure that we are moving towards the right path and not just blindly following the idea that we think should work.

In the last two steps of the sprint, I use the term minimum on purpose since the goal of the discovery is to test your product hypotheses with minimum effort. During the sprint, you don’t need to build a fully viable version of the solution. The prototype of that solution should be sufficient to test your assumptions and gain the necessary knowledge.

The Ten Properties Of Discovery Sprint

It goes great with our mental model
Discovery sprint is not designed to bring yet another layer of sophistication into your product thinking. It just provides basic checklists to ensure you have thought about the key hypotheses to be made when creating a solution.

With its universal format, it allows quick adoption into your organizational workflows. And if you want an adjustment, you can easily make one. In fact, I strongly recommend not being religious about frameworks, processes, and tools. Learn them, use them, and if it’s necessary, adapt them to your situation.

It brings focus
It aligns the team’s discovery work with the goals they need to achieve. If the desired outcome is increasing the percentage of signed-up users who reach the Aha moment during the onboarding flow, then all the effort should be dedicated to achieving that result. This requires ruthless prioritization as you have to eliminate the ideas that don’t contribute to getting to that desired outcome.

It yields validated learning
At the end of the sprint, you can tell if your assumptions were true or not. This happens once you analyze the results of the experiments you defined, built, and tested during the process. When doing discovery, you will notice that many of your experiments fail. But that’s fine since it’s the nature of experiments. Otherwise, they won’t be named that way.

Failing an experiment doesn’t mean you failed the sprint. Your sprint will be considered failed only when you don’t find out why the experiment didn’t work. Whether the experiment was successful or not, understanding the reasons behind its outcome is called validated learning. After every sprint, this learning gets added to your knowledge bank, making you a stronger domain expert and helping you run more efficient discoveries in the future.

It is time-boxed
Each sprint has a fixed start and end. The length of the sprint depends on the stage of the product and the scope of the experiment. However, the golden rule is that the shorter you keep it, the sooner you will gather validated learning about your solution, customers, and their needs. It will also help you get several iterations ahead of the delivery sprints, giving you enough room for experimentation to understand the potential of your idea before assigning it to your engineers.

In the beginning, when teams are new to the framework, some sprints might take longer than others. But over time, people begin to stick to fixed-length sprints. The first ones can go a few weeks but once the group gains experience, it tends to become a week-long activity.

It is iterative
Your work in discovery doesn’t stop with just one sprint. During the follow-up sprints, you might need to go back to make adjustments to your desired outcomes, improve your knowledge, and revise your hypotheses.

If your sprint was successful, you might want to double down in the next one, aiming to accomplish higher objectives. And in case there is a change in your product direction, you start new sprints entirely from scratch.

Some steps can be skipped
Not all six steps are always required in every sprint. If the desired outcomes remain the same but you want to understand the customer needs more, you start the next sprint by exploring the relevant problem space. Or, when you feel confident about the problems you have identified but need to think of better solutions, you can directly jump to the discovery ideation in the following sprint.

It applies to all experiment scopes
Discovery sprints apply to solutions of all sizes and levels of complexity, from product experiments (for example, improving the onboarding flow or building a small or medium-sized feature) to product initiatives (for example, creating a new product or large feature). Thus, I use the term solution inside the concept of a minimum viable solution prototype to keep the discussion generic for all cases.

It applies to all business stages
Discovery sprints are the product team’s response to the uncertain situations they are facing in discovery work. And uncertain situations happen to companies in all sorts of stages (startups, scale-ups, enterprises). Whether a team is building a new product or improving the existing one, there is always a need to have a better understanding of the customers and business and make smarter product decisions.

It applies to all types of problems
It is specifically designed to solve both customer and business problems. Whether your goal is to identify new value propositions or motivate your customers to form habits around the core action inside the product (getting them into the habit loop) or help the company improve its acquisition through viral loops, discovery sprints will bring efficiency and structure to your work.

It is a team activity
The product manager is the sprint moderator. However, they should gather a team that balances one another, while also bringing critical skills to the table. Designers can be extremely helpful in discovering the relevant problems and creating and testing the minimum viable solution prototypes. Engineers can provide huge value during discovery ideation and when defining the scope of the experiment.

Learn More About Discovery Sprint

The discovery sprint was first introduced in 2020 in my tech startup, PlayEngine. It was a game-changer, helping us create a top-notch business management solution for sports betting operators. This new approach to product discovery facilitated our acquisition of large-scale customers, all accomplished within the constraints of limited resources.

Starting in late 2022, the official version of the framework and process was introduced to various tech businesses in North America and Europe. Now, dozens of product teams use discovery sprints to build winning products for their target markets.

Ready to uncover how discovery sprints can empower your product discovery and creation processes? Contact me directly via rafayel@theconsultancy.partners.

About the Author

Rafayel Mkrtchyan is a managing partner at theConsultancy. He helps companies improve their product discovery, strategy, delivery, and growth processes. He teaches teams how to run customer and product development processes, set up a winning product strategy, build a sustainable growth model, develop outcome-, growth-, experiment- and data-driven mindsets, as well as robust their lean, agile, and design thinking skills.

About theConsultancy Partners

theConsultancy Partners is a consulting firm that helps businesses to build the path towards digitalization and prepare the organization to support the new economy of Experience-Led Growth.

To help our clients unlock their full potential towards digitalization and Experience-Led Growth we offer a 4-phase program, where we complete:

Analysis:
Perform analysis to find problems and opportunities in products, processes and org structure.

Level up:
Run masterclasses to level up teams to solve those problems and realize opportunities.

Implementation:
Work with teams day to day to implement solutions with our proprietary frameworks and methodologies.

Handover:
Handover all the knowledge and guidelines needed for sustainable growth.