3 ways not to do discovery — how we learned through our mistakes

In my previous article I covered how we mashed up two product discovery frameworks at Babbel — The Double Diamond and Continuous Discovery. When my colleague Anna and I applied this new framework in a project a couple of months ago, hearts full of hope to improve the way we do discovery at Babbel, we made key mistakes that kept us from reaching full effectiveness in our product discovery.

This is a short list of some mistakes we made and what we learned from making them.

We did not have a complete product trio

Thought leaders in the area of product discovery, such as Teresa Torres, suggest that the core product discovery team should consist of a Product Manager, a Product Designer and an Engineer /Tech lead— the product trio. Every single member brings a crucial perspective to the table that needs to be considered in an effective discovery process:

Designer -> Desirability
Product Manager -> Viability
Engineer -> Feasibility

For various reasons outside of our team’s control we, however, didn’t have a product manager or an engineer as a steady member of our core discovery team. The core team consisted of 2 designers. That’s it. A UX researcher, content designer and engineers participated sporadically, but the main load was carried by us two designers. And that was far from ideal:

• Many discussions and decisions took much longer than they should have. Nobody in the team carried the responsibility and authority to make final decisions about the direction of the product.
  A product manager would have been the right person to make calls about when to take informed risks, when we have enough evidence to move forward to higher fidelity experiments, which solutions to abandon etc. Without such a person we struggled through too many Slack and Google Meet discussions to come to conclusions.
• The perspective of feasibility was underrepresented. Not having an engineer as a permanent member of the team increased the risk of us exploring a solution that might be viable and desirable but would take way too long to implement. We were forced to regularly reach out to engineers and ask for their input. That was disruptive for their work flow and tedious for us because of the increased communication overhead.
• On the topic of communication overhead. All stakeholder management and communication over various phases of the project which lasted several weeks was performed by us designers. That kind of workload on top of actually trying to solve a user problem stretched our capacity quite a bit.

💡 Learning for the future 💡
Gather members for your discovery who can represent the desirability, feasibility, viability.

We rushed through ideation

The solution discovery was kicked off with a 3-hour ideation workshop which was based on the principles of design sprints. We chose that kind of workshop instead of a longer-drawn process because we wanted to move fast. So 12 participants from several functions (engineering, product management & design, content design, user research) came together to creatively solve the problem and then choose the most impactful solutions.

After exploring the problem space and individually working on solution concepts for about two hours we had all participants pitch their ideas in 2,5 minute presentations. And that’s when we made our mistake: After hearing all pitches we rushed into voting for the most promising ideas.

Now, voting is indeed the next step of the process, so there was nothing inherently wrong about moving on to the next phase. However, by doing so right after the pitches we missed out on the opportunity of creating a space for ideas to cross-pollinate. Creativity has a way to increase when collaborating. One idea inspires another, the concept improves, we all win. This is true especially in a team setup where various functions come together with their unique perspectives.

💡 Learning for the future 💡
Foster the collective creativity by creating a space where one idea can inspire another.

We fixated on one solution

After selecting the most promising concepts in said workshop, we focused fully on that solution. And while focusing on one solution at a time can be beneficial, it carries the risk of the confirmation bias.

“Confirmation bias is a cognitive error that occurs when people pursue or analyze information in a way that directly conforms with their existing beliefs or preconceptions. Confirmation bias will lead people to discard information that contradicts their existing beliefs, even if the information is factual.” — NN Group

It also holds the risk of the commitment bias.

“The escalation of commitment is a bias in which the more we invest in an idea, the more committed we become to that idea. The more the city explored this idea (and this idea alone), the more committed to the idea they became — despite its flaws.” — Continuous Discovery Habits, Teresa Torres

In her book “Continuous Discovery Habits”, Teresa Torres, recommends to test several solutions at the same time in order to avoid these cognitive errors. Doing that would have helped us to stay open to alternatate, maybe better ways to address the opportunity. Instead, we dove deep into the one solution for that opportunity without exploring and experimenting with adjacent solutions.
This wasn’t because we were oblivious of the risks of focusing on one solution. As it was our first time using this approach, we decided to focus on testing assumptions of one solution alone and agreed on adopting the compare and contrast method in the next iteration.
It lead to us staying below our possibilities, but we know how to improve in the future.

💡 Learning for the future 💡
Avoid confirmation bias by exploring various approaches to address the opportunity

Thank you Anna Stutter Garcia for being my partner in crime and making these mistakes, learning and growing with me.