Product Discovery Methods for a Product Delivery World
Part 2 of 2: A walkthrough of a method for collecting product feedback without sacrificing product discovery.
In my previous post, I discussed our goals for creating a method of concept value testing that collects product feedback while unlocking product discovery insights. In this post, we’ll go through each step of the methodology, step by step.
If you would like to try this approach with your product team, you can download this PowerPoint template. With this template, you can navigate customers through each question during your concept value test. There’s also notes on each slide to help you understand its purpose.
To explore each of the 14 steps of the methodology, let’s imagine we’re collecting feedback on an imaginary smartphone app called Song Catcher.
Song Catcher helps musicians capture their early musical ideas. It provides a variety of features to help artists record clips from their instruments or vocal performances. One of the key features is an A.I. assistant that can not only clean up the audio recordings, but also identify important musical information like the tempo of the recording and the key it’s in. The app also pairs with a Bluetooth foot pedal that allows the musician to record their ideas while keeping their hands free.
Step 1: Introduce the Job-to-be-done.
The best place to start having the customer explore your concept, is not by showing them your concept. We’ll get to that later. First, we want to have a conversation about the job your concept seeks to address.
In our example, the job is capturing early musical ideas. So we’ll start the study by introducing this idea. Here’s how it could sound.
“Thank you for joining us today. We’re going to be collecting your feedback on a concept we’re working on. But before we get to that, I want to talk to you about your process for capturing early musical ideas.”
Step 2: Identify the customer’s desired outcomes.
To adequately evaluate the concept, we want the customer to first think about what’s important when they’re trying capture early musical ideas. What does success look like for them? What are their most important goals? What challenges get in the way of achieving them?
As interviewers, we want to be listening for the components of Ulwick’s “need statement”. Typically, customers are trying to maximize the chance they’ll achieve their desired outcomes, minimize the friction that gets in their way, and/or obtain the greatest value from the tools they’ve invested in.
It’s important that we capture these desired outcomes. We’ll be reminding our customers, throughout the rest of the study, so they can refer to these outcomes as their baseline for evaluating the concept.
We’re also interested in the rich, contextual, details on how musicians approach the job of capturing early musical ideas. When is this most likely to happen? Where do they often engage in this job? Who (if anyone) do they partner with? What tools are they currently using today? How do they know they’re done or ready to move on? These are just some of the questions we might explore during this part of the concept value test.
Step 3: How important is it for you to achieve your desired outcomes?
After the customer articulates their desired outcomes for capturing early musical ideas, we then want to know just how important those outcomes are.
They could be critical to the success of the musician, or they may not be important at all.
I’m choosing to use a Likert scale to help the customer ground their response and for a deeper conversation (e.g., “so you’re saying a 2, can you tell me more about why it’s not important). The scores also help with qualitative analysis later.
The scores are not intended to be calculated for any type of statistical significance as it is unlikely, you’ll talk to enough customers about a single concept to achieve that. As stated in my previous post, these scales are intended to help me gauge the “temperature” of the customers’ responses. In short, I’m using a quantitative design technique to drive a more detailed qualitative discussion.
If you prefer not to use the scales, you can simply remove them and ask each question in open-ended form.
Step 4: How satisfied are you with your ability to achieve your desired outcome(s)?
Combined with the previous question, asking customers how satisfied they are with their current experience gives an important signal into the opportunity space for our concept.
If musicians tell us that capturing early musical ideas is important, and that they are currently unsatisfied with the way they do it today — it may be an important indicator that there is opportunity for Song Catcher in the marketplace.
Conversely, if musicians believe that capturing early music ideas is unimportant or they’re already satisfied with their current way of doing it, this might indicate that there’s not a lot of opportunity for our concept.
For more information about how opportunity scoring works, check out J.P. Carrascal’s fantastic blog post about the topic.
Step 5: How frequently do you engage in the job?
In this example, we want to know how often musicians find themselves capturing early musical ideas. To be clear, just because a customer doesn’t engage with a job frequently, it doesn’t mean that we should avoid concepts that seek to help with that job.
With this question, we’re trying gain important context. Clearly, if we were to identify that capturing early musical ideas is important, musicians are largely unsatisfied with the way they do it today, and capturing early musical ideas is a job they’re always engaged in — we would be right to be excited about the possible opportunity that awaits our concept.
Step 6: Introduce the concept.
We’re now ready to introduce the concept to the customer. In this case, we’ve given the concept a simple, memorable name and a brief description.
Step 7: Present a high-level walkthrough of the overall concept idea.
Now we can walk customers through our concept idea. In this case, we might show musicians some early sketches that convey the overall idea of what we’re proposing. Your visual assets don’t have to be high fidelity. In fact, in many cases, you don’t want them to be too polished — as the customer will get the impression that everything has been built already. They may be less inclined to give you critical feedback. They may instead fixate on minor design details, assuming that your team is going to build Song Catcher, whether they need it or not.
Step 8: Summarize the key benefits and the limitations of the concept.
We’re strong believers in discussing both the benefits and limitations of your concepts with customers.
You may discover that the benefits you think are most important to the customer, aren’t that important to them. Conversely, you might assume a limitation is a “deal breaker” for customers, only to find out that customers don’t see the limitation as a problem at all. You can save countless development hours using this step, as your team can avoid investing engineering cycles in things that don’t matter to the customer.
If you have time, you can even have customers put the benefits and limitations in order, from most important to least important. This activity effectively helps the customer define for your team what the MVP should be.
Step 9: How would this concept affect your ability to achieve your desired outcome(s)?
Now that the customer has had a chance to process your concept, we want them to reflect on how well they believe it will help them achieve their desired outcomes. At this point, it might be good to remind them what they had noted earlier in the interview.
For example, let’s imagine that a musician we’re talking to had noted earlier that for her, it’s important to, “minimize as much technology as possible,” when she’s creatively exploring musical ideas. It would be important to ask her whether she believed this concept would improve or reduce her ability to achieve that.
By focusing our attention on the customer’s desired outcomes, we’re able to go much deeper into their feedback and unlock important insights that can lead to meaningful product discovery.
Step 10: How would you feel if we decided not to release this concept?
This is a great question to gauge the customer’s investment in the idea. If you discover that they are largely “neutral” on whether you release the concept, that may be an indication that your concept is not providing enough value for them to believe that it must be released.
Conversely, if musicians were noting that they would be “very disappointed” if we chose not to release Song Catcher, that could be an exciting early signal for our team.
Regardless of their response, we want to know the reasons why they gave their rating. Use follow up questions like, “What would Song Catcher need to do, for you to be disappointed it wasn’t released?”
Step 11: How unique is this concept for achieving your desired outcome(s)?
It’s not always possible to be aware of every product that might compete with your concept. That’s why it’s a good idea to get a sense of how unique or novel your concept is, compared to other things they may have seen.
In this example, we’ll want musicians to indicate how novel Song Catcher is to them. We want them to compare it to existing tools or methods they’re already using to capture their early musical ideas.
Step 12: How believable is it that this concept would help you achieve your desired outcome(s)?
With this question we’re trying to get a sense of how skeptical customers are about our concept.
Perhaps musicians tell us that they’re skeptical that the Bluetooth pedal won’t suffer latency issues, affecting their ability to easily record ideas. Maybe they tell us that they’re skeptical of how well the microphones on their phones will record their early work.
Maybe they’re not convinced that A.I. can perform the tasks that we claim it will.
These are all fantastic conversations to have with customers before you release your concept onto the market. Asking about the believability of the concept helps us uncover the demand generation and reduction forces that I wrote about in the previous post.
Discussing early skepticism puts you in a much better position when it’s time to launch, because you’re going to have clarity around customers’ doubts. You’ll also know what scenarios you’ll have to deliver on, to make believers out of your earliest customers.
Step 13: How urgent is it for you to have this concept to achieve your desired outcome(s)?
Understanding how urgently your customers need your concept can give you an indicator for how aggressively customers are looking for a solution within the job space you’re exploring. It might also be an indicator of just how painful the job is that your concept seeks to improve.
It should also be noted that if customers are reporting that the release of your concept is “not urgent”, that is not necessarily indicative that the concept is somehow less valuable.
Customers could feel that your concept is so valuable that they want you to take more time, to ensure you get the concept right.
As with all the questions in this method, it’s the combination of the “score” the customer gives, along with the important qualitative verbatims that support their reasoning.
Step 14: What concerns do you have that might prevent you from trusting this concept?
The final question should collect any lingering concerns customers have about your concept. In this example, perhaps customers are concerned that A.I. will improve the quality of their early musical ideas so well, it removes their own creative input.
Questions around trust are important, particularly in technology areas that are emerging or technology that your target audience may find novel or even scary.
This completes all the 14 steps of how we talk about concepts with customers. You can test multiple concepts in a single interview or choose to deep dive on one.
I won’t claim that this approach is without its limitations, however, I have found that it has provided our team with countless insights as customers are able to go far deeper in their analysis of our concepts, compared to other methods that we’ve tried.
Download the template and try it for your next concept. And if you do, be sure to leave me a comment to let me know what you’ve learned.
