Designing early customer learning
How to test assumptions and reduce risk, quickly
Early in a project, we often encounter two types of people. The first type prefer to move immediately from idea generation to requirement writing. They view user testing as expensive, and something that happens only after careful planning and thinking. End users are considered, but only in abstract until the product or service is ‘ready.’
The second type just want to get out and try something. They are quick to act and recognize the value of getting customer feedback early in a development process. In our post-lean startup world, the second tends to receive the most praise. However, they are equally dangerous to a new idea as those who may wait too long to test and learn.
It’s important to learn well.
Learning well is faster, reduces more risk, and can have just as much impact on an idea’s success as the idea itself.
Learning well doesn’t require a larger budget, longer timeline, or expensive consultants. Anyone on a project team (and as many as possible) can design better learning by following a few simple steps.
Start by clarifying your learning goals, assumptions, and hypotheses
What are you ultimately trying to accomplish? What does success look like? What do you believe about your users and customers? You’re not ready to test anything until the team has alignment on what you’re testing and why.
Prioritize assumptions on two dimensions: uncertainty and risk
We want to understand the value of validating our assumptions. Testing is primarily about reducing risk and building confidence in an investment decision. Start with the assumption that carries the most risk and is most uncertain.
Make your assumptions testable
Not everyone thinks in testable terms. It’s easy to skip this step in the excitement to get out and test. But unless you can validate or invalidate a hypothesis, your learning will be suspect. I don’t suggest you follow a strict scientific method by isolating all other variables, but you at least need awareness of the factors that could influence the feedback you receive. A good hypothesis has success criteria embedded in it. E.g. 35% of customers will choose the premium service tier after reviewing all tier options.
More resources on writing testable hypotheses:
Ask yourself: What is the smallest thing we can do to test this hypothesis?
There are three factors to consider:
- Who is your target audience for the idea?
- Do you have quick/easy access to them?
- How does the target audience typically evaluate ideas like yours?
How you answer each question will affect what you build, if anything, to test your hypothesis. If you don’t have easy access to your target audience, you may have to explore remote testing or extend your timeframe for learning. Your prototype will look much different for an audience that evaluates ideas through product reviews than one that relies on free trials.
Create the “learning vehicle”
The learning vehicle should mimic how your audience typically evaluates similar solutions. If you’re testing an app, use PowerPoint or Balsamiq. If you’re testing a service, use a video, storyboard, or use the project team as actors. If it’s a physical product, create a landing page or brochure.
Your learning is going to be even more powerful if you have something ready to sell and can get customers to make a purchase or commitment. There can be a wide gap between what a customer says they will pay and what they actually pay for a solution. You want your learning to reflect reality to the greatest degree possible. I recommend reading more about Learning Launch, and getting a copy of the Sprint book.
Test with target users
When you’re testing early concepts, the most important questions to ask are:
- How might you use this?
- Why would you use something like this?
These two questions help unpack two critical factors: use cases and value. There are too many factors to consider when testing with users than can be covered in this post, but check out day 3 and day 4 of the GV Research Sprint process for more details.
Capture your learning
Poor documentation is the biggest mistake most teams make when running learning experiments like these. Revise your hypothesis based on your learning. Capturing what you learned, how you learned, and what you did next are all critical parts of learning well. It helps others learn from your success or failure, and bolsters your credibility with other stakeholders.
Your ultimate proof of success will be greater capacity and a better product-market fit.
