5 Simple Questions to Drive Validated Learning

John Cutler
Jul 23, 2016 · 4 min read

Here is a simple model that frames product development as an effort to trigger a change in user behavior. It is accessible and not too jargon-laden. It can be used to describe things on the highest level — a whole business, for example — or a specific, low-level design change.

Image for post
Image for post

Whose behavior will change because of the work?

Our product development efforts target people who share some common traits and characteristics. What are those traits and characteristics, and who are the people that meet those traits and characteristics?

Example: Customer Service Reps (CSRs) at technology savvy companies and their Customers.

What is their current behavior?

What pattern of behavior are we hoping to change? This behavior can be extremely specific — like a particular navigation flow or interaction — or it can be very broad like purchasing habits, lifestyle choices, or major life decisions. How often does it happen? How ingrained is the behavior? How does the Target benefit from the existing behavior, or at a minimum put up with it?

Example: Customers currently submit cases via an online form, and then wait up to 24hrs for a reply. The CSRs currently monitor a case queue and communicate with Customers using email and on rare occasions, the phone. The VP of CS believes that opening up a direct channel will overwhelm the team and may not provide them flexibility in responding to case volume.

How will their behavior change?

Something will change as a result of our intervention. Will the user start or stop doing a particular activity? Will they do something more or less often? Will their outlook change? Will they give you more money and your competitors less money? Will they use your impressive new search feature instead of spending time navigating the tab structure?

Example: Customers will use a communication channel “in-app” that resembles a chat interface instead of using the online case submission forms. They’ll feel comfortable interacting with a rep via chat. CSRs will respond to these in-app messages as if they were virtual telephone calls (and do so immediately, personably, and directly).

How will we intervene?

In the example above, the behavior change actually includes the proposed Intervention (in-app chat). The intervention might be a change in design, a change in tactics, or a change in in packaging, messaging, or technology. We’re mixing the pot in some way and hopefully something will happen.

Example: A new online home selling platform triggers a Target who previously would only sell using a broker to take on the task of selling their own house, and to fork over a fixed fee of $2,000. The Intervention is a virtual coach that adapts advice to local and regional requirements for home selling.

How will the new behavior benefit the user/business?

The behavior change must result in some benefit for it to “stick”. Without a significant payoff, you’ll be hard-pressed to persuade someone to change their mindset and habits. Big payoffs can reinforce significant behavior changes (like paying your company a lot more money). The benefit is the return on investment, where the investment is some shift from the status quo.

Example: Customers will enjoy the personal touch. CSRs will deliver faster customer service without the hassle of a phone conversation. The CSR’s employer will have higher customer retention while keeping their CSR budget reasonably small.

Validated Learning

In the spirit of validated learning, we attempt to validate the various assumptions — explicit and implicit — in the framework:

Image for post
Image for post
  • The Target exists, and can be isolated and targeted consistently

In most cases, just validating one piece of the puzzle (the Outcome for example) is not sufficient. You need the full picture. And it is easy to get bogged down in the weeds — a particular intervention, for example — and lost sight of the major benefits required to really change the ballgame.

Innovation

Additionally, on a high level we can also use this framework to understand the various types of innovation. Some examples:

  • Discovering new and underserved Targets

Innovation changes one or more parts of the equation in new and novel ways.


And that’s that. Talk these questions over with your team, and prioritize your product development efforts to successively validate the various parts of the framework.

John Cutler

Written by

Multiple hat-wearer. Prod dev nut. I love wrangling complex problems and answering the why with qual/quant data. @johncutlefish on Twitter.

John Cutler

Written by

Multiple hat-wearer. Prod dev nut. I love wrangling complex problems and answering the why with qual/quant data. @johncutlefish on Twitter.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch

Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore

Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store