Think About Outcomes
SCRUM & LEAN UX | Episode 9
What are the changes in customer behaviour that indicate your team has solved a real problem for them?
Lean UX Principle: Outcomes, not output
“Features and services are outputs. The goal they are meant to achieve are outcomes.” — Lean UX, by Gothelf & Seiden
Features achieve something, but because the feature is “Done”, doesn’t mean it achieved its purpose. Unfortunately, that’s often how product teams behave. The feature is an assumption. An outcome is a measurable change in customer/user behaviour.
Defining measurable outcomes can be tricky. I asked a company owner how they are monitoring customer engagement and happiness. I asked what customer behaviour is evidence the customer is satisfied. He replied: “If they pay their bills on time”. I laughed, but he wasn’t joking.
A Product Owner (PO) thinks it would be a good indicator if the time someone spends on the site increases. This PO assumes that the longer a visitor spends on the platform, the more connected they are.
A different Product Owner argued a shorter time on site is a good indicator. Visitors find what they are looking for faster. Search functionality, navigation or site performance might have improved.
So what are benefits for the end-user? Remember: ‘buying reasons’, not ‘selling points’. What are they asking for and why? What encourages potential users to potentially seek a solution?
Persuading customers is hardly ever a sound strategy in marketing. Helping them is.
There are Development Teams that, without ever interacting with users, develop user interactions. Lean UX teaches Scrum Teams to think ‘outcomes’ over ‘output’. Rather than mind-numbingly being force-fed what to build, they’ll crawl out of their caves and journey into the world they are about to change. Perhaps they even get to develop some empathy for those they are developing solutions for.
“Teams using Lean UX must be empowered to decide for themselves which features will create the outcomes their organisations require.” — Lean UX, by Gothelf & Seiden
The Scrum Guide also tells us that it’s the Development Team that determines the functionality to be build. In Scrum, becoming more adaptable requires changes in behaviour. Crawling out of the cave is a part of that change. And yet, change is not a destination.
Defining and measuring outcomes
In any project or campaign assumptions are made. Too often though, these assumptions aren’t validated right until the very end when the budget is spent and the timeframe has expired.
Working through a canvas (like the one above) will help teams define outcomes and distinct them from needs and features. It is about defining behavioural shifts up front. This brings us to Behaviour Driven Development (BDD).
But, stubborn (and somewhat lazy) creatures that we are, even when we define outcomes upfront, are we really measuring them?! We generally still suck at measuring what we build and learning from what we measure. I too have to acknowledge I am not even close to achieving my desired level of user engagement. With each new team I work with it takes a lot of time to get the hang of it again.
There can be a fair bit of bias too when results don’t turn out to be what we expected them to be. Like how a doctor might assume a patient didn’t follow the prescription, developers might assume the user just isn’t using the feature right.
So, the activities performed post-delivery acts as evidence a Scrum Team grasps the concept of ‘outcomes’: measurable changes in behaviour that indicate a real problem is solved.
- What activities is your team doing, to learn if they are right or wrong about their deliveries post-review/delivery?
- What are they doing to counter their success bias?
- Are they learning through indirect or direct interaction with end-users?
- Are they making a distinction between qualitative and quantitive data?
- Are they collecting stories?
This may sound ironic, but I’d argue it’s not about what Development Teams develop. It’s about the stories that are told through those developments. I am not talking about User Stories. Great stories don’t start with “As a user I want a […]”. This turns the user into a spoiled kid in the eyes of a developer.
Think of a development that took place that moved someone. There is your story. You know, a narrative about a real experience involving behaviour and emotion. We’re talking UX afterall.
Velocity is not value delivery
Many Scrum Teams make their velocity transparent; the speed at which they deliver. Velocity is indicated by Story Points or Estimated Effort (work hours). Both are output, not outcomes. They show originally estimated complexity or effort resolved, but not value delivered. It’s okay to monitor velocity, but it’s often monitored for the wrong purpose. Even though velocity can sky-rocket, both value delivery and customer experience could be plummeting. Velocity goes up, but the team might very well have been busy squeezing out and serving a massive turd.
Better experiences over working features
Everyone involved in the process will act (in a way) as a designer. But the users are rarely involved in this process. Everyone involved will exert influence on how they believe something should look and function. The creative process is limited by output oriented ad-hoc power-plays. That results in compromised experiences. Instead outcome orientation creates a larger view of the work beyond features.
Thanks to Willem-Jan Ageling for reviewing this episode.