Outcomes as Enablers of Business Impact
Raise your hand if you’ve ever been in a situation where you felt like a team you were a part of, or even an entire organization that you worked for, spent much of its time and attention on things that were unlikely to have significant business impact, or improve the lives of its customers. (If I were to say this to just about any random group of people, a lot of hands invariably go into the air, partially because it’s written in the past tense — it feels safer to say this about somewhere in our past than it does in our present.)
Signs That We’re Focusing Too Much on Outputs
As Christophe Achouiantz points out in the blog post Output vs Outcome vs Impact, it’s a common anti-pattern for organizations to focus a significant amount of attention on outputs, while outcomes are often an after-thought, or worse still, are rarely if ever considered. Chris calls out multiple warning signs of too much organizational focus on outputs. For example:
- Devoting a great deal of time and energy to balancing the sides of the iron triangle, aka the “triple constraints” — time, scope, and cost (it’s certainly desirable to finish on time, for instance, but an on-time delivery that the customer hates is a classic example of focusing on outputs, such as a date, at the expense of achieving meaningful outcomes, like solving a customer problem)
- Endless debates over small details, like whether a particular user story is 5 story points or 8 story points, or excessive focus on vanity metrics
- Heavy emphasis on optimization of one part of the value stream, which can certainly make a difference, and yet, if nobody is paying attention to what customers actually care about, we can end up delivering more of what the customer doesn’t want, faster
Success Theater
A significant amount of organizational dysfunction also has to do with what John Cutler describes as “Success Theater,” aka working in a Feature Factory, where organizations focus on things that are not delivering much business value. Below are a few of the Feature Factory manifestations that John writes about:
No measurement. Teams do not measure the impact of their work. Or, if measurement happens, it is done in isolation by the product management team and selectively shared. You have no idea if your work worked
Rapid shuffling of teams and projects (aka Team Tetris). Instead of compelling missions or initiatives, teams deal in feature and project assignments. Chronic multitasking and over-utilization
Success theater around “shipping” with little discussion about impact. You can tell a great deal about an organization by what it celebrates
Infrequent (acknowledged) failures and scrapped work. No removed features. Primary measure of success is delivered features, not delivered outcomes. Work is rarely discarded in light of data and learning. Often the team lacks the prerequisite safety to admit misfires
Note: See also John’s more recent Feature Factory post.
Differentiating Between Outputs, Impacts, and Outcomes
In his book Outcomes Over Output: Why customer behavior is the key metric to business success, Josh Seiden offers some great advice about what things make the biggest difference, and why. Consider Josh’s example below:
Imagine that you work at a charitable organization and you’ve been asked to build a well in a small village that lacks modern plumbing. You’ve been given funding by a foundation that wants to increase the standard of living in this village. They have observed that villagers spend a large amount of time every day walking to the river to carry water. The foundation believes that if the villagers had a well in the center of the village, they wouldn’t have to carry water such long distances anymore, and they could use their time for other activities–ones that would allow them to improve their standard of living.
Josh goes on to point out that in the social impact sector, one model that is popular is called the Program Logic Model, which looks like this:
Resources > Activities > Outputs > Outcomes > Impact
Using that model for the well project, it might look something like this:
- Plan the resources (people, materials, money) we need
- Undertake a set of activities (travel to village, acquire materials, build well)
- Create the output (the finished well)
- (Assuming the well works as planned) Achieve the desired outcome (people in the village spend less time carrying water)
- (Assuming the outcome is achieved) The well is an important factor in the desired impact — higher standard of living in the village
“Notice that the outcome–people spending less time carrying water–is a change in behavior that creates positive results… To see if our work is actually making a difference, we need checkpoints that are smaller, measurable, and more closely connected to the work that we’re doing. That’s where outcomes are important. By setting our outcome as ‘villagers spend less time carrying water’ we have an easier time assessing the quality of our work.”
Thinking in Terms of Outcomes
It takes practice to think in terms of outcomes, because, in reality, few organizations think in those terms when it comes to how they articulate business value, and the extent to which they view the value of what they deliver through the lens of successful customer outcomes.
Let’s say that we tell a team that their work will help make the business more profitable, or reduce business risk (to name a couple of common examples). At first glance, we might be tempted to think that making the business more profitable, or reducing business risk, sound like desirable objectives. That’s all well and good, but suppose there are multiple variables at play. For instance, a release might include many new features. We need to make sure we understand which things in the release have a causative relationship when it comes to customer satisfaction.
Note: For additional thoughts about metrics that matter, see this Excella guide I recently authored: Measuring Impact with Agile Metrics.
To put a finer point on the difference between outputs and outcomes, Josh goes on to say the following in his book:
We need to ask our teams to work on outcomes–the smaller, more manageable targets that, taken together, will create the impact we want. We do this by asking them to focus on changing customer behavior in a way that drives business results.
You can manage a team by telling them what to make: that’s called managing outputs. It’s a problem because features don’t always deliver business value. You can manage a team by asking them to target some high-level value, like growing revenue. That’s called managing impact. It’s a problem because it’s not specific enough.
What you want is to manage with outcomes: ask teams to create a specific customer behavior that drives business results. That allows them to find the right solution, and keeps them focused on delivering value.
Outcomes, Experiments, Hypotheses, and MVPs
Now let’s think in terms of practical application of some of the ideas mentioned above. One way to get there is to leverage some of the ideas from Lean Startup, where there is a heavy focus on experimentation. Returning to Josh’s earlier example, about building the well in the village:
When you start thinking critically about value delivery instead of features, you very quickly run into a problem: how can we be sure that the stuff we’re making is actually going to deliver value? For example, how do we know that the well in the middle of the village is actually going to raise the standard of living in the village? The simple answer is that you frequently can’t know in advance. This is why, when working with outcomes, you need a companion tool: the experiment…
Think about the idea that a well would increase the standard of living. How could we test that? Maybe we could bring in a small gas- powered pump and some hoses, and pump water from the river to a central location in the village for a week. We could see then how people might use their extra time. Would it make a difference in their quality of life? Testing our ideas this way, before we commit to an expensive construction project is an effective way to manage uncertainty in our planning.
There are many misconceptions and failed attempts to define a Minimum Viable Product, or MVP. For the sake of keeping this brief, let’s simply call an MVP a “controlled experiment which makes it possible to get meaningful feedback from a potential customer.”
Surfacing Outcomes: The Magic Questions
One of the most powerful things in Josh’s book is the “magic questions,” which can help us remember to focus on business outcomes:
1. What are the user and customer behaviors that drive business results? (the outcome we’re trying to create)
2. How can we get people to do more of those behaviors? (small experiments such as software changes, policy changes, promotions, and so on that are designed to cause those behaviors to happen)
3. How do we know that we’re right? (tests and metrics that can help assess progress)
User and Customer Behaviors that Drive Business Results
To wrap this up and help us move from abstract concepts to actionable steps, if we truly focus on user and customer behaviors that drive business results, it fundamentally alters the way in which we approach planning activities. Let’s face it, if we take a close look at the conversations that take place in many planning contexts, we spending a lot of time talking about what we’re going to build, and how we’re going to build it, without understanding much about the extent to which it will actually make customers’ lives better.
Tip: For additional insight on which things tend to make the biggest difference when it comes to what we measure, and what those things can and cannot tell us, check out Troy Magennnis’ talk from the Agile 2018 conference: What’s the Story About Agile Data? In his talk, one of Troy’s observations is that based on the many organizations he visits, when it comes to what they spend their analytical/reporting energy on, he sees huge effort spent on data related to current status (progress reporting), a significant amount of data related to portfolio management concerns (selection and prioritization), and painfully little data about customer validation (what “impact” our work is having).
Conclusion
We would be well-served to fashion our conversations in such a way that we cannot possibly forget to answer foundational questions, such as these:
- What are our customers trying to achieve?
- How do they achieve that outcome now?
- What can we do to make it easier for them to achieve that outcome?
As Peter Drucker once said:
Efficiency is doing things right; effectiveness is doing the right thing.
