Gusto is reflecting on how we observe our own activity and results. We have been wildly, improbably successful in our niche of payroll and benefits for small businesses. What do we do next? And how do we decide? Our conversation got me thinking.
“How are we doing?” Such a simple question to ask but how very many ways to mess up answering it. “How are we doing?” is immediately followed by, “…and what should we do next?” The frame we use to answer, “How are we doing?” biases the information available to answer, “…and what should we do next?”
The conversation revolved around the motto, “Outcomes over outputs”. I’ve heard this phrase for years, most recently from my friend Barry O’Reilly. It’s always given me a warm feeling, but when it came time to apply it at Gusto I realized that, while I had a gut understanding, I couldn’t really explain it. I dug deeper. This is what I found.
Commerce and Growth
Commerce is the process of creating value and receiving value in return where both parties benefit from the exchange:
This cycle is never static. Changes in demand and competition require growth in the loop just to stay in place. Even seemingly-stable businesses like the corner barbershop have to grow.
Other businesses, like Gusto itself, seek to grow in absolute terms. While we are proud of the tens of thousands of business and hundreds of thousands of those businesses’ employees that we serve, millions more businesses and their employees are still spending too much time, effort, and stress on compensation. We want to grow to serve them all.
On the path of growth we have to decide, among the way-too-many options, which services we will offer next. We need to answer the question, “How are we doing?” on the way to answering, “…and what should we do next?” The picture above is too simple to help.
Commerce, Expanded
On the producer side, we need to:
- Choose what to do next to create value.
- Do it.
- Deliver it to our customers.
On the consumer side, they need to:
- Find what we’ve done (we’ve had long-time payroll customers ask us for help with benefits only to have to sheepishly explain that we’ve offered support for benefits for years).
- Learn to use what we have delivered.
- Use it.
With this simplistic but more-detailed map, we can distinguish various points of observation:
- Effort. How hard are we working?
- Output. How much are we delivering to customers?
- Outcome. How much value are customers realizing?
- Impact. How much value flows back to us?
None of these forms of observation is unimportant, but some are more important in answering, “How are we doing?” and, “…and what shall we do next?”
Rant: Observe, Not Measure
When I arrived at Facebook 9 years ago I was astonished at the degree to which decisions were based on data. As long as I had the data (and a good story to go with it), I could expect decisions to go my way.
Unfortunately, the fashionable Cult of Numeracy has swung past “data is good” to “data is right and no data is bad”. This leads to short-term and narrowly-focused decisions, gaming the system for political leverage, and missed opportunities.
And so I’m careful to say “observations” here. Measurements are a good way to observe, but informed, principled human decision making is the goal and observations are the input for those decisions.
End of rant.
Tradeoffs
How do we choose an observation point if we want to make an informed decision of what to do next? Here are some of the criteria:
- Timely. Quicker is better.
- Cheap. Cheaper to collect and analyze is better.
- Aligned. Observations that automatically align our interests with our customers’ interests are better. For example, if we only make money if our customers make money, then doing good for us automatically does good for them.
- Faithful. Hard to game.
Sometimes these criteria are in conflict. We might take longer to make a more accurate observation. How should we resolve the tradeoffs?
Effort tends to be quick and cheap to observe, but observing it does nothing to align interests (we can work really hard on stuff that doesn’t matter or even actively harms customers) and effort is easy to game. One startup I worked at had the rule that it wasn’t tomorrow if you hadn’t gone to sleep, so if you were missing a deadline you tried to stay awake until you finished (with predictable results).
Output is a bit slower and harder to observe than effort, aligns interests a bit more (at least the customer is getting something), and is easy to game (see all the long feature lists of bloated software).
Outcome is the slowest and most costly to observe, but it does the most to align interests and is the hardest to game.
Impact takes the longest to observe, is cheaper to observe than outcome, does an okay job of aligning interests, and is possible to game.
Conclusion: Outcome > Impact > Output
The hardest problems to solve in deciding what to do next are aligning interests and maintaining fidelity. Hence the preference for:
- Outcome, if possible. Pay-per-use or gain-sharing business models have outcome observation built it.
- Impact, as a flawed proxy for outcome. For example, revenue growth is an impact observation. You might be able to harm customers and increase revenue for a while, which is why we prefer outcome.
- Output, as a flawed proxy for impact.
Appendix: Observing My Own Content
I decided to test the Outcome > Impact > Output model by applying it to my content production. Here are the results for the most recent quarter.
