Decision Quality vs. Decision Outcome: How Do You Know You’ve Made a Good Decision?
Think of the most recent product decision you made. Was it a good decision to conduct more customer research, skip usability testing or implement that item from your roadmap?
How do you know if it was a good decision? What makes a good decision? And what differentiates a good from a bad decision — its outcome or the decision itself?
Decision analysis: making decisions under uncertainty
Imagine you are the PM at an oil company, and your job is to drill for and find new oil reserves. How would you approach the problem? In decision analysis, Stanford professor Ronald Howard argues to combine all relevant and available information to assess the different options. Decision analysis is particularly helpful in complex decision-making environments, in which the decision can result in vastly different outcomes, such as when drilling for oil. As drilling for oil is very expensive, and you either find oil or you don’t, you’d likely want to take a range of considerations, including geological, biological, economic and business, into account.
Decision quality vs. decision outcome
Let’s assume you collected the available information, and you decide to move forward with location 2. The company gives you all the resources you need, and your team starts drilling. After spending $1 billion drilling a hole, you find oil. Hooray! You declare success!
But what was the alternative here? You and your team go ahead and spend $1 billion drilling a hole. You find nothing. What do you declare now? Failure?
Think about this in a little more detail. Setting aside the frequently overrated role of talent or intelligence in decision making, there are only a couple of scenarios: Your decision quality will be somewhere between low and high, and its outcome will be somewhere between negative and positive.
Let’s briefly walk through the four areas. If a high (low) quality decision leads to a positive (negative) outcome, your bonus will be $500k ($0). In these two scenarios, it does not make a difference if you’re being rewarded for the quality of decision or its outcome.
Looking at the two remaining scenarios, the story changes dramatically. You could be lucky (low decision quality resulting in positive outcome) or unlucky (high decision quality resulting in negative outcome). In the situation in which you’re lucky (unlucky), you’d be wrongly rewarded for your decision if your bonus is based on decision outcome (decision quality).
Measuring decision quality
So, which one is better: being accountable for your decision quality or for the outcome of your decision?
In my experience, it’s more common to reward PMs and their product team for their decision’s outcome. This is much easier to measure. However, it’s fairer to reward PMs for their decision’s quality, as that is the key contribution — before luck and bad luck.
So, how can we measure decision quality? As hindsight is 20/20, a decision’s quality needs to be judged by the underlying decision-making process. Circling back to decision analysis, did you collect all the available information to make the decision? What assumptions did you make? Were those assumptions reasonable and justifiable? How did you weigh risk and return?
To know if you made a good decision, you assess the process by which you made that decision — not its outcome.
If you found this post actionable or insightful, you can find more posts on my personal blog.
