Bias in Product Decision-Making: Availability Heuristic
Improving product outcomes by understanding how we make decisions.
This article is the second in a series on making better decisions as a product manager by recognizing and combating the heuristics that skew our thinking. I’ll start with a recap of decision-making as a product manager, and how heuristics can bias one of a PM’s most important contributions — sound & consistent decision making. In pt 1 — I focused on the recognition heuristic, and in this article we’ll be looking at the Availability heuristic.
Product Decisioning Making
Product management is centered around informed decision-making. Whereas engineers produce code, designers design, and analysts analyze, project managers primarily produce decisions.
Many folks have espoused different approaches and principles for successful long-term decision-making as a PM. My favorite is what I’ll call “The 60% Rule” from Brandon Chu’s “Making Good Decisions as a Product Manager.”
Chu defines being a good decision-maker as doing two things:
- Making decisions using the right amount of information
- Making decisions as quickly as possible
The aim isn’t to get every decision correct, or even to try to get as much confidence in every decision that is made. Rather, he advocates for a model of trying to get enough confidence to make ~60% of decisions correctly. Time is focused on more confidently making the smaller number of decisions that are truly important, while the majority of relatively less-important decisions get made quickly, without too much information.
PMs are required to make numerous decisions every day that can either speed up or slow down progress in a given direction. We want the direction we choose to be the right one, but a tension exists between how quickly we can pick a direction and how confident we can be in its accuracy, due to the information we must gather and synthesize in order to decide.
Gathering all the information required to make a 100% confident decision takes time. Increases in the information needed to increase confidence from 70%–99% require the most amount of time and tend to deliver the least amount of incremental value.
So what’s the cure? Figuring out which decisions are important enough to require the most information and time, and which can be made quickly with less certainty and information.
All PM decisions fall somewhere along this spectrum of importance and effort, and tools such as prioritization principles and KPIs can guide where and when to focus.
Decision-making and heuristics
I recently read Daniel Kahneman’s Thinking Fast & Slow. The connection between this model of PM decision-making and Kahneman’s model of System 1 and System 2 decision-making was evident.
We, as humans, have two modes of thinking: System 1, which is fast, intuitive, reactionary and low effort; and System 2, which is slow, more intentional and more logical.
Here are examples of appropriate tasks for each system:
- System 1: Determine which of two objects is bigger.
- System 2: complex logical reasoning about which outcome is more likely when there are multiple variables at play.
Why do we have two systems?
System 2 takes lots of energy and time, and we’ve evolved to be able to rely on System 1 to help us make faster decisions that, for the most part, serve us well throughout life. What is more interesting is the role a subconscious bias for System 1 thinking can play in leading to illogical decisions, where we really should be employing System 2.
This bias is explained by Kahneman’s theory of heuristics, which are simple mental processes for quickly forming judgments, making decisions or devising solutions to problems. These processes cause us to approach new decisions with new information by relying on historical information that may or may not be relevant and applicable to the decision at hand.
In other words, we substitute what should be a System 2 decision with a System 1 decision, and as a result, we often commit logical errors and end up with the wrong decision.
There are 4 main heuristics considered that seem prone to impact decision making in product:
Recognition: our bias for recognized options over unfamiliar ones
- Replicating any decision that was made before will automatically bias you toward it.
- We need to apply extra-deliberate due diligence in evaluating its merits, and see if the reasons it worked before are also true now.
Availability: our bias for simple, coherent explanations with clear causes and effect
- Decisions that advocate for simple explanations and fixes are more appealing and can lead us to ignore other relevant information or a lack of information.
- Any decision that is based on vivid and relatable customer stories needs to be counterbalanced with the relevant data and base rates.
Representativeness: our bias toward finding relationships between things and using that similarity to make decisions
- Decisions that rely on some similarity between an option and an outcome need to scrutinize whether that similarity is actually correlated with the outcome we want.
Anchoring and adjustment: our tendency to under-correct for priming or anchoring numbers
- If provided with an an initial estimate, regardless of whether it is relevant, make sure to apply more weight to relevant base rates for your decision, rather than the initial estimate or anchor.
Product decision-making and heuristics
Chu’s criteria for evaluating how important a decision is and how much information and time you should spend on it is a fantastic starting point. I tend to agree with him that “good decision-makers are quick decision-makers.” Time not spent on the majority of fast decisions can be reallocated to the fewer, most important decisions.
However, even with the right time and information required to allocations, heuristics can lead to poor decisions in both the important and unimportant decisions.
Why?
First: Making fast decisions without complete information, per the 60% rule, inevitably leads to heuristics biasing our judgement every day. Here, we can draw parallels between Kahneman’s System 1 and Chu’s faster, less-important decisions.
Second: Even with the fewer, more important decisions, heuristics can cause us to make unprincipled decisions that subvert the whole 60% model on the decisions that matter most. In the same way Kahneman notes examples of System 1 taking over when System 2 should be at work, heuristics can bias even the big decisions on which we spend more time and effort.
After all, this is their job in our lives outside product management — to help us make decisions or judgements that are mostly right most of the time, quickly and with less mental energy.
So what do we do?
Awareness is a good starting point. Once we are aware, there are a few simple tactics that can help us guard against heuristic-creep.
Here, I’ll dive deeper on the second heuristic: Availability. I’ll offer an example of how it can arise in product decision-making and propose a strategy for combating it. I’ll follow up with deeper discussions on the remaining two heuristics in subsequent articles.
The Availability Heuristic
Definition
Letting the ease with which an idea can be brought to mind influence the probability of that idea occurring or being the right option to pick.
Put differently, we are biased to favor intuitive explanations that make sense & more cleanly represent the scenario at hand, rather than those that require considering multiple complex inputs and do not cleanly explain the scenario.
People are more easily swayed by vivid stories with clear causes and effects.
We are less confident in complex explanations with numerous causes and effects at play.
In fact, we favor stories that are coherent so much, that we have a bias to ignore information or a lack of information that makes the story less coherent.
Example in product
We are tasked with prioritizing features for the next quarter with the goal of increasing our banking app’s primary KPI — % of customers making monthly payments with their account.
When doing so, we come across a recorded piece of customer feedback from Jill explaining that she loves our product, and wishes she could use it to spend more but can’t figure out how to add our card to apple wallet, and she usually pays with her iPhone.
Could driving our KPI really be as easy as supporting ease of use with apple pay and browsers?
The image of our customers yearning to use our product, but struggling to fit it into the ways they typically pay is appealing and makes intuitive sense… after-all, we worked so hard designing this great experience for our customers, the main barrier to success being the ability to add to Apple wallet validates our hard work and neatly explains the gap between what we thought would drive usage and actual customer behavior.
In fact, several members of the team can recall struggling to add their cards to their digital wallet in the past as well and reckon if they had been able to they likely would have used it more.
This story is top-of-mind when evaluating our usage data to prioritize for the next quarter, and guess what… the explanation also appears to make sense with our data.
We can see drop-off in our funnel between iOS customers adding money to their account & using the money to make transactions. Looks like they could be having trouble using their Wallet.
We show up to planning and prioritize this feature as a way to drive-up our KPI.
Several months later, with the addition to Apple wallet feature released, we see minimal adoption and are back to the drawing board for ideas to move up monthly payments.
In trying to explain the failed experiment, someone notes that the average adoption of Apple-wallet in our target demographic is actually quite low — 5%.
What happened?
We let the availability of the Apple wallet idea influence our decision, and blind us to the base-rates for Apple wallet usage in our target population.
How to combat.
Data, data, data. To be more accurate, ensuring you have the right mix of quantitative & qualitative inputs before making any decision.
Personas, and customer stories are great for humanizing your decisions and ensuring the team is focused on serving the customer.
But they can also have the adverse effect of making the team over-focused on serving one specific type of customer, falsely assuming that this customer’s problems and needs are indicative of the larger population of people we are aiming to serve.
They can also lead us to ignore numerous reasons your customers might struggle to adopt your product, in favor of a singular more succinct “silver-bullet” explanation that doesn’t accurately represent the diversity of needs in your target market.
Templates that require decisions get made with a mix of quantitative and qualitative data can help.
Furthermore, considering arguments both in favor of and against a given explanation can help expose this bias for availability, and prompt questions for less-available explanations.
Had we asked why the Apple-wallet idea might not work, we likely would have come to the base-rates for Apple-wallet usage in our user base, or at least questioned whether Jill adequately represents our user base, or if there is more diversity in our user base than we thought.
Implementing vigilance against our inherent biases observed in these heuristics will make us better decision-makers and, ultimately, lead to better product outcomes. I’ll follow up with a deeper discussion in subsequent posts on the other three heuristics: recognition, representativeness, and anchoring & adjustment.
