Know your bias before you build
Your job title makes you susceptible to bad product decisions
“What gets us into trouble is not what we don’t know. It’s what we know for sure that just ain’t so.” — Mark Twain
In the fable of the blind men and the elephant, blind men touch different parts of an elephant and argue over what it really looks like. Some touch the trunk, some touch the tail, and some touch the ears. Naturally, they all describe the elephant differently. They’re all correct, in their own limited way — but none of them really understands what an elephant actually is.
Businesses are like elephants in this story — they’re so complex that they’re hard to see all at once. So employees specialize into roles, each with a set of skills and values that help them get their job done. Marketers write ads, engineers write code, designers pick the colors, and so on.
These roles are convenient and necessary, especially to big organizations. Unfortunately that perspective can limit your thinking. Each role has its own built-in biases due to the nature of the role, and this can lead to arguing just like the blind men and the elephant.
In this article, I want to talk about the biases our roles impart on us when we make decisions about our products — specifically the irrational beliefs that a product isn’t successful because of a reason related to one’s job title. In many cases these biases occur because someone is good at their job (in the narrow sense), not despite it.¹
These are just my opinions from working in startups and talking with clients. If you think I’m wrong or missing something, let me know.
Let’s start with the product team roles: product management, design, and engineering. These are the people who are planning every feature and touching the product every day.
Product teams tend to be biased by some abstract, narrow vision of product quality. They try to make things that fit their own personal idea of correctness without really understanding what their customers want.
- Elegant Model Bias — The belief that a beautiful data model is an inherent good, and that good products are naturally tidy and clean under the hood. Sometimes engineers model systems in a vacuum and try to force that model onto the real world — but the developer’s design model and the user’s mental model are usually not a perfect fit. Good products fit users’ lives, and users’ lives have strange and funny shapes, making for less than beautiful models. If engineers design the perfectly abstractable, flexible paragon of clean code, they won’t design for the real world or for human beings.²
- Power Tool Bias — Engineers use highly generalized tools like programming languages to do their jobs, and sometimes they are tempted to design products with the same amount of flexibility. But tools like this have a huge learning curve and are extreme overkill for the vast majority of people. Most people want products that are simple and easy to use, don’t cause problems, and don’t require thinking too much. Our job is to understand what users actually want and when they want it, then design something so obvious that it “just works”. Just because you can, doesn’t mean you should.
- Shipping Bias — This is the tendency to see putting code on a live server as the end of a feature. In reality, the moment code is shipped marks the beginning of your customers’ experience. Engineers can feel excitement or pressure to ship features, even when there’s evidence it will make the user experience worse. Be careful measuring your progress by closed tickets, merged branches, and shipped code — these are only means to an end, not the end goal.
Designers and UX
Quick note: I’m a UX designer, so I’m probably blind to some of my own biases!
- Design Perfection Bias — the belief that the design of this product is what’s keeping people from using it. In reality, the exact color of your buttons is unlikely to be the thing that makes your project a success. This can also be true for usability — if nobody wants a product in the first place, making it easier to use won’t help.³
- Research Purity Bias — It’s true that product decisions should be made after talking to users, but some UXers take this to the extreme. To them, product teams are like an academic research lab that can only publish with sufficient evidence. But real businesses make decisions quickly, and research by academic standards makes early product development too slow. (Also UX people aren’t the only people talking to customers, and sometimes could stand to learn a lot from sales.)
Product managers are hired to talk to customers and make good decisions about the product, so they should be the least biased of all. But there are a few things that slant their decisions.
- Grind Bias — Product managers are supposed to be the voice of the market, but there are an infinite amount of other demands placed on PMs. Some product managers get lost in the hustle of trying to make everyone happy and forget to take time to build a great product. This can be exacerbated when PMs aren’t empowered to make decisions or goals from leadership are unclear, putting PMs in reaction mode. Work hard, but don’t let your nose get so close to the grindstone that you can’t see anything else.⁴
- Retained Bias — Many PMs used to be engineers, designers, or marketers, which usually means they understand one aspect of product much better than the other aspects. In my experience, this can cause them to keep the biases they used to have from their previous role. For example, if a PM used to be a designer, they might still over-value visual design and postpone resolving technical debt.
Now let’s talk about the other half of the company — the marketers, the people selling and managing accounts, and customer support staff.
People in this group tend to push for features without a good understanding of how they affect product quality, which leads products toward local maxima and bad user experiences over the medium and long term.
- Armchair Maker Bias — Sometimes marketing, sales, and support people get frustrated because they perceive obvious flaws in the product. Great products look simple, and sometimes that leads marketers to mistakenly think they could fix everything if they were in charge. Steve Jobs said “innovation is saying ‘no’ to 1,000 things”.
- Feasibility Blindness — Product people talk about finding a product that is technically feasible, financially viable, and desirable to customers; people with this bias are basically forgetting the feasibility aspect. “People would really love to buy a flying car, why aren’t we working on that?”
- The Whale Client Bias — The belief that if a customer with a lot of money wants a feature, we should build that feature. People in sales tend to be biased towards individual customers who can buy a lot, especially because that’s often how their commission incentives are set up. This leads to teams building one-off features that are good for a single customer when it would have been better to build something for 100 medium-sized customers. This bias makes sense for consulting shops doing work for hire, but it’s deadly to startups with plans to scale.
- Squeaky Wheel Bias — This is the belief that complaints to customer support are representative of customers’ experience of the product in general. Customer support people spend a lot of time solving problems on the phone, so they’re tempted to prioritize features based on what they hear the most complaints about. But customers don’t call for support when things are going well, or when the product is so unremarkable they fall out of the funnel before becoming customers at all.
- The Feature Hot Potato — The belief that if lots of customers request a feature, it’s a good idea to build that feature. When a customer wants a feature, they complain to customer support, who can toss the “hot potato” to product as a feature request. But many feature ideas that sound good at first are bad in practice — even ones requested by customers! They might overcomplicate the product for others, or are too prohibitive to build. The better approach is to pass on the root problem the customer is trying to solve and collaborate with product teams to address it in creative ways.
Now that we’ve covered the two “halves” of the company, let’s look at the people at the top.
Leadership can be biased by the 10,000ft view they have of the business and their customers. The devil is in the details, and busy executives often don’t deal with details.
- Big Vision Bias — The belief that the founder’s first conception of the company’s future is inevitable (see Manifest Destiny), and that deviation from that vision is giving up or a distraction. Since founders created the company, they tend to see it as their baby and get overly defensive. In reality, no founders’ vision survives first contact with customers, and the business that best serves customers will have differences from the founders’ original vision.⁵
CEOs and Executives
- Growth Bias — The belief that increasing the company’s near-term revenue, profits, or valuation is automatically a good thing. CEOs have bosses : the board will fire the CEO if the company doesn’t make money. To keep their jobs, some CEOs try to grow at the expense of the long-term health of the organization. It takes a brave person to turn away quick profits in favor of the long-term success of the business.
- Don’t Rock the Boat Bias — People managers are responsible for making people feel happy and productive at work. This is critically important, but can sometimes clash with the shifting priorities of the top brass. I’ve seen some managers that resist shutting down a project for fear of disappointing their team, even if they agree it’s proven to be unviable and investing in it further would jeopardize the company. These kinds of decisions can be tough for managers to explain to teams, but if you prioritize employees’ pet projects over your customers you’ll fail.
Here are some of the biases that all employees hold (as opposed to customers).
- Shortsighted Metric Bias — This is a more generalized version of the Growth Bias. Sometimes we are measured by our ability to move key metrics in the business, whether it be some release deadline, revenue, profit, net promoter score, hiring goals, or something else. Sometimes this drives us to make the wrong decision, like getting website visitors that don’t convert or hiring the wrong person for the job. Narrow or short-sighted goal-obsession can lead you away from serving users, the dilution of your brand, and incurring technical debt. (One way to avoid this is by pairing multiple measurements that offset each other.)
- “This Company Matters” Bias — The belief that it’s important to the world that your company exists. In reality, your product might be useful, but the world doesn’t care who makes it. Even if your brand has a devout following, what customers love is your brand — not your office building or your org chart. Your company is important to employees because it gives them an identity and community, but customers generally don’t care. One way this manifests itself is the unfortunate tendency of companies to organize their product around the structure of the company rather than the other way around.
What can we do about it?
Bring a diversity of roles into product decisions to help counteract bias. This does not mean everyone gets to vote on an answer, it just means you should be talking to lots of different people before making a decision.
Make it everybody’s responsibility to find a product that you can build and sell. Everyone in your company should be working toward the same goal: product/market fit. Hire people who think about products holistically, not people who perform their role with blinders on. Don’t over-specialize roles too early.
Finally, simply recognize your own bias. As a designer, I sometimes have to remind myself that pixel perfection and usability might not be the most important things for the project I’m working on. Give your teammates the benefit of the doubt when they raise concerns. You’re not going to get your way all the time, and that’s a good thing.
Originally posted at https://sampiercelolla.com/articles/know-your-bias-before-you-build/
- Here’s a great list of cognitive biases on Wikipedia
- See also premature optimization
- “Perfect is the enemy of good” — Voltaire
- This popular Ben Horowitz article describes the bad form of grinding well: Good Product Manager, Bad Product Manager
- See also sunk cost fallacy