Thinking in Bets: Making Smarter Decisions When You Don’t Have All The Facts — Book Notes

Annie Duke

Si Quan Ong
Aug 12, 2018 · 13 min read

Published: 2018

  • A bet: a decision about an uncertain future. The implications of treating decisions as bets made it possible for me to find learning opportunities in uncertain environments. Treating decisions as bets, I discovered, helped me avoid common decision traps, learn from results in a more rational way, and keep emotions out of the process as much as possible.
  • Thinking in bets starts with recognizing that there are exactly two things that determine how our lives turn out: the quality of our decisions and luck. Learning to recognize the difference between the two is what thinking in bets is all about.

Chapter 1: Life is Poker, Not Chess

  • Pete Carroll was a victim of our tendency to equate the quality of a decision with the quality of its outcome. Poker players have a word for this: “resulting”. When I started playing poker, more experienced players warned me about the dangers of resulting, cautioning me to resist the temptation to change my strategy just because a few hands didn’t turn out well in the short run.
  • Resulting is a routine thinking pattern that bedevils all of us. Drawing an overly tight relationship between results and decision quality affects our decisions every day, potentially with far-reaching, catastrophic consequences.
  • When we work backward from results to figure out why those things happened, we are susceptible to a variety of cognitive traps, like assuming causation when there is only a correlation, or cherry-picking data to confirm the narrative we prefer. We will pound a lot of square pegs into round holes to maintain the illusion of a tight relationship between our outcomes and our decisions.
  • Our goal is to get our reflexive minds to execute on our deliberative minds’ best intentions.
  • “No, no,” von Neumann said. “Chess is not a game. Chess is a well-defined form of computation. You may not be able to work out the answers, but in theory there must be a solution, a right procedure in any position. Now, real games,” he said, “are not like that at all. Real life is not like that. Real life consists of bluffing, of little tactics of deception, of asking yourself what is the other man going to think I mean to do. And that is what games are about in my theory.”
  • Chess, for all its strategic complexity, isn’t a great model for decision-making in life, where most of our decisions involve hidden information and a much greater influence of luck. This creates a challenge that doesn’t exist in chess: identifying the relative contributions of the decisions we make versus luck in how things turn out.
  • Poker, in contrast, is a game of incomplete information. It is a game of decision-making under conditions of uncertainty over time. Valuable information remains hidden. There is also an element of luck in any outcome. You could make the best possible decision at every point and still lose the hand, because you don’t know what new cards will be dealt and revealed. Once the game is finished and you try to learn from the results, separating the quality of your decisions from the influence of luck is difficult.
  • The quality of our lives is the sum of decision quality plus luck.
  • We make this same when we look for lessons in life’s results. Our lives are too short to collect enough data from our own experience to make it easy to dig down into decision quality from the small set of results we experience.
  • We get only one try at any given decision — and that puts great pressure on us to feel we have to be certain before acting, a certainty that necessarily will overlook the influences of hidden information and luck.
  • We are discouraged from saying “I don’t know” or “I’m not sure”. We regard those expressions as vague, unhelpful and even evasive. But getting comfortable with “I’m not sure” is a vital step to being a better decision maker. We have to make peace with not knowing.
  • What makes a decision great is not that it has a great outcome. A great decision is the result of a good process, and that process must include an attempt to accurately represent our own state of knowledge. That state of knowledge, in turn, is some variation of “I’m not sure.”
  • An expert in any field will have an advantage over a rookie. But neither the veteran nor the rookie can be sure what the next flip will look like. The veteran will just have a better guess.
  • If we misrepresent the world at the extremes of right and wrong, with no shades of grey in between, our ability to make good choices — choices about how we are supposed to be allocating our resources, what kind of decisions we are supposed to be making, and what kind of actions we are supposed to be taking — will suffer.
  • When we think in advance about the chances of alternative outcomes and make a decision based on those chances, it doesn’t automatically make us wrong when things don’t work out. It just means that one event in a set of possible futures occured.
  • Any prediction that is not 0% or 100% can’t be wrong solely because the most likely future doesn’t unfold.
  • Decisions are bets on the future, and they aren’t right or wrong based on whether they turn out well on any particular iteration. An unwanted result doesn’t make our decision wrong if we thought about the alternatives and probabilities in advance and allocated our resources accordingly.
  • When we think probabilistically, we are less likely to use adverse results alone as proof that we made a decision error, because we recognize the possibility that the decision might have been good but luck and/or incomplete information (and a sample size of one) intervened.
  • Redefining wrong allows us to let go of the anguish that comes from getting a bad result. But it also means we must redefine “right”. If we aren’t wrong just because things didn’t work out, then we aren’t right just because things turned out well.

Chapter 2: Wanna Bet?

  • Whenever we choose an alternative, we are automatically rejecting every other possible choice. All those rejected alternatives are paths to possible futures where things could be better or worse than the path we chose. There is potential opportunity cost in any choice we forgo.
  • Our decisions are always bets. We routinely decide among alternatives, put resources at risk, assess the likelihood of different outcomes and consider what it is we value.
  • In most of our decisions, we are not betting against another person. Rather, we are betting against all the future versions of ourselves that we are not choosing.
  • At stake in a decision is that the return to us will be greater than what we are giving up by betting against the other alternative future versions of us.
  • We bet based on what we believe about the world.
  • Part of the skill in life comes from learning to be a better belief calibrator, using experience and information to more objectively update our beliefs to more accurately represent the world. The more accurate our beliefs, the better the foundation of the bets we make.
  • We form beliefs in a haphazard way, believing all sorts of things based just on what we hear out in the world but haven’t researched for ourselves.
  • Daniel Gilbert: “Our default is to believe that what we hear and read is true. Even when that information is clearly presented as being false, we are still likely to process it as true.”
  • Truthseeking, the desire to know the truth regardless of whether the truth aligns with the beliefs we currently hold, is not naturally supported by the way we process information. We might of think ourselves as open-minded and capable of updating our beliefs based on new information, but the research conclusively shows otherwise. Instead of altering our beliefs to fit new information, we do the opposite, altering our interpretation of that information to fit our beliefs.
  • Once a belief is lodged, it becomes difficult to dislodge.
  • The potency of fake news is that it entrenches beliefs its intended audience already has, and then amplifies them. The Internet is a playground for motivated reasoning. It provides the promise of access to a greater diversity of information sources and opinions than we’ve ever had available, yet we gravitate toward sources that confirm our beliefs, that agree with us.
  • Blind-spot bias (an irrationality where people are better at recognizing biased reasoning in others but are blind to bias in themselves) is greater the smarter you are.
  • When someone challenges us to bet on a belief, signaling their confidence that our belief is inaccurate in some way, ideally it triggers us to vet the belief, taking an inventory of the evidence that informed us.
  • Being asked if we are willing to bet money on it makes it much more likely that we will examine our information in a less biased way, be more honest with ourselves about how sure we are of our beliefs, and be more open to updating and calibrating our beliefs.
  • Offering a wager brings the risk out in the open, making explicit what is already implicit (and frequently overlooked.)
  • What if, in addition to expressing what we believe, we also rated our level of confidence about the accuracy of our belief on a scale of zero to ten?
  • Incorporating percentages or ranges of alternatives into the expression of our beliefs means that our personal narrative no longer hinges on whether we were wrong or right but on how well we incorporate new information to adjust the estimate of how accurate our beliefs are.
  • By saying “I’m 80%” and thereby communicating that we aren’t sure, we open the door for others to tell us what they know. They realize they can contribute without having to confront us by saying or implying “you’re wrong.”

Chapter 3: Bet to Learn: Fielding the Unfolding Future

  • What are the obstacles in our way that make learning from experience so difficult?
  • There is a big difference between getting experience and becoming an expert. That difference lies in the ability to identify when the outcomes of our decisions have some thing to teach us and what that lesson might be.
  • Any decision is a bet on what will likely create the most favourable future for us. How we figure out what — if anything — we should learn from an outcome becomes another bet.
  • To reach our long-term goals, we have to improve at sorting out when the unfolding future has something to teach us, and when to close the feedback loop.
  • If making the same decision again would predictably result in the same outcome, or if changing the decision would predictably result in a different outcome, then the outcome following that decision was due to skill.
  • If our decisions didn’t have much impact on the way things turned out, then luck would be the main influence.
  • For any outcome, we are faced with this initial sorting decision. That decision is a bet on whether the outcome belongs in the luck or skill bucket.
  • The way we field outcomes is predictably patterned: we take credit for the good stuff and blame the bad stuff on luck. The result is we don’t learn from our experience.
  • Maybe the solution that has evolved is to compensate for the obstacles in learning from our own experience by watching other people do stuff.
  • Instead of feeling bad when we have to admit a mistake, what if the bad feeling came from the thought that we might be missing a learning opportunity just to avoid blame?
  • Keep the reward of feeling like we are doing well compared to our peers, but change the features by which we compare ourselves: be a better credit-giver than your peers, more willing than others to admit mistakes, etc. In this way, we can feel that we are doing well by comparison because we are doing something unusual and hard that most people don’t do.
  • A good strategy for figuring out which way to bet would be to imagine if that outcome had happened to us.

Chapter 4: The Buddy System

  • Members of a decision pod can be formed by anyone where members can talk about their decision making. Forming or joining a group where the focus is on thinking in bets means modifying the usual social contract. It means agreeing to be open-minded to those who disagree with us, giving credit where it’s due, and taking responsibility where it’s appropriate, even when it makes us uncomfortable.
  • Complex and open-minded thought is most likely to be activated when decision makers learn prior to forming any opinions that they will be accountable to an audience (a) whose views are unknown, (b) who is interested in accuracy, © who is reasonably well-informed, and (d) has a legitimate reason for inquiring into the reasons behind participants’ judgments/choices.
  • They should also encourage and celebrate a diversity of perspectives to challenge biased thinking by individual members.
  • For engaging in the difficult work involved in sobriety, local AA groups given tokens or chips celebrating the length of individual members’ sobriety. The tokens are a tangible reminder that others acknowledge you are accomplishing something difficult. There are chips for marking one to 65 years of sobriety. There are also chips given for every month of sobriety in the first year.
  • A diverse group can do some of the heavy lifting of de-biasing for us.
  • After 9/11, the CIA created “red teams” that are dedicated to arguing against the intelligence community’s conventional wisdom and spotting flaws in logic and analysis.
  • A growing number of businesses are implementing betting markets to solve for the difficulties in getting and encouraging contrary opinions.

Chapter 5: Dissent to Win

Ideal-type model of self-correcting epistemic community:

  1. Communism — data belong to the group

2. Universalism — apply uniform standards to claims and evidence, regardless of where they came from

3. Disinterestedness — vigilance against potential conflicts that can influence the group’s evaluation

4. Organised Skepticism — discussion among the group to encourage engagement and dissent

  • When presenting a decision for discussion, we should be mindful of details we might be omitting and be extra-safe by adding anything that could possibly be relevant. On the evaluation side, we must query each other to extract those details when necessary.
  • Don’t disparage or ignore an idea just because you don’t like who or where it came from.
  • The substance of the information has merit/lack of merit separate from where it came from
  • One way to disentangle the message from the messenger is to imagine the message coming from a source we value much more or much less.
  • After the outcome, make it a habit when seeking advice to give the details without revealing the outcome.
  • Another way a group can de-bias members is to reward them for skill in debating opposing points of view and finding merit in opposing positions.
  • Lead with assent. Listen for the things you agree with, state those and be specific, and then follow with “and” instead of “but”
  • Ask for a temporary agreement in engage in truthseeking. If someone is off-loading emotion to us, we can ask them if they are just looking to vent or looking for advice.

Chapter 6: Adventures in Mental Time Travel

  • just as we can recruit other people to be our decision buddies, we can recruit other versions of ourselves to act as our own decision buddies.
  • Several organizations and companies with an interest in encouraging retirement planning have resources that allow clients to “meet” their future selves as they make retirement decisions. In the simplest versions of these tools, clients plug in their age, income, savings practices and retirement goals. The apps then show the client the financial situation and lifestyle their future-self can expect, compared with the present.
  • Business journalist and author Suzy Welch developed a popular tool known as 10–10–10 that has the effect of bringing future-us into more of our in-the-moment decisions. Every 10–10–10 process starts with a question… What are the consequences of each of my options in ten minutes? In ten months? In ten years?
  • We can build on Welch’s tool by asking the questions through the frame of the past: “how would I feel today if I had made this decision ten minutes ago? Ten months ago? Ten years ago?”
  • Our problem is that we’re ticker watches of our own lives. Happiness is not best measured by looking at the ticker, zooming in and magnifying moment-by-moment or day-by-day movements. We would be better off thinking about our happiness as a long-term stock holding.
  • Ulyssess contract — involve raising a barrier against irrationality. A kind of precommitment contract designed to lower/raise barriers that interfere with rational action
  • A “decision swear jar” is a simple kind of precommitment contract. We identify the language and thinking patterns that signal we are veering from our goal of truthseeking. When we find ourselves using certain words or succumbing to the thinking patterns we are trying to avoid because we know they are signs of irrationality, a stop-and-think moment can be created.
  • Belief -> Bet -> Set of outcomes
  • Thinking about what futures are contained in that set (which we do by putting memories together in a novel way to imagine how things might turn out) helps us figure out which decisions to make.
  • Figure out the possibilities, then take a stab at the probabilities.
  • The best poker players think beyond the current hand into subsequent hands: how do the actions of this hand affect how they and their opponents make decisions on future hands?
  • Whether it involves sales strategies, business strategies or courtroom strategies, the best strategists are considering a fuller range of possible scenarios, anticipating and considering the strategic responses to each, and so on deep into the decision tree.
  • Prospective hindsight — imagining that an event has already occurred — increases the ability to correctly identify reasons for future outcomes by 30%
  • Backcasting: we imagine we’ve already achieved a positive outcome, holding up a newspaper with the headline “We Achieved Our Goal!”, then we think about how we got there.
  • Premortem: an investigation into something awful, but before it happens. Imagining a headline “We Failed To Reach Our Goal” challenges us to think about ways in which things could go wrong that we otherwise wouldn’t if left to our own devices.

Like what you’re reading? Then join my monthly reading recommendations newsletter where you’ll get more of such book notes (+ what I think you should read!):

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store