The Gambler’s Fallacy is Not a Fallacy
A bold title like that calls for some hedges — here are two. First, this is work in progress: the conclusions are tentative (and feedback is welcome!). Second, all I’ll show is that rational people would often exhibit this “fallacy” — its a further question whether real people who actually commit it are being rational.
Off to it.
On my computer, I have a bit of code call a “koin”. Like a coin, whenever a koin is “flipped” it comes up either heads or tails. I’m not going to tell you anything about how it works, but the one thing everyone should know about koins is the same thing that everyone knows about coins: they tend to land heads around half the time.
I just tossed the koin a few times. Here’s the sequence it’s landed in so far:
T H T T T T T
How likely do you think it is to land heads on the next toss? You might look at that sequence and be tempted to think a heads is “due”, i.e. that it’s more than 50% likely to land heads on the next toss. After all, koins usually land heads around half the time — so there seems to be an overly long streak of tails occurring.
But wait! If you think that, you’re committing the gambler’s fallacy: the tendency to think that if an event has recently happened more frequently than normal, it’s less likely to happen in the future. That’s irrational. Right?
Wrong. Given your evidence about koins, you should be more than 50% confident that the next toss will land heads; thinking otherwise would be a mistake.
I’ll spend most of this post defending this claim for koins, and then talk about how it generalizes to real-life random processes — like coins — at the end.
All you know about koins is that they tend to land heads about half the time. You can infer from this that on average — across all flips — the koin’s chance of landing heads on a given toss is around 50%. What are the ways that this could be true?
One (obvious) possibility is that the chance of heads is always 50%. Call this hypothesis:
Steady: On each toss, the koin has a 50% chance of landing heads.
Given your knowledge about koins, you should leave open that Steady is true.
Should you be sure it’s true? If so, then the gambler’s fallacy would indeed be a fallacy. But you shouldn’t be sure of it, for here are two other hypotheses that would also vindicate your evidence that koins tend to land heads around half the time:
Switchy: When the koin lands heads (tails), it’s less than 50% likely to land heads (tails) on the next toss.
Sticky: When the koin lands heads (tails), it’s more than 50% likely to land heads (tails) on the next toss.
The Switchy hypothesis says that the koin has a tendency to switch how it lands. For example, perhaps after landing heads (tails), it’s 40% likely to land heads (tails) on the next toss, and 60% likely to switch to tails (heads). Similarly, the Sticky hypothesis says the koin has a tendency to stick to how it lands. For example, perhaps after landing heads (tails) it’s 60% likely to stick with heads (tails) on the next toss, and 40% likely to land tails (heads).
We can represent hypotheses like Steady, Switchy, and Sticky with what are known as Markov chains: a series of states the koin might be in, along with its chance of transitioning from a given state at one time to other states at the next time. For instance, our example of a Switchy hypothesis can be represented like this:
This diagram indicates that whenever the koin is in state H (has just landed heads), it’s 40% likely to land heads on the next flip and 60% likely to land tails on the next flip. Vice versa for when it’s in state T (has just landed tails). We can similarly represent our Sticky and Steady hypotheses this way:
Given their symmetry, all of these hypotheses will make it so that the koin usually lands heads around half the time. (For aficionados: their stationary distributions are all 50–50.) Since that’s all the evidence you have about koins, you should be uncertain which is true.
It follows from this uncertainty that, given your evidence, you should commit the gambler’s fallacy: when it has just landed tails you should be more than 50% confident the the next toss will land heads; and vice versa when it has just landed heads.
Why? I’ll focus on explaining a simple case; the Appendix below gives a variety of generalizations.
Let’s suppose you can be sure that one of the three particular Sticky/Switchy/Steady hypotheses in Figures 1–3 are true, but you can’t be sure which. Suppose you know that the koin has just landed tails (as it has). Given this, you should be more than 50% confident that it’ll land heads — you should commit the gambler’s fallacy? There are two steps to the reasoning.
First, you know that if Switchy is true, it has a 60% chance to land heads; that if Steady is true, it has a 50% chance to land heads; and that if Sticky is true, it has a 40% chance to land heads. So if you were very confident in Switchy, you’d be around 60% confident in heads; if you were very confident in Steady, you’d be around 50% confident in heads; and if you were very confident in Sticky, you’d be around 40% confident in heads. More generally, it follows (from ) that your confidence in heads should be a weighted average of these three numbers, with weights determined by how confident you should be in each of Switchy, Steady, an Sticky. total probability and the Principal Principle
That is, where P(q) represents how confident you should be in q, your confidence that the next flip will be heads given that it has just landed tails should be:
P(H) = P(Switchy)*0.6 + P(Steady)* 0.5 + P(Sticky)*0.4
Notice: whenever P(Switchy) > P(Sticky), this will average out to something greater than 50%. That is, whenever you should be more confident that the koin is Switchy than that it’s Sticky, you should think a heads is more than 50% likely to follow from a tails, and (by parallel reasoning) that a tails is more than 50% likely to follow a heads.
Upshot: whenever you should be more confident the koin is Switchy than that it’s Sticky, you should commit the gambler’s fallacy!
And you should be more confident in Switchy than Sticky — this is step two of the reasoning.
Why? Since you start out with no evidence either way, you should initially be equally confident in Switch and Sticky. And although both of these hypotheses fit with the observation that the koin tends to land heads about half the time, the Switchy hypothesis makes it more likely that this is so — and therefore is more confirmed than the Sticky hypothesis when you learn that the koin tends to land heads around half the time. This is because Switchy makes it less likely that there will be long runs of heads (or tails) than Sticky does, and therefore makes it more likely the overall proportion of heads will stay close to 50%.
We can see this in action by working through a small example by hand, and through bigger examples on a computer.
Small example first. Suppose all you know about the koin is that I’ve tossed it twice and it landed heads once. Why does Switchy make this outcome more likely that Sticky?
To land heads on one of two tosses is simply to either land HT or TH, i.e. to land one way initially and then switch. Switchy implies that such a switch is 60% likely, whereas Sticky implies that it is only 40% likely. (Meanwhile, Steady implies that it is 50% likely.) Therefore Switchy makes the “one head in two tosses” outcome more likely than Sticky does.
It follows, for example, that if you were initially equally confident in each of Switchy, Steady, and Sticky, then after learning that it landed heads once out of two tosses, you should become 40% confident in Switchy, 33% confident in Steady, and 27% confident in Sticky. Plugging these numbers into our above average shows that you should then be a bit over 51% confident that it’ll switch again on the next toss — i.e. should commit the gambler’s fallacy.
The reasoning in this small example generalizes. The closer the koin comes to landing heads 50% of the time, the more ways there are to do this that involve switching between heads and tails many times; meanwhile, the closer the koin comes to landing heads 0% or 100% of the time, the fewer switches there could have been. Switchy makes the former sorts of outcomes more likely; Sticky makes the latter sorts of outcomes more likely. So when you learn that the koin tend to land heads roughly 50% of the time, this is more evidence for Switchy than Sticky — and as a result, you should commit the gambler’s fallacy.
So far as I know, there’s no tractable formula for determining these likelihoods by hand. But since the systems are Markovian, we can use “dynamic programming” to recursively calculate the likelihoods on a computer.
For example, if we toss the koin 100 times we can plot how likely each our the three hypotheses would make various proportions of heads:
Note that although all three hypotheses generate bell-shaped curves centered around 50% heads, the Switchy hypothesis generates a tighter bell curve around 50% heads. That means that it makes “the koin landed heads roughly around half the time” more likely than the other two hypotheses do.
For example, suppose you started out ⅓ confident in each of our hypotheses. Then if we let “the koin landed heads roughly half the time in 100 tosses” mean “50% ± x” for various values of x, here’s how confident you should wind up in each of the hypotheses after learning that it landed heads “roughly half the time”, along with the resulting confidence you should have that it’ll switch from tails to heads on the next flip:
The Fallacy in Real Life
That’s what we should say about the gambler’s fallacy with koins: it’s rational. What should we say about the gambler’s fallacy in real life?
I think we should say the same thing. Most people don’t — shouldn’t — be sure of how the outcomes from (most of) the random processes they encounter are generated. Many of these outcomes plausibly are either Switchy or Sticky — for example, whether it rains on a given day, or whether a post on Twitter will get signifiant uptake, or whether the next card drawn from this a is a face card. Many others are at least open to doubt.
So people should often leave open that various versions of the Sticky and Switchy hypotheses are true. And since they don’t (can’t) keep track of the full sequence of outcomes they’ve seen, what they know about the processes is often much more coarse-grained — e.g. that a given outcome tends to happen around 50% of the time. (See the Appendix for generalizations to other percentages.)
As we’ve just seen, if that’s what they know then they are rational to commit the gambler’s fallacy. Instead of revealing a basic misunderstanding about statistics, such a tendency may reveal a subtly tuned sensitivity to statistical uncertainty.
Of course, this doesn’t show that the way real people commit the fallacy is rational: they might commit it for the wrong reasons, or in too extreme a way. (See Brian Hedden’s post on hindsight bias for a discussion for how we might probe those questions — and why it is difficult to do so.) But the mere fact that people commit the gambler’s fallacy does not, on it’s own, provide evidence that they are handling uncertainty irrationally — after all, it’s exactly what we’d expect if they were being rational.
Objection: What about coins? Obviously coins have no “memory”, so when it comes to coins, people should be certain that hypotheses like Switchy and Sticky are false, and instead be certain that Steady is true.
Reply: Should they? Should you? Real coins are much more surprising than statistics textbooks would lead you to think. For example, despite the ubiquity of the notorious “coin of unknown bias”, it’s actually impossible to bias a coin toward one of its sides. Perhaps more surprisingly — and more to the point — it turns out that the way real people tend to flip coins leads them to have around a 51% chance to land the side that was originally facing up. So depending on the procedure you use for flipping your coin repeatedly (do you turn it over, or not, when you go to flip it again?), Steady may actually be false and some version of Switch or Sticky true!
Given subtleties like that, it’s rather implausible to insist that someone who has never taken a statistics course nor studied coins in any detail should be certain that hypotheses like Sticky and Switchy are false about real coins, or other more complex gambling mechanisms. As we’ve seen, so long as they shouldn’t be certain of that, they should commit the gambler’s fallacy.
Conclusion
Given people’s limited knowledge about the outcomes the random processes they encounter and the statistical mechanisms that give rise to them, they often should commit the gambler’s fallacy. So the mere fact that they exhibit this tendency should not be taken to show that they handle statistical uncertainty in an irrational way — if anything, it’s evidence that they’re handling it as they should! At the least, we need more detailed information about the way and degree to which people commit the gambler’s fallacy for it to provide evidence of irrationality.
What next?
If you have comments, questions, or criticisms, please comment or email me! As I said, this is work in progress.
If you want to see more details, check out the Appendix in the full post here.
For more recent work on the gambler’s and “hot hands” fallacy, see this fascinating recent paper.
Originally published at https://www.kevindorst.com.
