Alternative “Facts” vs. Alternative “Inferences”

Suppose you have seen the following sequence in three coin tosses:

HHH

Which of the following statements is a “fact”?

  1. The coin is fixed and will always land “head.”
  2. The coin is fair.

The short answer is that neither of these statements constitutes a “fact.” They are both inferences, conditional on two things: the observer’s prior about the universe and the actual indisputable fact — that you have gotten three heads in a row. We may or may not believe that the coin is actually fair, more or less, and you just had a series of chance occurrence and that is not improbable (1/8). Since we don’t think, a priori, a coin that always lands “head” is very probable, if at all possible, we (rightly, I think) think that that is very likely. BUT there is nothing in the actual “fact” that is uncontested that makes the “fringe” belief that the coin is fixed improbable. If anything, it actually reinforces that beliefs.

We might believe that the coin is close to being fair, if not fair, for a variety of reasons. We might believe that the existence of a magic coin is not very likely, if at all actually possible. We might have seen many coin tosses before, even if not necessarily from this coin, and believe that (not improbably) that this is coin is just like any other (even if we have no explicit or informed reason to believe one way or another). Or, we might be lazy and think that 50–50 chance is reasonable. None of these actually says that the specific coin in question is not fixed. The “conspiratorialist” who believes that this particular coin is fixed has no reason to believe that his belief is wrong in terms of actual undisputed “facts.” The only “fact” that would convince him otherwise is the coin turning up “tail” in some toss. In other words, to prove the conspiratorialist otherwise, you need to invest in an actual experiment, where you are willing to pay a cost to show that, in some N tosses, where N>3, there is at least one “tail.” (or, if the conspiratorialist has a more nuanced belief, that true P(H) = .9 or something, enough of a result to tickle his p-value.)

The bottom line is that “alternative inferences” are easy, apparently quite common, rather natural, and in fact often undergirded by real facts that are beyond dispute. We often think others are wrong because they are drawing different inferences from ours from the same set of facts, but that only means that they are coming to the observations with different sets of priors through which the facts are evaluated.

Asymptotically, all beliefs converge if we are exposed to the same stream of data, drawn from the same distribution — that is, as the number of observations approaches infinity. But do we really draw from the same distribution any more? The supply of data, especially in face of there being too much data on the whole, has been shifting to meet our filters. I don’t just mean the news about politics: we draw disproportionately from the data that we have easy access to and are interested in for whatever reason— on the few occasions I had dealt with marketing related data, I was amused that the demographics of the samples I had were heavily skewed towards younger, richer, and tech-savvy. I suppose that’s fine if you want to know what tickles the p-value of this presumably economically valuable crowd. But if you want to know how your product might sell outside this group, this would be a poor sample. So we sample the subsets of the vast data to explore further because they fit our priors (and cost considerations), and so continues the cycle. And, to be honest, it is silly for consumers of data to actively spend more to acquire data that they don’t know what to make of, when there is plenty of data that are cheaply available that they can. (In political psychology, we call this Drunkard’s Search, and I’m actually far more sympathetic to the drunkard of the story than not.) But the bottom line is that the convergence of beliefs, given the bias in the sampling process, will be either very slow or nonexistent. This is the real problem. (And this is before we even bring up the bit about sample size approaching infinity.) Observe there is nothing “alternative” about different observers having different samples of data and the samples that they have being compatible, to different degrees, with different beliefs, which, in turn, lead to varying inferences . A wise user of data analytics should be cognizant of this as the natural byproduct of (almost inherent) biases in both the sampling process AND different priors that observers bring to the table. Even while recognizing the variability, however, acknowledging the “facts” and their limits should not be too much of a problem: that HHH has taken place is beyond dispute, and the probabilities with which this would have been produced by the priors that P(H) = 1/2 and P(H) = 1 can be calculated. The dispute between different observers lies with regards the probability P(P(H) = 1), and the “facts” currently available contribute rather little to resolving this — only collection of additional facts can. If the facts are unavailable, there is, I think, little point in going beyond acknowledging that the differences exist and finding an agreement as to what additional data, if they can be found, would make the resolution possible in one direction or the other. (This is where experiments, aka A/B testing, come in: sometimes, the vast majority of however big data that you may have has nothing to do with the actual dispute, and the subset that is actually pertinent might be too small to yield sufficient statistical power. The only solution is to artificially collect more data of this underrepresented subset.)

I don’t think “alternative facts” really exist: there are very few undisputed facts that exist independent of interpretation in the first place. Most “facts” we think of are really inferences, usually very reliable and robust, but still inferences extrapolated from limited facts. None of us have actually been to the moon, save a handful who participated in the Apollo project — if it actually happened (Ha! Just kidding.) We “know” the moon landing took place because there are many circumstantial clues consistent with it being real and it strikes us as absurd that something like that could have been faked. But these exist in the realm of “inferences,” not facts, and they reflect far more what and who we are rather than what we “know.” What really troubles us is that there are people, perhaps even many people, who do not draw the same inferences.

I am not suggesting that the problem is trivial. Far from it, that the problem is “alternative inference” rather than “alternative facts” indicates a problem much deeper. If it were merely a matter of “facts,” it is easy to demonstrate contrary facts that are indisputable and that will be end of it. The belief that the coin is fixed, in the above example, will be broken by a single outcome where the coin lands “Tail.” If the coin is anything near fair, it will take place soon enough. Alternative inference, however, rests on a much more “sophisticated” foundation, both “theoretical” and “empirical,” in the sense that they comprise a complete theory of how the universe works and generates data, and enough data to sustain those “theoretical” foundations. A complete set of “alternative mental models” exist, in other words. If human sociality is built around common beliefs about the world around us, that, even without stating things explicitly, we know what constitutes moral, proper, and “normal” way of behavior and interaction and follow along accordingly, this absence of common thinking indicates that their is no common “human community” that holders of different inferences share. In other words, “alternative inferences” show not so much that they “know” different things, but that they ARE different.

Just lecturing “them” that their “facts” are wrong would be absurd — it would be equivalent to telling them that HHH means that P(H) = around .5. Yes, HHH and P(H) = .5 are far from incompatible — P(HHH|P(H) = .5) is in fact quite high. BUT HHH is far more compatible with P(H) = 1. If we were trying to convince someone who believes P(H) = 1 from coin tosses, we should just get more coin tosses. Yes, we got HHH in three tosses that we have. What would you expect if we had N= 10? Note that we are not necessarily imposing our belief on the other here — we might actually be proven wrong and we could get 10 H’s, however improbable it might be, and if so, the conspiratorialist may have a point, but if so, it will be based on a firmer empirical footing, one way or another. But, most likely, it won’t be, and there is no alternative fact to the ten coin tosses. Of course, that means that we actually have to pay to run the experiment to gather the data we need, not try to squeeze inferences out of the limited data that we have (even big data is limited in face of biased sampling, and all data is biased in some fashion — the only question is, are they so biased that we should be skeptical of them.)