The Null Hypothesis Loves You
and Wants You
To Be Happy

Our last best hope for understanding.


Of all the machinery with which we perform science, the least appreciated mechanism has to be the null hypothesis.

The null hypothesis, Wikipedia staidly informs us, is “a general statement or default position that there is no relationship between two measured phenomena.” Disprove it, and you have a reason to believe that the phenomena you’ve measured are connected in some way; find evidence for it, and you can show that an alternative hypothesis is flawed.

Galileo measures the mass of two cannonballs, one heavy, one light, then (apocryphally) drops them off the Leaning Tower of Pisa. That they land on the ground at the same time actually tells us very little about the force of gravity. At most, it tells us that the alternative hypothesis, the one about mass dictating falling speed, is wrong, because the dissimilar weights took the same amount of time to fall. The Aristotelian hypothesis, that heavier objects fell faster, was flawed; Galileo’s experimental results supported the null hypothesis.

It would take nearly a century for Newton’s observations of planets and moons to disprove a fundamentally similar null hypothesis — the one about there being no relationship between the masses of two objects and the orbit into which they enter, of which falling is a Very Special Case — and provide us with the inverse square law. It would take two centuries further for closer measurements of Mercury’s orbit to reveal discrepancies that Newton’s model couldn’t explain, leading Albert Einstein to propose a relationship between mass and a phenomenon we can observe only through its side effects: the very curvature of space-time.

Mathematics is where we prove things. In science, we only disprove more and more rigorous null hypotheses. It’s fair to think of any scientific theory as a statement that “although we tried everything we could think of, we can’t get physical reality to stop doing this.

Consider, then, the unconditional devotion of the null hypothesis. In a universe whose components constantly interact, at scales from Planck length to parsec, it stands apart, denying any form of predictable interaction between any of them. And then it invites you to disprove it. We raise the null hypothesis only to cut it down, over and over again, in the name of reason and comprehension and progress, and the null hypothesis never complains. The opponents of other null hypotheses certainly do, when they see their colleagues claim victory over a null hypothesis which is not yet slain, but every time a null hypothesis dies, its wake is a ghost town; the guests are all at the wedding of the alternative hypothesis and the scientist who formulated it.

What greater love hath any idea than this?


The null hypothesis is science’s first and last line of defense against one of the most terrifying properties of the human mind: the capacity to find patterns. Our pattern-matching ability is innate. We are the species whose niche is anything we can adapt to ourselves, and that facility for adaptation is built in part on our capacity to recognize patterns such as “when food sits in fire, its texture changes and it becomes easier to eat, until it burns.”

You don’t notice most of the patterns you recognize. Your neocortex does that for you, collecting descriptions of edges from the bursts of electricity that your optic nerves send to it as photons land on your retinas. From those edges it reconstructs shapes, and from those shapes it calls up associations with patterns you have established over time: that is a chair, this is my desk, down there is the carpet.

https://twitter.com/jack_daniel/status/516615752724393984

Same goes for your other senses. By the time a piece of sense data registers to you as information — that is to say, data with some context around it — it’s already been through several different layers of processing, matching the processed results to previously observed patterns, and being fed to other processors depending on what matched and what didn’t. Then you get to do the same thing with the information. It’s this latter process we call thinking, but the most significant difference is the degree of control we perceive ourselves having over it.

As volatile as the early years of the twenty-first century have been, we live in an age of nearly unprecedented regularity. Before the advent of the steamship in the mid-19th century, there was no regularly scheduled travel across the Atlantic Ocean. Thanks to the unpredictability of wind, a sailing ship might take weeks to cross, or never arrive at all. Today, going from a metal box that travels on rails to a metal tube that takes you through the air from Brussels to Newark is a long morning’s errand, since you’re traveling west. A long campaign against entire regiments of null hypotheses went into making that possible, on the fields of materials science, physics, chemistry, and biology. Before we could send people up in airplanes reliably, we had to filter through countless imaginary relationships in order to find relationships that did exist between physical properties we could control.

Apophenia is the technical term for those imaginary relationships. It’s what happens when you look up at the full moon and see a rabbit or a person’s face, or when you believe there’s a secret message beyond “the ratio between a circle’s circumference and diameter is constant” encoded in the digits of π. Conspiracy theories thrive on apophenia. It was first named in order to describe the onset of schizophrenia: apophany in contrast to epiphany. It is the unavoidable consequence of being a member of a species whose super-strength is its self-adapting, pattern-matching brain. Apophenia is our super-weakness. The null hypothesis shields us against it with its own body.

You probably know apophenia better as its cute and cuddly subcategory, pareidolia, or as the Internet knows it, Things with Faces. “That’s a face” is one of the very first patterns you pick up on as an infant, along with the sounds that will later go on to form the phonetics of your native language. Language is another sort of pattern — a generative pattern — that is so important to the human experience that we experience pareidolia in sound as well. Back when vinyl records and cassette tapes were still a thing, the United States had some rousing moral panics over backmasking — not the simple yet sometimes surprising recording technique of using one recording played backward as a track in another recording, but the belief that rock musicians were deliberately encoding secret messages in their records and tapes, fueled by coincidences such as an urban legend that Paul McCartney was dead around the same time that some guy up late played a Beatles record backward and heard the words “turn me on, dead man” creaked out.

Though I can’t be too hard on the panicking Christian Right of the 1980s; auditory pareidolia can be some legitimately scary stuff. The auditory pareidolia I experience in the Manhattan subways — a faint, creepy, faraway music — had me questioning my sanity until I decided to observe it happening. Having a partner I can trust not to think I’m crazy if I ask “are you hearing music?” helped a lot here. The fact that we understand the basics of how sound waves behave also helped, and before too many subway trips had gone by, we correlated the frequency of hearing music only to subways, and furthermore only to when there was additional perceptible evidence of trains going by in nearby tunnels. Null hypothesis supported: probably not that crazy, yet.

Galileo’s contemporary, Benedetto Varchi, wrote in 1544 that

the custom of modern philosophers is to believe always, and never to test all that which is found written by the good authors … but that does not mean that it would not be more certain, and more delightful to do otherwise, and to sometimes descend to experience in some things.

Here’s the thing about pareidolia: it interacts really strangely with altered states of consciousness. Well, that’s fine, you say, it’s not like I walk around in an acid haze, to which my response is, how much sleep did you get last night? Sleep deprivation is an altered state of consciousness, as is being hungry, as is being satiated, as is being buzzed, as is multitasking. We make decisions differently when we’re under time pressure than when we have time to consider them. If you’re going to measure phenomena, ultimately, with your senses, then those measurements need to be the same under different sensory conditions. Like Philip K. Dick said, “Reality is that which, when you stop believing in it, doesn’t go away.”

The talk he said that in is called “How to Build a Universe that Doesn’t Fall Apart Two Days Later,” and it’s a long and introspective discussion of subjective experience, at times painful in the significance that it attributes to thoughts born out of nothing more than coincidence and imagination. Apophenia on display, as it were. Dick’s writing can be inscrutable, but that’s often precisely because he thinks he sees something with a pattern in it, reproduces the image as faithfully as he can, and the pattern just isn’t there. No, Phil, I want to explain to him, those were isolated incidents that appeared significant because what you wrote about reminded someone else you trusted of a thing that was significant to him, and you continued to attach significance to things you noticed that seemed related to it. Though you have to admit, the narrative is compelling.

Relaxing the null hypothesis makes for great storytelling, but it’s an unsettling way to live. Which information and which perspectives we take into account, when we try to decide whether a pattern we’ve matched is real or an apophenic false alarm, affects our ability to determine whether something has gone away or whether we’ve just stopped believing in it. In the praxis of science we try to keep the false alarm rate down with things like study size and statistical inference and meta-analyses and replication, and we still get it wrong a lot.

Yet through it all, the null hypothesis is always there for us to take up when we need it most — when we have to decide which of our perceptions and beliefs to trust. The vision of echo chambers that Dick describes was, as it turned out, a prescient one, making the null hypothesis an even stauncher ally in a world where there are entire industries around convincing people about what will make them happy.

Every day that we slaughter a null hypothesis, we obtain another handle with which to manipulate reality. When two phenomena are related, changing one, e.g., mass or wingspan, yields observable changes in the other, e.g. drag. We could not have had airplanes without fluid dynamics, but we could not have had fluid dynamics without Newton or Galileo, even if their findings merely disconfirmed earlier beliefs or turned out to be not quite comparing the right phenomena.

I challenge you, therefore, dear reader: think carefully about the null hypotheses with which your own hedonic treadmill is paved. They seek only to save you from apophenia and deception. Disprove them and find glory, but ignore them at your peril.