“Don’t Worry, We Speak Good Science.”
We’ve all seen an X-Files re-run at some point. Some of us grew up on it, like a weird sort of mother’s milk steeped in the pervasive idea that all is not what it seems and that institutions are — almost by definition — not to be trusted.
“Trust no one” was the catch-phrase for the series and the personal motto of FBI Special Agent Fox Mulder (played by David Duchovny), the slightly more conspiracy-prone of the show’s two lead characters.
“Trust no one” was catchy, and it made for good paranoic entertainment — but it left something out.
A word.
Else.
Because really, what Agent Mulder was saying in his haunted, emphatic murmurs was: “Trust no one else.”
It was assumed that he could trust himself — and presumably also his partner-in-paranoia, Agent Scully. Such implicit trust lies at the heart of good partnerships.
But this is not good science.
Popular Science, Minus the Science.
You’ve probably seen some of those translation tragicomedy emails where sentences have been translated from English to French to Russian and then back to English again — often with hilarious consequences that make international violence seem surprisingly rare, given the capacity for misunderstanding.
(I like the possibly-apocryphal warning on a Buddhist temple in Bangkok, which probably should have said “Men Only.” Instead: “It is forbidden to enter a woman even a foreigner if dressed as a man.”)
Science is often the same way. The common language of the academician or the professional researcher is not the way regular people talk. The formal grammar and spelling might be the same, but often the similarities end there. Scientific papers come rich in a jargon, format, and style ready-made to intimidate non-specialists and provide a virtual force-field against the general public.
A scientific paper submitted to a scholarly journal written at an eighth-grade reading level — the target at which most mass-market American newspapers aim — would be rejected faster than you can say Nullius in verba.*
Luckily, we’ve got science journalists.
I view science journalists —a group of which I would count myself one — as being something akin to cultural matchmakers. We’re here to convince the dubious, mercurial public that the dour-seeming, standoffish scientist who talks funny (and who is clearly no fun at parties) is actually perfect for them… If they’d just give him a chance on a couple of dates.
As in romantic matchmaking, this requires putting the benefits that each party offers into terms the other party can understand. There is an element of translation.
And while ideas may not always be lost in translation…they are almost always given a long leash.
This slightly tweaked, mass-palatable version of scientific findings is commonly known as “Popular Science.” There’s even a magazine that goes by this same name, founded in 1872 and still active today.
However, one of the nice things about the peer-reviewed journals that constitute the official body of established scientific knowledge is that (this is the theory, anyway…) things that wind up published are first credibility-checked by colleagues committed to the principles of scientific rigor and skepticism, and at least passably familiar with the current consensus views within that particular domain.
“Popular Science” journalism, on the other hand…
Hell, we’ll take anybody with half-decent spelling who’s old enough to legally buy silly-putty.
And unfortunately, many of the “science stories” that make it to the general public are severely skewed by the journalist community’s somewhat laxer credentials.
The First Principle.
Scientists are people too.
In fact, too is probably not putting a fine enough point on it.
Scientists are people first, and scientists second.
Professed allegiances aside, all of us start out as standard-issue Homo Sapiens infants, and those that go on to scientific research endeavors require both training and vigilance to stand guard against the natural inclinations that make us unscientific.
Many of you will be familiar with the work of scientist and author Daniel Ariely. Ariely is best known for his book Predictably Irrational, which details the (frustratingly many) ways in which our brains are built to use “cognitive shortcuts” that guide us to behave in useful ways, though not ways actually grounded in reality. (“Useful,” in this sense, being the useful 50,000 years ago on the Serengeti version of the term — which is not always in sync with usefulness in modern day Hong Kong, Poughkeepsie or Barcelona.)
Richard Feynman was one of the leading physicists of the 20th Century, a contemporary of Einstein, involved with the Manhattan Project that created the first atomic bombs, and widely regarded as an outstanding genius in a line of work with enough geniuses in circulation to merit further gradations.
Feynman stated the “Trust No One” problem succinctly:
“The first principle is that you must not fool yourself. And you are the easiest person to fool.” — Richard P. Feynman
Being one himself, Feynman understood that scientists — like everyone else — each have their own leanings and inclinations. And also, that they enjoy being right.
But trying to prove yourself right is a tricky business. Even the most conscientious scientists are fans of their own biases. And if trial after trial after trial simply confirms a researcher’s initial suspicions, the tendency is to clap one’s hands together and declare victory.
Of course, this is why the Scientific Method enshrines the exact opposite methodology.
The Scientific Method is built around repeated attempts to disprove a hypothesis. Especially if it’s your own hypothesis.
Feynman had it right: “You are the easiest person to fool.”
The Intuitive Appeal of a Walk in the Park.
“Long visits to nature linked to improved mental health, study finds.”
This was the title of an article on my search-results screen, and I was looking for just such an article. I’d heard a lot of second-hand citations of studies tying cognitive health — increased creativity, speed of calculation, relaxation — to exposure to natural environments. I wanted to know more.
I only got three paragraphs into the article before I stopped, my skepticism-alarm buzzing.
The slippery slope of bad logic seemed surprisingly slippery.
I want to look now at the first three paragraphs in their entirety, to see if you smell the same logic-rat I did. Here they are:
People who visit parks for 30 minutes or more each week are much less likely to have high blood pressure or poor mental health than those who don’t, according to new research by Australian and UK environmental scientists.
A study led by The University of Queensland (UQ) and the ARC Centre of Excellence for Environmental Decisions (CEED) suggests people might need a minimum “dose of nature”.
UQ CEED researcher Dr Danielle Shanahan said parks offered health benefits including reduced risks of developing heart disease, stress, anxiety and depression.
Paragraph 1 gets it right. It talks about a pattern in the numbers between people who visit parks, and those people’s health.
Paragraph 2 slips just a little bit — not unforgivably, but we can sense where it’s going. The study “suggests that people might…” The article hasn’t actually gotten anything wrong yet, because both suggestions and might-ness are soft terms. It might…but then again, it might not. By definition, this statement can’t be wrong.
But in Paragraph 3, the other shoe drops: “…parks offered health benefits…” Here the equivocation goes away. There’s not a “maybe” in sight. What was merely an identified relationship in Paragraph 1 has suddenly morphed into a cause-and-effect statement.
And it took only 88 words to do it.
C+C Confusion Factory
The difference between Correlation and Causality is one of the most important distinctions to understand — not just in science, but in logic.
And not just in formal logic, but in “walking around the world” logic.
The classic example features ice cream cones and shark attacks.
Every year in the northern hemisphere, there is a rise in shark attacks that marches in statistical lockstep with the increasing sales of ice cream cones. Why? Because ice cream sales spike during summer. And this is also when people go to the beach on vacation, and thus become convenient targets for the local shark.
We all know that walking down the street with an ice cream cone doesn’t make you the likely victim of a shark attack, any more than an attacking shark is likely to offer you ice cream as a mea culpa.
The correlation does not imply any causality.
Any science journalist needs to understand this difference.
Now it would be fair to point out that what this article’s third paragraph actually says is “UC CEED researcher Dr. Danielle Shanahan said…” [emphasis mine]. And very possibly, the journalist was completely correct, merely quoting a scientist who was confusing causality with correlation.
Indeed, this could be the case. Because Paragraph 4 is a direct quote from the same doctor:
“If everyone visited their local parks for half an hour each week, there would be seven percent fewer cases of depression and nine percent fewer cases of high blood pressure,” she said.
This was the point where I could not control my skepticism, and I went looking for the actual study. (You can see it here.)
As I suspected, the study was purely correlational.
It was based on self-reported data collected from a reasonable sample size of Brisbane-resident Australians, who had taken a survey. The researchers normalized the resulting data, compensating for factors known to impact health — things like socioeconomic status, number of days worked per week, and the presence of children under 16 present in the home.
Still, I can think of a ton of potential invisible causal variables — things to fill the role of summertime in the ice-cream-and-sharks example — that could reasonably allow the factual statement in Paragraph 1 to be true while making the causal claims in Paragraphs 3 and 4 completely false.
Now don’t get me wrong, I’m not a part of some nefarious anti-parks lobby, and I love green spaces and nature as much as the next guy — possibly more than the next guy.
But that’s exactly it.
I suspect that the researchers love city parks and green spaces too, and that they were trying to prove their own hypothesis, not disprove it.
In their scientific paper — the sort of thing the public never reads — the authors were appropriately circumspect, and didn’t step beyond what they could prove:
“…our results show nature experiences, if causal in nature, could simultaneously lead to a suite of health benefits…”
Note the “if” and the “could” — as opposed to the conspicuously-absent “do.”
But in the “Popular Science” adaptation of their research, that nuance is lost. The public is presented with hoped-for possibilities as if they are established facts. And bear in mind, we’re talking about the small subset of the public that actually seeks out and reads “science journalism”. A few more word-of-mouth garblings, and who knows how a story like this winds up being understood?
I’ll be the first to admit: in the context of public parks and people walking in nature, a slip-up like this has a pretty limited downside. It’s hard to imagine a true disaster scenario from overspending public dollars on greenery.
But the same can’t be said of all scientific studies. Gross historical screw-ups from phrenology to forced sterilizations to pre-frontal lobotomies could all have been drastically reduced or even nipped in the bud if science journalists of the time had done a better job delivering the public the unvarnished facts.
Want to hear the real irony in all of this?
Providing the public with good information is only correlated with the public making smart decisions. It is not causal. (And sadly, history provides plenty of evidence for this.)
Now go take a nice, long walk in the park.
Not because it’s healthy for you. But because it might be.
* Nullius in verba is the slogan of the British Royal Society, a scientific group founded in 1660 and active to this day. Translated in English, the phrase is “Take nobody’s word for it.”
If you enjoyed this article, please click the little green heart below. That would be incredible.
