Here’s why scientific studies seem to change every week
Why science is so wishy-washy
It happens all the time in journalism. In between stories about car thefts and corrupt politicians, the nightly news pays a tribute to science with a premature, shocking headline:
New study says salt is good for you after all!
New study links salt with depression!
New study says dogs can understand when you talk to them!
New study says dogs just aren’t that smart!
Why can’t science just come to a conclusion and stick to it?
Although these titles can seem misleading, they are a good opportunity to look deeper into the scientific process. The reason these titles seem to vacillate wildly on basic topics, is just one scientific study doesn’t constitute ‘scientific truth’.
Here’s what I mean.
When journalists give information about a given topic, they’re typically focused on the conclusions/discussions section of the published paper in question. If it’s been a while since you’ve last read a scientific paper, here’s a brief overview of their anatomy.
- First, comes the abstract. This typically contains a sentence that describes background, an overview of the methods, the goals, and a truncated version of the conclusion.
- Next, there’s the introduction. This section is intended to provide more background information on a given topic and let people know where the research is coming from (and demonstrate that their study is feasible).
- Then comes the methods section, a detailed description of what the researcher did to gather data, often including ‘hiccups’ that came along the way. (Science can be so, so messy.)
- Then there’s the discussion, where researchers talk about what their data could imply or mean for other studies.
- And finally (sometimes this is used interchangeably with discussion) there’s the conclusion. What are the major takeaways of the study.
This typical structure reveals some of the problems with only revealing the conclusions.
Say, you’ve heard that salt consumption is linked with depression and want to know how significant that is. First, you should have some background knowledge. That’s how you can confirm whether or not this particular study has any backbone.
Perhaps more importantly, you’ll need to look at the methods. If the study relied on self reported surveys, for example, you have to consider the fact that the information gathered will be based on a person’s opinion about themselves (how depressed they feel) and their skills at accurately reporting how much salt they consume.
Science is very much a living thing and, as such, it’s constantly evolving, renewing its cells, and using resources to grow in order to get closer to the truth. To get to the bottom of things, after one study is done, other scientists have to see if the results (or the experiment itself) are reproducible. Then, they might check to see if a different experiment produces the same results, or if the results accurately describe nature (humanity and its iPhones included). Even more revision and work is required if that study is ever going to lead to a theory.
EDIT (7/6): The previous sentence seems to imply that all studies/scientific discoveries move hierarchically from hypothesis → conclusion → theory, however, this is not the case. Hypotheses, conclusions, and theories are all valuable to science, but their differences are important to understand.
Studies like these are also a bit limited. Some scientific studies can only tell you about correlations, which quite often can be meaningless coincidences. Don’t believe me? Check out Spurious Correlations. Turns out, the length of the winning word in the Scripps Spelling Bee correlates with the numbers of deaths from venomous spiders.
Humans are incredible at finding patterns. We make them up. We see faces in smudges of ink, we can piece together elaborate stories using shrapnel from a dozen sources, and as Tyler Vigen illustrates, we can make data plots look like they’re linked to each other.
Cydonia, a region of Mars, is a perfect example of just how good human brains are at recognizing human faces. The picture on the left is of a rock (mountain?) that bears shocking resemblance to a human face… If, of course, you’re looking for a human face (surprisingly enough, some people don’t see a face on first glance).
The photo on the left was taken by the Viking 1 orbiter in 1976, and while the chief scientist on Viking 1 dismissed the face as a “trick of light and shadow”, some people absolutely flipped.
They viewed the face, perhaps arrogantly, as a sign of civilization. A civilization that was probably produced by a universe with no creativity, considering its remarkably human look. Either the Martians themselves were humanoid, or they just really like human beings.
Some began to look at other rocky structures in the pictures and label them as pyramids, camps or bases where Martians roamed. Others thought that the explanation of the humanoid face was that Cydonia was once home to an ancient human civilization, which I must say does make more sense. Humans love to commemorate themselves.
Whatever story humans came up with, they all led to the same conclusion that transformed the Martian region into: the Crumbling City of Cydonia, Home to Ancient Space Travelers and/or Human Loving Natives!
In fact, a blog posted on “ancient-code.com” claims that the pictures show that there’s more to Mars than just a “lifeless planet”, and that the images “left experts confused” about their artificial features.
Which brings us to the picture on the right. It’s the same rocky mass, minus the obvious face, taken in 1998, after over 20 years of technological progress and exploration. Suddenly, the Cydonia lost much of its enigma and was stripped of its title as most popular Martian tourist site. Personally, I think acknowledging the truth was worth losing a potential science fiction story.
I realize that “the media” is partially (mostly? slightly?) to blame for the breakdown in science communication. But, considering I know so little about the world of television and ‘all purpose journalism’, who am I to judge? It is also quite possible that journalists, perhaps even science journalists — depending on their background, don’t really realize how misleading the some of those Shocking New Study reports can be.
In any case, I’m a big believer in adapting to your environment. Perhaps instead of burning down the CNN or FOX or the New York Times, we should try to understand a little bit more about what goes on behind the scenes. While I’m pretty sure getting a behind the scenes look is the whole point of journalism — and striving to produce more reliable news stories is a noble and necessary goal — it will take time (and a bunch of smart people) to reform the entire media ecosystem.
By triple checking, finding original (primary) data, and being a bit more vigilant, we don’t have to be deprived of good information.
You’ve reached the end. If you liked this science story, you might like this one, too.
Hi! I’m flagellate. When I’m not swimming in agar, or writing about my favorite microbes, you can find me making up stories about the end of the world, and helping science get funded.
