We’re Taking The Science Out Of Science

Every day we’re bombarded on social media with stories and articles about how science proves _____. It sure sounds like science has been busy. But should we believe all these claims?

I hate to be the bearer of bad news, but more likely than not, science did not prove whatever it is the writer of the article would have you believe.

In fact, the very notion of science proving something is flawed.

What is science?

Science is not an outcome or result. It is a processes. It is a way of thinking and a way of learning about the world. A single study or experiment can certainly point us in a direction to for further study, but that doesn’t mean we should go around making claims based off it. There are so many unknowns that go into any given experiment that it’s difficult to be sure of our conclusion based on a single study.

A poker analogy

Consider a game of poker. You’re faced with a decision of someone going all-in and you’re deciding whether to call or fold. You have a fairly strong hand (let’s say 2 pairs), but there’s a number of possible hands that could beat you. You decide to call, “For science!” You had the winning hand take the pot. Based on that experiment you conclude that any time you have a fairly strong hand and someone goes all-in, you should call.

That doesn’t sound like a very reasonable conclusion does it? That’s the kind of logic used by most articles claiming that science proves something. In poker there are so many factors that can affect the outcome of such a decision, like the opponent’s play history (how often they bluff, how aggressively they play), how many people are left that can still be in the hand after you, how much money the all-in represents, etc. But none of that matters to you, because you have empirical evidence that you made that right decision.

This is pretty obvious case a premature conclusion, and it’s easy to see what’s wrong with that conclusion. But let’s dig a little deeper with our analogy.

Increase the sample size

Let’s say you play for a while and repeat this experiment 50 times. Out of those 50 times that scenario came up, you did the same action of calling, you won 35 times. That’s certainly better than chance, and its statistically significant, so you conclude that it is a good strategy.

Sample bias

That sounds pretty good, you now have multiple instances upon which to based your conclusion.

But wait.

All your experiments were at the same table against the same opponents. Maybe it only works at low-stake tables where all the players just play for fun and aren’t very good. Does that same strategy work on more experienced players?

Maybe you try to take that into account and try your strategy at different tables with different stakes to control for that variable.

But you played all your games during the weekend where there are more casual players, compared to weekdays when there are more serious players.

This is where a single experiment, even with a reasonable sample-size can often be insufficient. Depending on how the sample was created, there are often biases introduced by the sample, whether intentionally or unintentionally.

Where does that leave us?

Just because we can’t be sure of the general applicability of our findings, that doesn’t mean that the information we got was useless. We just need to be careful that we don’t overstate our claims, and that we do additional research to back-up or challenge any conclusions.

One of the corner pieces of science is that it is falsifiable, that is to say it is possible to disprove any claim made by science with new and better evidence. After all, how can we know what is right if no one is allowed to disagree?

When people cavalierly throw around statements like “SCIENCE PROVES THAT EMOJI USE WILL GET YOU LAID” both the meaning of the word “science” and that hard work that goes into scientific research is cheapened.

Help put the science back in science by taking refusing to propagate click-bait articles that misappropriates the word for their own ends.

Originally published at hopeandcynicism.com on January 24, 2016.

Show your support

Clapping shows how much you appreciated David Ly’s story.