Wait but maybe…

We have all heard it said that correlation is not causation. Maybe you have also heard the even more dry-sounding statistics term “sampling bias” mentioned. Sampling bias is when a sample is not representative of the full population, skewing the results. “Ok”, you may be thinking, “so statisticians should be careful about that when designing studies and such”. But it’s more than that, sampling bias is all around us, all the time. I typically think of it in another way, as “wait but maybe…

Let’s take winter bathing. Winter bathing is healthy. We know this because people who do it live significantly longer than other people on average. Also it makes sense, your body feels all fresh and challenged.

Wait but maybe… the type of people who do winter bathing tend to be healthier than other people. For that reason they will live longer than people who don’t do winter bathing (on average). So actually we don’t know anything about how healthy it is, the health difference between the bathers and the non-bathers might be fully explained by the sampling bias.

Another example: Pre-school education is important for kids. We know this because kids who go to a good pre-school do better later in life. And it makes sense, they get started on learning early. Seems quite straightforward. Wait but maybe the parents who send their kids to a good pre-school are different from other parents in many ways. They are probably richer, they are more invested in the future of their kid, etc. So again, we don’t actually know anything about how important pre-school is. It is possible that the fact that kids who go to a good pre-school do better later in life on average, is explained fully by their home environment, and the pre-school has no effect at all.

And it goes on and on. Opening any news paper and reading it looking for opportunities to think wait but maybe, you will always find examples, especially if the articles are about food, science or society.

This is also why we keep getting contradicting information about nutrition, and there is basically no good knowledge about the topic. People who eat more salad live longer, so salad is probably healthy. Wait but maybe the people who choose to eat salad are more focused on their health in general, and live longer for that reason, not because of the salad. The same point is true for all dietary advice.

And a lot of other advice.

- “You should work out in this way using this workout program. Look at me, I worked out in this way, and I am super fit.” Wait but maybe you just have great genes, and a lot of programs would’ve made you super fit. That doesn’t mean it will work for me, or that it would be the best approach for me.

- “You shouldn’t mix beer and liquor, you will get a headache.” Wait but maybe the times where I have drunk both beer and liquor, are nights where I was drinking more overall. Maybe that’s the only reason I got the headache, and the mixing had no effect in itself.

- “I think it’s good to run. I always feel better and more energized in the periods of my life where I go jogging a lot”. Wait but maybe you feel good for other reasons in these periods, and that gives you the energy to go jogging. Maybe the jogging in itself does nothing.

Etc etc.

All of the initial claims on this post may be true. Maybe winter bathing and eating salad is healthy. The point is that we have no good idea whether they are or not, and it is important to realize this and consider it, both in politics and in your personal life.