How To Really Understand Statistical Significance?

We intuitively misunderstand “statistical significance”!

Illustration created by the author

Statistical significance is one of those “scientific” concepts/tools that has steadily been misused over the years (especially in the social sciences). In this essay, I aim to clarify certain fundamental misunderstandings related to this concept and its application.

We will start with an introduction to our intuitive understanding/application of statistical significance. Then, we will dive one level deeper and cover the scientific business of statistical significance. Finally, we will discuss the core issues with our current usage of statistical significance and explore potential remedies as well. Without any further ado, let us begin.

Intuitive Statistical Significance

Human beings use intuitive statistical models all the time. We observe real-world events, build hypotheses over time, and then establish them as truths using our intuitive statistical models. Of course, we don’t express words like “statistical” and “significance” for such purposes. Here is a hypothetical example:

You are driving downtown and stop in front of a signal near the town square. You let out a big sigh of frustration. You do this because you know that if you get a red…