“How unusual is it to see the estimate we got?”
We know we will reject the null hypothesis when mean(X) falls outside of type I error rate alpha. Assume alpha = 5%. If the statistic score(z, t, binomial … etc) of mean(X) is larger than the statistic score of 95%, then we will reject the null hypothesis.
P-value is the probability of extreme cases of null hypothesis (mu > mu0, mu < mu0 … etc). If P-value < alpha, then the mean of extreme cases falls outside of type I error rate area. So we reject the null hypothesis.
The trick is to consider our statistic or estimate is an extreme cases or the boundary. And if it is extreme ( smaller than alpha ), then we reject null hypothesis.
When p-value < alpha, reject !