Hypothesis Testing in layman terms
If you are into Machine Learning or Data Science there is a very high chance that you might have come across this term. And a higher chance that you might have not understood what does it exactly mean. But I assure you that by the end of this article you would have pretty good understanding around this term and also its implication in Machine Learning.
Let’s get started.
Hypothesis testing is one of the two methods of Inferential statistics (confidence interval is another). In inferential statistics, we take a sample of data from population and then calculate a statistic(it can be anything like mean, standard deviation etc). We then use the value of statistic to infer(estimate) the value of corresponding parameter.
Parameter is to population as statistic is to sample. When we calculate mean, standard deviation etc of a population its called parameter, when calculated for sample its called statistic.
So Hypothesis is a proposition made as a basis for reasoning, without any assumption of its truth. There’s a reason why I have highlighted certain words and you would know its reason by the end of this article. There are two types of hypothesis Null hypothesis and Alternate hypothesis.
Null Hypothesis
Let us try to understand this with the help of an example. We want to check whether or not there is a difference between average income of Indian employees in the year 2019 and 2020. So as the word null means zero or no, null hypothesis would be there is no difference or zero difference between the average income for the 2 years.
Null hypothesis can also be a proposition which has been made earlier and that proposition is accepted.
Mathematically:
Null hypothesis : average income for year 2019 = average income for year 2020
Alternate Hypothesis
The alternate hypothesis would say that there is a difference between the two values (which value is greater and which value is smaller is a different question but there is a difference). In our above example it would be that both the incomes differ.
Mathematically:
Alternate hypothesis : average income for year 2019 ≠average income for year 2020
Alternate hypothesis can also be a statement that differs from the proposition that is accepted by the people.
Real life example
It has been proposed by Trump that anti-malaria tablet would cure Covid-19. This proposition would become our null hypothesis. It has not been proved that it cures Corona, it is only proposed, there is no significant proof. Now a researcher comes up and says that No, the anti-malaria tablet doesn’t cure Covid-19. This would become our alternate hypothesis.
When we deal with hypothesis we never prove that a hypothesis is correct we just prove that other hypothesis is incorrect. Similar to our judicial system, a person is innocent until proven guilty. If we cannot prove a hypothesis wrong, it is accepted until it is proved incorrect.
The researched would have to come up with significant results to prove his proposition. If he fails to do so we would accept that anti-malaria tablet cures Covid-19.
Implication in Machine Learning
Let us see how is this concept used in our field. Suppose that we have to build a Linear regression model. We know that linear regression model would have some features and weights(parameters) assigned to those features. How do we come to know that this model is helpful? hypothesis testing comes to our rescue.
Linear regression’s equation is y= β0 + β1x1 + β2x2 + ….. + βnxn
The null hypothesis says that the model has no effect, meaning β0= β1 = β2 = …… βn =0. And this is what the null hypothesis will always claim, it will claim that our model is useless.
Alternate hypothesis would be that model with these parameters has some effect and is better than the model with above beta values. Meaning β0, β1, β2 …… βn ≠ 0
We will have to prove that this model is significant using some hypothesis test. If we fail to reject null hypothesis, then our model is useless.
Terminologies
- Reject Null hypothesis: This means that we have proved that null hypothesis is wrong and we go ahead with alternate hypothesis. But we never say we accept alternate hypothesis. We always talk in terms of Null hypothesis. This term implies double negation, reject is one negation and null is second negation.
- Fail to reject Null hypothesis: This means that null hypothesis is correct and alternate is wrong, it is clear from above that we won’t say we reject alternate hypothesis. The definition of hypothesis is, Hypothesis is a proposition made as a basis for reasoning, without any assumption of its truth. We don’t prove the truthfulness of a hypothesis we just prove that its incorrect. So we never accept null hypothesis as we don’t prove its truthfulness, we say that we Fail to Reject it as we don’t have any strong evidence against it. This term implies triple negation, fail is one negation, reject is second and null is third.
- P value: Since long we have been talking about significance, to reject null hypothesis we need to prove significance and to prove significance we perform some statistical test which gives us a p-value.P-value is nothing but the probability value. The p-value is the probability of obtaining results as extreme as the observed results of a statistical hypothesis test, assuming that the null hypothesis is correct. This p-value helps us to prove significance. How to calculate p-value is out of the scope of this article.
We set certain threshold and if the p-value is less than the threshold we reject the null hypothesis. Generally that threshold is set to 0.05 meaning that there is 5% chance of we making a wrong decision. It means that if we reject the null hypothesis 100 times for p-value less than 0.05 then only 5 times our decision might be wrong rest 95 times it would be correct.
Why do we reject null hypothesis when p-value is less than 0.05?
The hypothesis test that we perform assumes that null hypothesis is true. If the null hypothesis is true then we should get high probability. If we are getting very less probability, less than 5% then we are pretty sure that our assumption that null hypothesis is true is incorrect, hence we reject null hypo when p-value is less than 0.05. P-value is also the probability that what we have seen (in above example that coefficients are non zero) is due to random chance. If the probability is small then we are sure that the change is not due to random chance, so we reject null hypothesis.
Note: The threshold is generally 0.05 i.e. 5% and we can reduce it to 1% as well if we want to reduce the chances of making an incorrect decision. For example if we are performing a hypothesis test for a drug then we need to reduce the chances of making an incorrect decision. In such cases we might keep threshold as 0.01 or 1%.
Another takeaway from this article is:
Hypothesis testing at its core checks whether our statistic belongs to the null hypothesis distribution or some other distribution. If it does not belong to our null hypothesis distribution we say that our statistic comes from some other distribution and reject the null hypothesis.
If the above statement isn’t clear. Do check my other article which is written on the similar lines and would you give a clear understanding of the above statement.
I hope I have explained what is hypothesis testing in a nut shell. Feel free to drop comments or questions below, you can find me on Linkedin.
