Introduction to Hypothesis Testing

You are a manager of a food delivery company, Foodzen. Everything was going well. But then COVID happened. Sigh.

Closed restaurants and the desire to capture the rising demand of grocery delivery compelled you to try your luck in the grocery segment. You decided to start with Gurugram. After going through the reports you figured out the mean number of grocery orders booked in a week is 40k. However, intuitively you thought it to be higher than this value. So you went for an online survey in the city to test your hypothesis.

So what is this hypothesis testing?

Hypothesis testing is a way to check whether the apparent behaviour is statistically significant or not. An effect is statistically significant if it’s occurrence isn’t merely a chance. Here we have to check whether we have the number of average orders as 40k or it’s greater than that.

Stating the hypothesis:

This brings us to the Null and Alternative hypothesis.

Null hypothesis states that there is no significant difference between the original value and the one that comes from intuition.

Alternative hypothesis states that null hypothesis is not true and behaviour shows some divergence from the actual behavior.

Null hypothesis Ho : P = 40,000

Alternative hypothesis Ha : P ≠ 40,000

Analysis Formulation:

The variable P you have just seen is nothing but the mean of the data.

P-value : The probability of getting the sample statistics(mean, standard deviation, etc.) given the null hypothesis is true and is known as p-value. You calculate it based on the values from the samples(smaller subsets of the area/population).

Here we’ll take 100 sub-regions of Gurugram based on the area and calculate the mean value of the number of orders. Suppose that comes out to be 45k.

Result Interpretation:

Our next step would be to calculate the p-value which is the probability of getting the statistic mean at least 5k away from the actual value, when the null hypothesis is assumed to be true.

If the p-value comes out to be less than the significance level dented by alpha(α), we reject the null hypothesis. Otherwise we can’t reject the null hypothesis. Alpha is generally taken as 0.05.

P-value is the difference between the calculated statistic(mean in this case) divided by the standard error. So the p-value in this case comes out to be higher than our significance level, α = 0.05, so we can’t reject our null hypothesis.

Alternative approach : Using critical values

A critical value is the point on the scale of the test statistic beyond which we reject the null hypothesis. It is derived from the level of significance(α) of the test.

The critical value for a two-tailed test is generally taken as 1.96, which is based on the fact that 95% of the area of a normal distribution is within 1.96 standard deviations of the mean.

We compare the critical values(based on α) and test statistics(like z-test, t-test, chi-square test, etc.) to verify the hypothesis.

Z-test :

Sample is assumed to be normally distributed and calculated on basis of parameters like mean and standard deviation. Null hypothesis is that the mean of the sample is the same as the original dataset.

z = (x — μ) / (σ / √n)

x= sample mean

μ = population mean

σ / √n = population standard deviation

T-test:

It also assumes the distribution to be normal and is used to compare the mean of two samples.

t = (x1 — x2) / (σ / √n1 + σ / √n2)

x1 = mean of sample 1

x2 = mean of sample 2

n1 = size of sample 1

n2 = size of sample 2

Chi-square test:

It is used in the case of categorical variables. Null hypothesis is that the two variables are independent.

Χ2 = Σ [ (Or,c — Er,c)2 / Er,c ]

Or,c = observed frequency count at level r of Variable A and level c of Variable B

Er,c = expected frequency count at level r of Variable A and level c of Variable B

If the value of the test statistic is lower than the critical value, we accept the null hypothesis or else reject the hypothesis.

Hypothesis testing is an essential procedure in statistics owing to the evaluation of two mutually exclusive events about a population, followed by determination of viable event based on the supporting sample data.

For further reading, head over to Towards Data Science for applications of the same in Python/R.