Hypothesis Testing in Python Part 2: Two-Sample and ANOVA Tests

Photo by Myriam Jessier on Unsplash

Hypothesis testing is a crucial concept in statistics, used to make inferences about populations based on sample data. This article delves into two-sample t-tests and ANOVA tests using Python, providing clear explanations and practical examples. You can follow along with this notebook on Kaggle for a hands-on experience: Hypothesis Testing in Python.

Performing t-tests

T-tests are used to determine if there are significant differences between the means of two groups. There are three types of t-tests:

1. One-sample t-test: Compares the mean of a single group against a known mean.
2. Two-sample t-test: Compares the means of two independent groups.
3. Paired t-test: Compares means from the same group at different times (e.g., before and after a treatment).

Hypothesis Testing Workflow

The general workflow for hypothesis testing is as follows:

1. State the hypotheses: Define the null hypothesis (H0) and the alternative hypothesis (H1).
2. Choose significance level (α): Commonly set at 0.05.
3. Collect and summarize the data.
4. Perform the statistical test.
5. Make a decision: Reject or fail to reject H0 based on the p-value.

Two-Sample Mean Test Statistic

In a two-sample t-test, we compare the means of two independent groups to determine if they are significantly different from each other. The test statistic can be calculated as follows:

Where:

Here’s how you can calculate it in Python:

import numpy as np
from scipy.stats import ttest_ind

# Example data
group1 = np.random.normal(10, 2, 30)
group2 = np.random.normal(12, 2, 30)

# Two-sample t-test
t_stat, p_val = ttest_ind(group1, group2)
print(f"Two-sample t-test: t={t_stat:.3f}, p={p_val:.3f}")

Output

When you run the above code, you might get an output similar to the following:

Two-sample t-test: t=-3.428, p=0.001

This output indicates that the t-statistic is approximately -3.428 and the p-value is 0.001. Since the p-value is less than the common significance level of 0.05, we reject the null hypothesis and conclude that there is a statistically significant difference between the means of the two groups.

Calculating p-values from t-statistics

The p-value indicates the probability of obtaining test results at least as extreme as the results actually observed, under the assumption that the null hypothesis is correct. In Python, we can use the scipy.stats library to calculate p-values.

Why is t Needed?

The t-distribution is used instead of the normal distribution when the sample size is small or the population standard deviation is unknown. It accounts for the extra variability by having fatter tails.

The t-Distribution

The t-distribution is similar to the normal distribution but with heavier tails, which means it is more prone to producing values that fall far from its mean. As the sample size increases, the t-distribution approaches the normal distribution.

From t to p

To convert a t-statistic to a p-value, we use the cumulative distribution function (CDF) of the t-distribution. This can be done using Python’s scipy.stats.t.cdf function.

Paired t-tests

A paired t-test compares means from the same group at different times. The test statistic for a paired t-test is:

where “d bar” is the mean of the differences and sds_dsd​ is the standard deviation of the differences.

Is Pairing Needed?

Pairing is needed when the measurements are not independent, such as in pre-test and post-test scenarios. If the measurements are independent, a two-sample t-test should be used instead.

Visualizing the Difference

Visualization can help understand the difference between groups. Box plots, histograms, and scatter plots are commonly used to visualize the data before performing t-tests.

Using ‘ ttest()'

Python’s scipy.stats.ttest_ind function can be used to perform a two-sample t-test, and scipy.stats.ttest_rel for a paired t-test. Here's an example:

import numpy as np
from scipy.stats import ttest_ind, ttest_rel

# Example data
group1 = np.random.normal(10, 2, 30)
group2 = np.random.normal(12, 2, 30)

# Two-sample t-test
t_stat, p_val = ttest_ind(group1, group2)
print(f"Two-sample t-test: t={t_stat}, p={p_val}")

# Paired t-test
pre = np.random.normal(10, 2, 30)
post = pre + np.random.normal(1, 1, 30)

t_stat, p_val = ttest_rel(pre, post)
print(f"Paired t-test: t={t_stat}, p={p_val}")

ANOVA Tests

ANOVA (Analysis of Variance) tests are used to compare the means of three or more groups. The null hypothesis for ANOVA is that all group means are equal, while the alternative hypothesis is that at least one group mean is different.

Visualizing Many Categories

Box plots and violin plots are effective for visualizing data from multiple categories. Here’s an example using seaborn:

import seaborn as sns
import matplotlib.pyplot as plt
import pandas as pd

# Example data
data = pd.DataFrame({
    'Group': np.repeat(['A', 'B', 'C'], 30),
    'Value': np.concatenate([np.random.normal(10, 2, 30), 
                             np.random.normal(12, 2, 30), 
                             np.random.normal(11, 2, 30)])
})

sns.boxplot(x='Group', y='Value', data=data)
plt.show()

Conducting an ANOVA Test

The scipy.stats.f_oneway function can be used to perform a one-way ANOVA test in Python. Here's an example:

from scipy.stats import f_oneway

# Example data
group_A = np.random.normal(10, 2, 30)
group_B = np.random.normal(12, 2, 30)
group_C = np.random.normal(11, 2, 30)

# One-way ANOVA
f_stat, p_val = f_oneway(group_A, group_B, group_C)
print(f"ANOVA test: F={f_stat}, p={p_val}")

Pairwise t-tests

If the ANOVA test indicates significant differences, pairwise t-tests can be performed to determine which groups differ from each other. This can be done using the statsmodels library:

import statsmodels.stats.multicomp as mc

# Example data
data = pd.DataFrame({
    'Group': np.repeat(['A', 'B', 'C'], 30),
    'Value': np.concatenate([np.random.normal(10, 2, 30), 
                             np.random.normal(12, 2, 30), 
                             np.random.normal(11, 2, 30)])
})

# Pairwise t-tests
comp = mc.MultiComparison(data['Value'], data['Group'])
post_hoc_res = comp.tukeyhsd()
print(post_hoc_res)

Conclusion

Hypothesis testing using t-tests and ANOVA tests is a fundamental skill in data analysis. Python, with libraries like scipy and statsmodels, provides powerful tools for performing these tests and interpreting the results. By understanding and applying these methods, you can make data-driven decisions with confidence. For more detailed examples and interactive code, check out the Hypothesis Testing in Python notebook on Kaggle.

Check the next part, Part 3, to continue your learning journey.

Appreciation for Exploring My Work

Thank you for taking the time to explore my work. If you found it valuable, please consider showing your support through an upvote or leaving a comment/feedback to help improve the notebook.

Contact

LinkedIn | GitHub | Kaggle | DataCamp