Taking on T-Tests with SciPy
Ah T-Tests, one of the many ways to determine statistical significance. But what exactly are they? And how exactly does everything work?
T-Tests, otherwise known as a Student’s T-Test, are a simple way to compare averages between two groups to assess whether or not they are different from each other. They are frequently used when one either does not know the true population standard deviation, or if the sample size is small. This is all done assuming that the data is normally distributed.
There are two kinds of T-Tests, a one-tailed T-Test, and a two-tailed T-Test.
A one-tailed T-Test is used to determine if the mean of one group is much higher than the other, or much lower, depending on which you select. Only one is selected for the test.
On the other hand, a two-tailed T-Test is used to see if mean of one group is higher or lower than the other group.
one group is higher or lower than the other group.
To start, you have to actually gain a basic understanding of your data. Once you do, you’ll know what questions to ask, and then determine your null and alternative hypothesis, 𝐻0 and 𝐻1 respectively.
For example, a company is trying to use a new process of brewing coffee that results in less acidity. What kind of test is used for this?
That’s right, a one-tailed T-Test! So, let’s say that the population mean of the new coffee, or 𝜇 (pronounced “mew”), is 4.4 ph. We will be estimating this using the sample mean, x̄ (x-bar).
First, we’re going to set 𝐻0 and 𝐻1. For our null hypothesis, we’re going to assume that there will be no difference in acidity, and for the alternative hypothesis we will state that the new coffee’s acidity will be lower than standard coffee.
In more mathematical terms:
𝐻0:𝜇=4.4
𝐻1:𝜇<4.4
Next, we need to determine the alpha value, 𝑎, so that we can compare it to the resulting P-Value and determine if our results are statistically significant.
Wait, what the heck are those?!
The alpha value is how much risk we’re willing to take when it comes to the probability of the experiment returning a false positive (we falsely reject the null hypothesis) or a false negative (we falsely fail to reject the null hypothesis). The lower 𝑎a is, the lower the chance of a false positive. However, if an alpha value is too low then you risk getting a false negative! Typically, 𝑎 is set to 0.05 or 0.01, 5% or 1% risk.
Simplified, the P-Value is the probability of seeing a sample statistic at least as extreme as the one observed assuming 𝐻0 is true.
A low P-Value indicates that it is unlikely that the results you obtained from your experiment happened by chance, while inversely a high P-Value suggests that it is likely that your results did happen due to natural variation.
For the coffee experiment, we’ll just set 𝑎 to 0.05.
Now we can begin our experiments! Or more accurately in this case, grab the example experiment data and check it out. Our coffee company only made 100 cups of coffee using this new method, so this is a pretty good sample size!
Note that that as the number of samples, 𝑛n, becomes larger, the resulting distribution of the mean will reach a more distinct bell curve shape! This applies no matter what the initial sample or population distribution actually was. This is called Central Limit Theorem, and you can read more about it here and here!
#Generating our coffee experiment using NumPy. In this example, the true population mean is 4.8 but we don't know that yet!import numpy as np
np.random.seed(42)
sample = np.random.normal(size = 100, loc = 4.8).round(decimals=2)
Now… what exactly does this tell us? Well, from looking at the mean we can see that on average, this new coffee’s PH lies at 4.7, meaning so far the mean, x̄, is higher than 𝜇. But we can’t stop here, it’s time to move on to calculating the P-Value!
We could use a bunch of different math formulas to figure out what the P-Value is, however, we don’t have to! The SciPy library has everything we need!
Remember, these are the current known values we’ll be using:
𝑎=0.05
𝜇=4.4
x̄=4.7
Using that, we can then use the handy stats ttest feature to find the P-Value. Since we’re testing how a single process may have changed the acidity of the coffee, we’ll be using SciPy’s 1 Sample T-Test function. Since we’re using SciPy’s ttest_1samp function, it will calculate the degrees of freedom for us. Otherwise we would need to do it by hand, shown and explained here.
Since this function assumes that we’re using a Two-Tailed T-Test, we’ll have to divide the results by two
from scipy import statsa = 0.05 # Alpha Value
mu = 4.4 # Population Meant_stat, p_value = stats.ttest_1samp(sample, mu)
print(p_value/2)#returns 0.0007570321424783612
Our resulting P-Value was 0.00076! From that, we can see that our resulting P-Value is absolutely smaller than 𝑎, meaning we can reject our null hypothesis! We do not have sufficient evidence to conclude that the acidity is lower in the new coffee however, so more experiments will need to be done in the future!
But wait, what about a two-tailed T-Test? Well, we’d be doing something similar! Say this coffee company was simply trying to figure out if the new coffee’s acidity was simply different. Our null and alternative hypothesis would simply be:
𝐻0:𝜇=4.4
𝐻1:𝜇≠4.4
Our alpha value would say the same, but this time when we’re interpreting the results, we’ll be dividing it in two! -0.025 and 0.025 respectively. When interpreting the results, we’ll be looking at the P-Value to see whether or not it either falls below -0.025 or 0.025. If it does, then that means that we reject the null hypothesis! You do not need to divide the P-Value by two for a two-tailed T-Test since SciPy is calculating that value by default.
There are many other kinds of T-Test functions within SciPy, we only used the one-sample T-Test function! Be sure to check out the documentation to see what else there is!
Additionally, this is only a brief review and introduction of this subject! If you’re interested in further reading, Wikipedia can be a great place to start! There are tons of resources out there covering different parts of this subject, so go out and explore!
