🎢 Hypothesis Testing: Confidence Interval, Level & Margin of Error🎢
In the series of Hypothesis testing, we last discussed the Central limit theorem.
Building upon that, in this post, our prime focus will be on covering aspects such as Confidence Interval, Confidence Level & Margin of Error.
Intuition
In the world of statistics and data analysis, we often encounter situations where it is practically impossible to measure an entire population.
Instead, we take samples from the population to draw conclusions about various parameters of interest, such as averages, proportions, or differences.
However, working with samples introduces uncertainty, and we need tools to quantify that uncertainty.
This is where confidence intervals come into play. It helps us to understand the level of uncertainty in our estimates.
Example: Coffee Preferences Among University Students
Suppose you want to know the average number of cups of coffee consumed daily by university students.
The university has a large population of over 10,000 students. It’s impractical and time-consuming to ask every student about their coffee consumption. So, you decide to survey a random sample of 100 students and ask them how many cups of coffee they typically drink per day.
You find that the average number of cups of coffee consumed in the sample is 3.5 cups, with a standard deviation of 1.2 cups.
But these 3.5 cups may not remain the same every time. Every drawn sample may give another result. But when we repeat these samples multiple times then the chances are high that it may give the population estimates. But it is not always true.
Every statistician or Program expert, repeats these sampling processes multiple times and measures the probability. This probability is nothing but the confidence level.
Confidence Level
Confidence level provides the likelihood of sample statistics meeting the population parameter.
The most common levels of confidence used are 90%, 95%, and 99%.
Normally, it is assumed to be 95%. However, every organization or domain expert provides their own confidence level.
In the given image, if you see it carefully the True Mean is 10 and there are multiple samples drawn. 95% of all sample statistics meet the population parameter. Red intervals depict those 5% where sample statistics don’t match the true mean.
It is often expressed as (1 — α), where α (alpha) is the significance level or the probability of making a Type I error.
For a 95% confidence level, a Type 1 error says that there is a 5% chance we mistakenly conclude that there is a significant effect which in reality isn’t.
Confidence Interval
Given the confidence level, we can define our confidence interval stating that these are the likely values for the population parameter (average cups of coffee consumed per day).
A confidence interval provides a range of values within which we can reasonably expect the true population parameter to fall, based on our sample data and a specified level of confidence.
The confidence interval consists of an upper and lower limit, which represents the range of values within which we can reasonably expect the true population parameter to fall.
Connecting to Sampling Distribution
If you remember, while learning sampling distribution, we learned that the mean (point estimate) of sample distribution centered around the true mean.
It may or may not lie there. The magnitude of how far or how close this estimate would be from the true mean is not known to us.
Confidence interval eliminates this limitation by providing a plausible range of values you can expect your estimate to fall in even if you redo your test.
So, instead of relying solely on a single-point estimate, we consider a confidence interval.
Kindly follow, Ashish Arora, and give a clap to this if you are finding this better! Your support motivates me and helps me reach more people.
Confidence Interval is calculated as:
In order to calculate the confidence interval, it requires three key components:
- Pont Estimate (Sample Mean)
- Confidence Level or Level of Significance
- Margin of Error (Measure of Uncertainty)
For example, if you conduct a survey and find that the average age of a sample of 500 participants is 35 years with a margin of error of ±2 years at a 95% confidence level, this means you’re 95% confident that the true average age of the entire population falls within the range of 33 to 37 years.
But how come these 2 years are calculated by a margin of error, Let’s find that.
Margin of Error:
It represents the maximum likely distance between the point estimate and the true population parameter.
A smaller margin of error indicates a more precise estimate, while a larger margin of error reflects greater uncertainty.
The margin of error is calculated by multiplying the critical value by the standard error.
Margin of Error = Critical Value * Standard Error
Or,
Margin of Error = Z * (σ / √n)
I know you must be thinking about what this standard error and critical value is now, why are we multiplying it and what will we get from this.
Standard Error
The standard error measures how much the sample statistic is expected to vary from sample to sample. The larger the standard error, the less precise the estimate is likely to be.
When the population standard deviation is known (which is often not the case), the formula for the standard error is:
Standard Error = Population Standard Deviation / √(Sample Size)
In practice, the population standard deviation is often unknown, so we use the sample standard deviation as an estimate.
Standard Error = Sample Standard Deviation / √(Sample Size)
This formula makes sense because it tells us that the larger the sample size, the smaller the standard error will be. This is intuitive because as you collect more data points, your estimate of the population parameter (e.g., mean) becomes more accurate and less variable.
Dividing by square root is a mathematical adjustment that helps reflect the diminishing variability in the estimate of the population parameter as your sample size grows larger.
One last thing about the standard error is that people often confuse it with the standard deviation of the sampling mean which is not.
Critical Value
A critical value is a specific value derived from a probability distribution (such as the normal distribution) that corresponds to a particular level of confidence.
The critical value indicates how many standard deviations you need to move away from the mean of the distribution to capture a certain percentage of the data.
It doesn’t have its own unit but rather it is used to scale the standard error based on the desired level of confidence.
By multiplying the critical value by the standard error, you effectively scale the standard error to the appropriate level for the chosen confidence level.
Here’s a simplified example:
Let’s say you’re constructing a 95% confidence interval for the mean height of a population based on a sample. The critical value associated with a 95% confidence level might be 1.96 (approximately). If the standard error of the sample mean is 0.5 inches, then:
Margin of Error = Critical Value * Standard Error Margin of Error
= 1.96 * 0.5 = 0.98 inches
This means that you expect the true population mean to be within approximately 0.98 inches of your sample mean with 95% confidence.
But how do we calculate this critical value?
To calculate the critical value, we have to decide two things:
- Probability Distribution (Z-dist/T-dist/Chi-dist)
- One Tail Interval or Two Tail Interval
One-Tailed Interval: Used when you’re interested in a specific direction of effect, like whether a parameter is greater or smaller than a certain value.
Two-Tailed Interval: Used when you’re interested in any significant difference from a certain value, regardless of the direction. It’s more general and covers deviations in both directions.
- For a two-tailed interval, divide your alpha by two to get the alpha value for the upper and lower tails.
For example, if the desired level of confidence is 95%, the significance level for each tail would be 0.05/2 = 0.025.
Suppose you are using the z distribution, based on that you will look at the z table for the probability 1–0.025=0.975.
In the Z-table, the probability value of 0.975 the critical value is 1.96.
In our tea, example we did not have any idea, whether the students drank less than 3.5 or more than 3.5. Hence, a two-tailed interval will be used.
Margin of error = 1.96*(1.2/√100) = 0.25 (rounded)
95% Confidence Interval = (3.25 cups, 3.75 cups)
The width of the confidence interval is influenced by factors:
- Sample size: Smaller the sample size, Higher the variability, and bigger the margin of error, and lower the degree of precision in point estimation.
- Confidence Level: Higher confidence levels, such as 99%, provide a higher degree of certainty but result in wider intervals. Wider the intervals, Greater the likelihood of capturing the true value but in response Lower the degree of precision in point estimation.
- Higher Variability: Higher the variability, Higher the chances of including extreme values leading to Larger Margin of Error and Lower Precision in Point Estimation
— — — — — — — — — — — — —
So, this is all from this post, in the next post we will discuss how to conduct Hypothesis Testing, what are the various types of hypothesis testing, and a sound intuition of it.
Happy learning!
