ESTIMATION STATISTICS AT A GLANCE

Estimation is a technique in statistics used to quantify the effect of sample data on a population parameter. Estimation statistics is the process or methods for measuring the true value of a population parameter using the sample statistics.

This article, as part of the #7days_of_stats by DSN AI+ club University of Ibadan, promises to be enlightening as it provides insight into

• Why Estimation Statistics?
• Effect Size
• Interval Estimate and
• Meta Analysis

WHY ESTIMATION STATISTICS?

In statistics, the probability value called p-value has been used to inform the statistical significance of a hypothesis. This approach has been misinterpreted over the years due to the fact that it only presents the likelihood that an effect exist without commenting on the size of the effect, knowing fully well that it is possible for a result to be statistically significant and trivial and also statistically non-significant and important. Hence, the need to know the effect size of an experiment in order to relate better with the outcome.

Types of Estimation Statistics

There are three major types of estimation statistics. They are:

1. Effect size
2. Interval Estimation, and
3. Meta Analysis

Let us begin by exploring each of these estimates in statistics.

EFFECT SIZE

Effect in statistics is the actual quantification of difference or relationship of the result in an experiment.

Effect size method, which is also referred to as point estimate, refers to a suite of statistical tools from the field of estimation statistics which helps in quantifying the exact size of the effect in the result of an experiment. Effect sizes generally depends on the assumption that ‘control’ and ‘experimental‘ group values are normally distributed and have the same standard deviations. Effect size methods can therefore be grouped based on the type of effect to be quantifies. The two main groups are:

• Association and
• Difference

The result of an effect size can be interpreted based on goals of the interpretation and statistical method used. In practice, effect size tells the reader the magnitude with which an event is likely to happen. For example, using the prevalent covid-19 situation, if an experiment is carried out on a newly discovered treatment and the statistical test shows that it has the likelihood of curing the disease, effect size will give us an exact quantifiable measure of how much a dose of the treatment can cure covid-19. In general, knowing the likelihood that a treatment is viable and also the extent of its viability is more insightful.

Calculating and Interpreting Effect Size

Effect sizes either measure the sizes of associations between variables or the sizes of differences between group means. This measure must be chosen based on the goal of the interpretation. This include:

• Standard Result e.g Cohen’s d calculation
• Unit Free Result e.g Correlation coefficient

Cohen’s d

This is an appropriate effect size for comparison between two means. It can be used for example, to accompany the reporting of t-test and ANOVA results. It is widely used in meta-analysis. In general the rule of thumb in cohen’s d suggest that d=0.2 is considered ‘small’ effect size, 0.5 ‘medium’ and 0.8 a ‘large’ effect size. This implies that if two groups’ mean differ with 0.2, the difference is considered small, though it is statistically not significant.

Cohen’s d calculation is not provided in python, but we can do it manually thus:

In python, cohen’s d can be defined as follows:

Pearson r Correlation

The association between variables is often referred to as the ‘r family’ of effect size methods. The pearson correlation coefficient measures the degree of linear association between two real-valued variables. The rule of thumb for interpreting pearson’s r is:

• -1.0: perfect negative relationship
• -0.7: strong negative relationship
• -0.5: moderate negative relationship
• -0.3: weak negative relationship
• 0.0: No relationship
• 0.3: weak positive relationship
• 0.5: moderate positive relationship
• 0.7: strong positive
• 1.0: perfect positive

The pearson’s correlation coefficient can be calculated in python using the pearsonr () spicy function. For example, quantify the size of association between two samples of random gaussian numbers in python as follows:

INTERVAL ESTIMATE

This is the use of sample data to calculate a range of possible values of unknown population parameter. An interval estimate is defined by two numbers between which a population parameter is said to lie. For example, a<µ<b is an interval estimate for the population mean µ. It indicates that the population mean is greater than a but lesser than b.

Interval estimate is the use of sample data to calculate the interval of possible values of an unknown population parameter, which is in contrast to point estimate, a single number. There are 3 types of interval estimation.

• Confidence Interval
• Tolerance Interval
• Prediction Interval

Confidence Interval

Confidence interval is the range of values in which our true values actually lies (our predicted population parameter). It is a range of values which is used to estimate the population parameter from a sample statistics. Confidence interval, as an estimate interval with a specific level, is the probability that the interval estimate will contain the parameter.

Confidence interval can be calculated using the formula below

Prediction Interval

Prediction interval is an estimate of an interval in which a future observation will fall within a certain probability given what has already been observed. Prediction intervals are used in regression models to generate prediction based on specific predictor values. Like confidence interval, prediction intervals are upper bound and lower bound. Unlike confidence intervals, prediction intervals predict the spread of individual observations rather than the mean.

The prediction interval for new response ynew, with predictor’s value xh can be determined using the general formula

click here for more on how to calculate prediction interval .

Tolerance Interval

Tolerance interval is a statistical interval within which with some confidence level, a specific proportion of a sample population falls. A tolerance interval defines the upper and /or lower bounds within which a certain percent of the process output falls with a stated confidence. Minimum percentage of the population that the interval should cover and a confidence level is needed to generate the tolerance interval.

Click here for more on how to calculate tolerance Interval

Difference between Confidence, Prediction and Tolerance Intervals.

1. These are distinct approaches to determine the range of an estimate in statistical analysis. Confidence interval gives a range of the likely location of a true population parameter, but as the sample size increases, the interval eventually converges to a single value, the true population parameter.
2. Prediction interval informs about the distribution of individual values as opposed to the uncertainty in estimating the population mean and it will not converge to a single value as the sample size increases.
3. To compute tolerance interval, it is necessary to specify one percentage that expresses how sure you want to be (confidence level) and another percentage that expresses the fraction of the population the interval will contain (population coverage).
4. If the first value (confidence level) is set to 50% then tolerance interval is simple the same as prediction interval. If the confidence level is set to a higher value say(90%-99%) then the tolerance interval is wider than a prediction interval. Tolerance intervals do not converge to a single value as the sample size increases just like prediction interval
5. Prediction intervals are always wider than confidence intervals due to the added uncertainty involved in predicting a single response versus the mean response.

META-ANALYSIS

Meta-Analysis is a quantitative approach used to systematically assess previous research studies in order to derive conclusion about the body of the researches. Meta analysis are useful when many similar studies with conflicting findings are combined into a stronger finding, than any single study, using statistical methods.

The aim is to combine these independent observations into an average effect size and draw an overall conclusion regarding the direction and magnitude of real-world effects.

Although, not often used in applied machine learning, it is useful to note meta-analyses as they form part of this trust of new statistical methods.

Conclusively, estimation statistics, which are majorly divided into two distinct parts called point and interval estimate, can with statistical test give better insight for generalizing about a particular population parameter using sample statistics.