Maximum Likelihood Estimation

An intuitive graphical introduction

The main goal of statistical inference is learning from data. However, data we want to learn from are not always available/easy to handle. Imagine we want to know the average income of American women: it might be unfeasible or highly expensive to collect all American women’s income data. Besides, even in a scenario when this data collection was feasible, there might be imperfect compliance among interviewed women, since some of them might be not willing to share their data.

So the idea of statistical inference is to draw a sample from the population of our interest (in the above example, American women) and then, observing this sample, estimate population features (which are unknown parameters) using sample data. The resulting estimates of the true, unseen parameters are called ‘statistics’ and, in order to be reliable, they need to have some robustness’ characteristics:

• they have to be unbiased: the bias is defined as the difference between the expected value of the statistics and the true parameter. It might be sufficient to reach just the asymptotic unbiasedness, which means, having bias = 0 as n (sample size) tends towards +∞.
• they have to have the lowest variance possible. Indeed, between two unbiased estimators, the most efficient will be that with the lowest variance. Thanks to the Cramer-Rao theorem, we can set a lower bound for the variance of any estimator so that, if our estimator is unbiased and has the variance…