Unbiased, efficient, and consistent statistical estimators: What are they actually?
I’m a stat guy so I’d write my first Medium post about stat.
It has always been confusing to me when I read journal articles or CrossValidated: some people said this estimator is consistent while some say that is efficient. These all seemed familiar to me (as I’m a stat graduate after all). But I’ve, to be honest, never get hold of the concrete definitions of those adjectives.
I checked the definitions today and think that I could try to use dart-throwing example to illustrate these words. I’d add ‘biased’ here for the sake of completeness.
Unbiased estimator: If your darts, on average, hit the bullseye, you’re an ‘unbiased’ dart-thrower.
Efficient estimator: Efficiency can be absolute and relative, I’d cover relative one (more common) here. Relative efficiency consist of two additive parts: unbiasedness and variance. So if you’re throwing the darts more steadily (less spread in the dart board) and more accurate (less bias), you’re an ‘efficient’ dart-thrower than the others.
Consistent estimator: This is often the confusing part. Consistency in the statistical sense isn’t about how consistent the dart-throwing is (which is actually ‘precision’, i.e. variance). It is rather talking about the long term performance. Now consider you’re not concentrating on one dart-throwing competition but a whole career. If you’re learning something throughout the experience and the dart pattern on the board becomes more and more concentrated on the bullseye during your career, you’re a ‘consistent’ dart-thrower.
Ok, I’ve to make a disclaimer at this point: these metaphors are never meant to be unbiased, efficient, or consistent, but they’d (hopefully) serve as a starting point for understand these concepts, especially among people from a non-statistical background. If you’re in doubt of the meaning or want to know more, you’re mostly advised to find out the proper mathematical definitions, which should be readily available online.
