Precision vs Recall: A Simple Story
In a recent interview I was asked to rigorously define Accuracy, Precision, and Recall. I was already sweating bullets, but this simple question threw me for a loop.
My interviewer started by drawing a table for which I filled in the green and red labels.
He then asked me again, “Using these values, how do you define precision and recall?”
I was racking my brain, thinking back to my undergraduate machine learning courses to try and dredge the knowledge back. As a practitioner, I use these ideas all the time, but I couldn’t remember the equations.
I did, however, remember the story of Police vs Assassins.
This simple story is a Mnemonic device that helps me to intuitively remember the difference between Precision and Recall.
An Assassin is given a list of targets, and told to take them out. There is no care for casualties as long as the Assassin takes out all of their targets. This is like Recall.
Out of all the True Positives and False Negatives the assassin was supposed to target, how many did they actually finish off? There is no measure of False Positives here, because the Assassin doesn’t care about casualties.
Alternatively, the Police has a list of suspects to detain. They should care very much about detaining people erroneously. False positives are much more important to avoid, while we’re OK with letting some criminals free. This is like Precision.
Out of all the True Positives and False Positives, the Police want to ensure we get as many of the criminals as possible, while taking the False Positives into account. As such, the Police don’t control for False Negatives (criminals that escape) because Precision focuses more on False Positives.
