Let me take an example. Consider a dumb spam classifier that always says no ‘spam’ i.e never identifies a spam mail.
Here the accuracy would be 83.3% and one would be tempted to think that the model has performed very well while in reality, this model is completely useless since it never classifies a mail as spam. In such a case ‘Accuracy’ as a metric completely fails. Here Precision and Recall are the metrics which should be used.
Precision means out of all the examples the classifier labeled as positive, what fraction were correct? On the other hand, recall means out of all the positive examples there were, what fraction did the classifier pick up?
Now whether we want high Precision or High Recall depends upon the business problem at hand. Here are two examples:
- Consider a system that detects fraud in banking transactions. The bank would want a high recall since it is very important that all fraud is identified or at least suspicions are raised.
2. Consider a model built to detect negative sentiments on the e-commerce site of a company. Here we can go for higher Precision value at the expense of losing Recall since we don’t lose much in this case and the source of data is so massive anyway.