Performance measurement (Part-II)

Understanding precision, recall, F1 score, Macro ,micro averaging methods and roc-auc curve for measuring the performance of machine learning models.

Table of content:

1. Precision
2. Recall
3. F1 score
4. Micro Averaging Method
5. Macro Averaging Method
6. ROC-AUC curve

As we have understood what is confusion matrix and why do we need it in our previous blog. If you have not yet visited it,please check it out here.

Let us discuss about some other terminologies in detail-

Precision

It tells us of all the points our model declared to be positive, what percentage of them are actually positive.

Precision

Recall

It tells us of all the actual positive points , what percentage of them are actually predicted as positive by our model.

Recall

F1 Score

It conveys the balance between precision and recall. In simple words we can say that it is actually the harmonic mean of the two.

2*((precision*recall)/(precision+recall))

F1 Score

For multi-class classification problem we have micro and macro averaging methods.

A macro-average will compute the metric independently for each class and then take the average (hence treating all classes equally), whereas a micro-average will aggregate the contributions of all classes to compute the average metric.

Let us understand them one by one.

Micro Averaging method

It is calculated my taking out the sum of all the true positives, false positives,false negatives of all the classes and then calculating the over all precision and recall.

Let us suppose we have 3 classes with class label as 1,2,3.

Then we calculate TP-1,TP-2,TP-3 and similarly we calculate FP-1,FP-2,FP-3 and FN-1,FN-2,FN-3 add them and now use the total TP,FP and FN for calculating precision and recall.

Micro F1 score

Macro Averaging Method

It is calculated by taking out the precision and recall for each class separately in one vs all way and then take out the average of all precision and recall values to find the F1 score.

using the same example as above-

We can calculate precision for class 1 ,2 and 3 separately and recall also in the same manner and then-

Total-Precision = (precision-1+precision-2+precision-3)/3

Total-Recall = (Recall-1+Recall-2+Recall-3)/3

AUC-ROC Curve

ROC curve stands for receiver operating characteristic curve.

It is a graph showing the performance of a classification model at all classification thresholds.

An ROC curve plots TPR vs. FPR at different classification thresholds where as AUC represents degree or measure of separability.

Here AUC stands for area under the curve.

Higher the AUC, better the model is at predicting positive class as positive and negative class as negative.

ROC-AUC curve

Hope you all are clear with performance measurement.

References :

The Area Under an ROC Curve