Confusion Matrix

We have all studied about the matrices and vectors in our Schools and Colleges lectures. Well matrix is a kind of N dimensional array that represents rows and columns and has its various types used across in field of Statistics and Science. But here we are not talking about the matrices that we learnt in our math lectures.

In the field of Artificial Intelligence and Machine Learning, there is another concept and completely different matrix we use in our algorithm. Confusion Matrix, as the name suggests it has confused everyone who tried to dealt with it in their first attempt. As, giving more and more time to it and using it more often we always had and can surpass this enigmatic concept of Confusion Matrix. Confusion Matrix is a tabular representation of the data and is used for describing the performance of the Classification model based on test data and predicted data. It is used to measure the effectiveness of the model. Confusion Matrix has its own terms and terminology which makes it easier to understand.

Let me explain each term one by one:

TP — True Positive — Question we asked has the answer positive and we got the answer positive.

TN — True Negative — Question we asked has the answer negative and we got the answer negative.

FP — False Positive — Question we asked has the answer positive and we got the answer negative.

FN — False Negative — Question we asked has the answer negative and we got the answer positive.

We have two terms Accuracy and Error as defined as:

Accuracy = Number of correct Predictions / Total number of Predictions

Error = Number of incorrect Predictions / Total number of Predictions

Let’s take an example of an occurring of an event and based on this event we’ll draw a confusion matrix and check the accuracy for this event. Let there is raining outside for sometimes and not raining for sometimes and our machines predicted the value which based on the event of raining and not raining.

We have two terms Accuracy and Error as defined as:

Accuracy = Number of correct Predictions / Total number of Predictions

Error = Number of incorrect Predictions / Total number of Predictions

Let’s take an example of an occurring of an event and based on this event we’ll draw a confusion matrix and check the accuracy for this event. Let there is raining outside for sometimes and not raining for sometimes and our machines predicted the value which based on the event of raining and not raining.

When we see in above tables of event of Rain and No Rain, 7 out of 10 predictions are right. From this we can see the accuracy and error:

Accuracy = Number of Correct predictions / Total number of Predictions

Accuracy = 7/10 * 100 = 70%

Number of correct predictions for each class.

Rain predicted as Rain (TP) = 3

No Rain predicted as No Rain (TN) = 4

Number of Incorrect Predictions for each class.

Rain predicted as No Rain (FN) = 2

No Rain predicted as Rain (FP) = 1

From the above Confusion Matrix, we can see there is two different types of error: False Positive (FP) and False Negative (FN). Now based on the situation we try to minimize these errors. In some cases, False Positive (FP) is much riskier than False Negative (FN) and we minimize the FP. Similarly, for some cases False Negative (FN) is much riskier than False Positive (FP) and we try to minimize FN.

False Positive (Type 1 Error) — As we can see from above example, when it was not raining and machine predicted as raining. i.e. Not occurring of an event predicted as occurred.

False Negative (Type 2 Error) — when it was raining and machine predicted as not raining. i.e. Occurring of an event predicted as not occurred.

If we see the example of FP and FN based on different situation and we need to minimize them.

At Airport security checking, when keys or coins is been mistaken with weapons, then it is known as False Positive and we need to minimize FP here more than FN. In Quality checking, when a good quality product gets rejected is known as False Positive.

In Healthcare sector, when a machine diagnosed a non-corona patient as positive for corona is known as False Positive. Whereas, if machine diagnosed a corona patient as negative for corona is known as False Negative. And, in this case we need to minimize the FN case as it is riskier than FP case.

A very good example of False Positive and False Negative can make simpler to understand it.

When a pregnant woman is said to be pregnant (TP). When a male is said to be not pregnant (TN). When a male is said to be pregnant (FP) and when a pregnant woman said to be not pregnant (FN).

There are few more terms related to Confusion Matrix that we need to understand and have to deal with them based on different situations.

Sensitivity (or, Recall) also known as True Positive Rate (TPR) –

What percent of happening of an event is been correctly identified out of total happening of events.

Recall (or, TPR) = TP/ (TP + FN)

Precision — what percent of happening of an event correctly identified out of total predicted happening of the events.

Precision = TP / (TP + FP)

False Positive Rate — what percent of not happening of an event incorrectly identified out of total not happening of events.

FPR = FP / (FP + TN)

F — Measure

F — Measure is the Harmonic mean of the Precision and Recall. It describes the performance of the model by taking both Precision and Recall into account.

F — Measure = 2 * Precision * Recall / (Precision + Recall)

Depending on different situations, we need to minimize False Positive and False Negative. For controlling FP and FN, we use F-Beta for it.

· For minimizing the False Positive, we have to maximize the Precision by taking the beta value as 0.5

· For minimizing the False Negative, we have to maximize the Recall by taking the beta value as 2.0

ROC-AUC Curve

Receiver Operating Characteristics (ROC) — Area Under Curve (AUC) is used to draw between TPR and FPR.

Our model accuracy depends on the area within the diagonal line and the curved line. Greater the area will be more accurate our model will be. ROC-AUC curve is used for balanced Classification model.

Precision — Recall curve

Precision — Recall curve is drawn between Precision and Recall (TPR) for imbalanced Classification Model. Logic is also same here. More the area within the curve better our model will be.

I hope you have understood it well. And to follow the confusion matrix, and ROC-AUC curve for real dataset, you can check my code for this on my GitHub Profile.

if anything you want to add or anything that incorrectly written here then kindly help. It will be of great help to me.