Confusion to Clarity ( Confusion matrix demystified )
I would like to start this post by saying that “Confusion Matrix” really confused me too :) I really thought that it was the same for everyone , and hence the name “Confusion Matrix” , but really not.
The name was arrived in terms of the model being built, and used to gauge how much the machine learning model is confused with identifying the correct classes.
Story:
Somewhere in the future ( maybe 2100 AD) , in Eekaako , there was a medical screening centre which had state of the art AI systems to screen patients coming in. The screening centre was specialised in screening for a rare disease named “Eklakophobia” which broke out that year.
Although rare , the disease can spread extremely fast and infect the surroundings unless a strict quarantine protocol is followed ( Something like COVID ).
AI entry:
The job of the AI system is to screen the individuals and classify them as either positive or negative for “Eklakophobia”.
Since AI is also human, it can make mistakes and we have to be tolerant with it, but how much ?
There are 100 patients coming in for screening , out of which 60 are really infected with “Eklakophobia”, the remaining 40 are not.
So , the ground truth is
Infected : 60
Not Infected : 40
All the 100 patients are screened by the AI system and the results of the AI system are as follows
Infected : 50
Not infected : 50
The confusion :
Since this is a highly contagious disease, we cannot afford to leave even a single patient who is actually infected with the disease.
But the AI predictions are not matching with the ground truth and we need to identify how much we have deviated from the reality.
Let’s check the AI result of every instance ( every patient ) with the ground truth and put them into the below categories
True Positive ( TP ) — Really infected , AI also predicts as infected
True Negative (TN) — Really not infected , AI also predicts as not infected
False Positive (FP) — Really not infected , AI predicts as infected i.e, AI has falsely predicted as positive for infection.
False Negative (FN) — Really infected, AI predicts as not infected i.e, AI has falsely predicted as negative for infection.
Lets’s draw the confusion matrix
What these number say ?
I think, now we get some clarity from the confusion…Uh, confusion matrix :)
Infected
Truth — 60 AI prediction — 50
Not infected
Truth — 40 AI prediction — 50
The ideal prediction should be
TP — 60
TN — 40
FP — 0
FN — 0
But from the confusion matrix , we see that there are 10 FPs ,which means that 10 not-infected persons were misclassified as infected.
There are 30 FNs which means that 30 people who were infected were misclassified as not-infected.
Oh God !! The issue is even more serious that we envisioned.
At a glance the initial numbers showed that we had only 10 misclassifications ( https://medium.com/r/?url=http%3A%2F%2FAI_Error )
But confusion matrix revealed that there are 30 infected people who were screened as not infected. This could have been catastrophic….
Thanks to the confusion matrix for saving the day !!!!
Always we get to clarity from confusion, but we need to look into it deeply !!!
