ML Classification-Why accuracy is not a best measure for assessing??
Let’s know what is Accuracy?
Before saying accuracy is a best or poor measure too evaluate. Knowing what is accuracy in classification ML problem is more important. The condition or quality of being true, correct, or exact; freedom from error or defect is accuracy in English. Let’s talk about accuracy in ML using a example.
Example: Let’s take flight accident data, predicting ALL ‘Landed Safely(1)’ as Landed Safely (1) and ‘Crashed(0)’ as Crashed(0) is said to be 100% accuracy in Classification problem. But, practically 100% accuracy or 0% error is impossible.
Now, After knowing about what is accuracy, let’s talk about ‘How good accuracy as measure of assessing?’
We can’t directly say accuracy is poor measure to evaluate. When the data is balanced accuracy is a good measure of evaluating our model. In other hand if data is imbalanced then accuracy is not a correct measure of evaluation.
Let’s know, ‘What is balanced and imbalanced data?’
On taking our flight accident example, if data contain Landed Safely(1) around 50% and Crashed(0) around 50% in Target variable then it is considered as balanced data. In the other hand if Landed Safely(1) is in more percentage comparatively than Crashed(0) then the data is considered as imbalance data
Why accuracy is poor in imbalanced data?
Consider, When Landed Safely (1) is of 90% and crashed is of 10% in Target variable. Even when model fails to predict any Crashes its accuracy is still 90%. As data contain 90% Landed Safely. So, accuracy does not holds good for imbalanced data.
In business scenarios, most data won’t be balanced and so accuracy becomes poor measure of evaluation for our classification model.
At this point we have a question, When accuracy is not good measure of evaluation then which metric holds good?. Let’s talk on other evaluation metrics in ML Classification problems.
From Confusion Matrix we get more metrics to evaluate the model,
True positives: Target variable labeled as positive that are actually positive
False positives: Target variable labeled as positive that are actually negative
True negatives: Target variable labeled as negative that are actually negative
False negatives: Target variable labeled as negative that are actually positive
Recall: Recall in this context is defined as the number of true positives divided by the total number of elements that actually belong to the positive class (i.e. the sum of true positives and false negatives, which are items which were not labeled as belonging to the positive class but should have been).
Precision :The ratio of correct positive predictions to the total predicted positives. Recall — Also called Sensitivity, Probability of Detection, True Positive Rate. The ratio of correct positive predictions to the total positives examples.
From the above metric it is evident that it is risk to have high false negative, high precision and high accuracy together is a good measure of evaluating but practically both does not maintain there balance in model.
Here we have F1 Score, which is nothing but mean of Recall and Precision. But its not simple mean rather it is harmonic mean.
Conclusion:
Accuracy is not always best or always poor measure for assessing. In cases of imbalance data we are provided with many metrics like Recall, Precision, F1 Score and much more. Measure of assessing depends on business scenario.
This is my first post on medium on ML concepts. Guide me to learn more. More to come…
Contact me in linked in : https://www.linkedin.com/in/krishna-raj-parthasarathy-3a0798111/
Email: raj191996@gmail.com
Love to hear from you…
