MODEL EVALUATION IN MACHINE LEARNING.
Model evaluation is the process of using different evaluation metrics to understand a machine learning model’s performance, as well as its strengths and weaknesses. It evaluates the performance of different Machine Learning models, based on the same input dataset. The method of evaluation focuses on accuracy of the model, in predicting the end outcomes.
Model evaluation is important to assess the efficacy of a model during initial research phases, and it also plays a role in model monitoring.
There two methods of model evaluation. They are
1. Holdout
2. Cross Validation
Holdout: this method is divided into three sets; testing set, training set, and validation set. The test data is used to calculate the future performance of the model, Training set is used to build the predictive models whereas validation set is used to assess the performance of the model built in the training place
Cross-validation: is a procedure of dividing the whole dataset into data samples, and then evaluating the machine learning model using the other samples of data to know accuracy of the model.
Classification and Regression are the two major examples of supervised Machine Learning, which constitute the majority of machine learning applications. In other to evaluate the performance of machine learning model, there are some metrics to know. In this article, I will provide ways in which models can properly be evaluated.
I. Confusion matrix: a confusion matrix is used to evaluate classification model. It shows the number of correct and incorrect predictions made and compare with the actual result in the data. Example our classes succeed and fail when we feed data with known outcomes into the model we’ll know if the model made correct predictions or not once we have the model’s predictions
There are 4 terms in the confusion matrix:
- True Positives: When the predictions are TRUE, the actual output was also TRUE.
- True Negatives: When the predictions are FALSE, and the actual output was also FALSE.
- False Positives: When the predictions are TRUE, and the actual output was FALSE.
- False Negative: When the predictions are FALSE, and the actual output was TRUE.
II. Accuracy: one of the most popular ways of evaluating a machine learning model is through the use of accuracy. This is good but it often gives wrong results due to model imbalance, overfitting, and so on.
III. Precision : This measures the proportion of predicted cases actually turned out to be positive. Precision is useful in cases where False Positive is a higher concern than False Negatives. The importance of precision is in music or video recommendation systems, e-commerce websites, etc. where wrong results could result in customer dissatisfaction.
IV. ROC/AUC: a graph showing the performance of a classification model. This curve plots two parameters. True positive rate and False positive rate. The ROC chart shows false positive rate (1-specificity) on X-axis, the probability of target=1 when its true value is 0, against true positive rate (sensitivity) on Y-axis, the probability of target=1 when its true value is 1. Thus, the Area Under Curve is the plot between false positive rate and True positive rate at different values of [0,1].
References
developers.google.com
knowledgehut.com
saedsayad.com
analyticsvidhya.com
