What is Model Performance measurement in machine learning?

As we discussed in the previous post about the K-Nearest Neighbor algorithm, it is a simple method to classify new data starting from known values. However, there must be a way to measure how well the predictions are made, to decide if it is the correct algorithm to solve our problem or if we need to make some adjustments to the model.

Within Machine Learning this is known as Model-performance Measures, in addition to measuring efficiency, it can also be used to compare the performance of two algorithms, or to assess whether the model performance gets worse or better when evaluating new data.

Depending on the type of data obtained, different performance measures are applied, for KNN and other similar classification algorithms we use accuracy as a performance metric, where accuracy is the total number of correct observations divided by the total number of observations made.

To calculate model accuracy on the data, the training data set must be used to fit the classifier model, but taking into account that it cannot be calculated with unseen data, only known data so as not to generate false assumptions.

The data set must be divided into two parts before training the model. 20–30% of the data is commonly used as test sets and the rest of the data is for training sets. Then, the classifier is trained with the training set, this generates a model that will be used to generate predictions with the test data, finally, the prediction result is compared with the real value to obtain the average of successful observations over the total labeled test data.