Regression Algorithms: Which Machine Learning Metrics?

Published in

MyDataModels

6 min readNov 3, 2020

Can Machine Learning predict the size of a person based on his/her age and weight? Can Machine Learning compute the price of a house based on its surface, location, and number of rooms? Can Machine Learning determine the revenue of a company based on its marketing budget? These are all typical examples of regression problems.

Metrics for Regression Algorithms

Last time we wrote about binary classification and the corresponding Machine Learning Metrics. We explained that Binary Classification consists of assigning an individual to one of two classes. An illustration is a medical diagnosis for COVID-19 infection. A person is either infected or not.

Regression, in contrast to binary classification, provides an estimate of a continuous value. For instance, what is the forecasted height of a male subject aged 52 who weighs 80 kilos?

Another instance of a regression problem could be to evaluate the sales (in units) of a new breakfast cereal based on the advertising budgets allocated for Facebook, Instagram, Twitter, TV, and radio. We might also need to understand each media’s contribution to sales to optimize the marketing funds further. The underlying link between the age, the weight of a person, and his size is a common regression problem. Similarly, the dependency between a marketing budget and the resulting sales is a regression case.

Each point on the graph is the cereals sales at a specific time. The data points vary as the marketing budget increases. Still, there is (in this case) a linear relationship between the two, where the revenue increases with the marketing budget. The idea is that the dependency between two variables (in our example, the marketing budget and the sales) can be modelled through a graph.

This graph might be linear, or it might be Gaussian or polynomial. This graph allows us to extrapolate what the sales will be, based on the marketing budget planned. The best regression algorithm is the one that best fits the chart with the actual data points.

Let’s consider an existing house.If the data points were the 3D geographical coordinates of numerous house walls, the perfect regression model would be the house’s original floor plan. The more critical the errors, the less accurate the graph, and hence the regression model. How can errors be measured in the case of regression algorithms?

Machine Learning Metrics: Cost Function

When building a regression model, we are attempting to reduce the error an algorithm does. To do that, we select a function to measure the error, also called cost function.

The Machine Learning Metrics we have seen so far for classification, such as accuracy, do not apply to regression. Instead, we use metrics designed for analyzing continuous values. The different Machine Learning Metrics used to evaluate the results of the prediction are :

Mean Absolute Error (MAE),
Mean Absolute Percentage Error (MAPE),
Mean Squared Error (MSE),
Root-Mean-Squared-Error (RMSE),
Maximum Error (ME),
R² or Coefficient of Determination.

Machine Learning Metrics: Mean Absolute Error

MAE is the average of the absolute differences between the actual value and the model’s predicted value.

The bigger the MAE, the more critical the error is. The MAE unit is the same as the predicted variable unit, i.e., a distance is estimated in km, a weight in kilograms.

Therefore, the MAE cannot compare regression models’ performance for distinct categories of data. It is robust to outliers, i.e., extreme values. Hence, it is not suitable for applications where you want to pay more attention to these outliers.

Machine Learning Metrics: Mean Absolute Percentage Error

MAPE or Mean Absolute Percentage Error is the average absolute difference between the actual value and the value predicted by the model divided by the real value.

Its usage is comparable to the MAE, only, since it is a percentage, it allows for comparison between regression models designed for diverse categories of data. It does not give a specific focus to outliers. However, in some cases, we want to use a cost function that emphasizes outliers.

Machine Learning Metrics: Mean Squared Error and Root Mean Squared Error

MSE or Mean Squared Error is one of the most popular metrics for regression algorithms. It is merely the average of the real value’s squared difference with the regression model’s predicted value.

As it squares the differences, it is harder on outliers, leading to over-estimating how bad the model is. Its unit is the square of the variable’s unit.

The RMSE or Root Mean Squared Error is the average root-squared difference between the real value and the predicted value. Its use is similar to the MSE.

Machine Learning Metrics: Maximum Error

ME or Maximum Error is the absolute value of the most significant difference between a predicted variable and its real value.

It is interesting to spot out rapidly how well the model integrates outliers. Typically, if the Maximum Error is much bigger than the RMSE, it might mean that the model has not correctly predicted outliers.

Machine Learning Metrics: R² or Coefficient of Determination

R² or Coefficient of Determination is a prevalent metric. R² uses two mean squared error calculations. While the former is the mean square of each real value versus the average of observations, the latter is the mean squared error of the actual value versus the predicted one.

R² is the one complement of the ratio between these two MSE. R² score ranges from -∞ to 1. The closest to 1 the R², the better the regression model is. If R² is equal to 0, the model is not performing better than a random model.

If R² is negative, the regression model is erroneous. Therefore this last Machine Learning Metric is an excellent tool to evaluate the efficiency of a regression model.

Machine Learning Metrics for Regression In a Nutshell

Let’s consider a perfectly well-centered Gaussian model with very few outliers, all close to the average. In this case, the RMSE is an excellent metric to use. We have seen various metrics to evaluate the performance of different regression algorithms. Depending on the situation, some metrics might be more relevant than others.

In all cases, the goal is to get a fair estimate of the distance between the predicted model and the actual values. Whether the MAE, MAPE, MSR, RMSE, ME, or R² is used, the critical point is to have a subtle understanding of these metrics to use them adequately.

Stay tuned for our next article about multi-class metrics !