This article is intended to give practical advice on one of the metrics for measuring accuracy of Machine Learning models. At the end of this article, you will be able to understand the Mean Absolution Error and how it measures model accuracy.
Have you ever built a machine learning model ? For prediction, classification or any other purpose? But how accurate is your model and how do you measure its accuracy?
Yes, this is the trillion dollar question every Machine Learning Developer or Engineer must answer for every model built. More often than not, we measure the quality of a model based on how accurate it makes predictions.
If you tried to predict the cost price of a 4-bedroom house with a model you built, how close would the predicted cost price be to the actual cost price ?
In an attempt to answer this question, some people predict values with their training data. They then compare these predictions to the actual target values in the training data.
Does this make our predictions accurate ? Maybe not. But let’s see why.
If you attempt to make predictions for the home prices of up to 100000 houses, some of your predictions will be as close to the actual price for the homes, while others could be far off the actual price as possible.
There are many ways of measuring a model’s accuracy. However, the Mean Absolute Error, also known as MAE, is one of the many metrics for summarizing and assessing the quality of a machine learning model.
What exactly does ‘ERROR’ in this metric mean ? We do a subtraction of Predicted value from Actual Value as below.
Prediction Error → Actual Value - Predicted Value
This prediction error is taking for each record after which we convert all error to positive. This is achieved by taking Absolute value for each error as below;
Absolute Error → |Prediction Error|
Finally we calculate the mean for all recorded absolute errors (Average sum of all absolute errors).
MAE = Average of All absolute errors
Given any test data-set, Mean Absolute Error of your model refers to the mean of the absolute values of each prediction error on all instances of the test data-set. Prediction error is the difference between the actual value and the predicted value for that instance.
Statistically, Mean Absolute Error (MAE) refers to a the results of measuring the difference between two continuous variables. Let’s assume variables M and N represent the same phenomenon but have recorded different observations.
For a given scatter plot of x points, where point j has coordinates (Mj, Nj). Our Mean Absolute Error (MAE) will be the average vertical distance between each point and the N=M line. This is also known as the One-to-One line. MAE will also at this point be the average of total horizontal distance between each point and the N=M line.
In Machine Learning, MAE is a model evaluation metric often used with regression models. For a worked example of MAE calculation, do check my next article on Mean Absolute Error (MAE) ~ Sample Calculation.
Also in coming articles I will give an explanation of other metrics for verifying accuracy of our model such as Root mean squared error (RMSE). I will also compare their advantages, disadvantages and similarities.
