Regression Metrics in Machine Learning

Regression Metrics are essential tools used to assess the performance of predictive models that aims to estimate continuous outcomes. These metrics provide information about how well a regression model fits the data and makes predictions. They help researchers, data scientists and analysts determine the effectiveness and accuracy of their regression models.

In Regression analysis, the goal is to build the model that accurately predicts the relationship between independent variables (features) and a dependent variable (target).

Some Common Evaluation Metrics Used in Regression are R-Squared, Adjusted R-squared, Root Mean Squared Error, Mean Squared Error and Mean Absolute Error.

R-Squared :-

• It is Coefficient of Determination.
• R-Squared Indicates how better is my model compared to basic mean model.
• It is Calculated as

• The Range of R-Squared is 0 to 1.
  If R-Squared is close to 1 → It is much better model than mean model.
  If R-Squared is close to 0.5 → It Requires Tuning → 0.4, 0.6.
  If R-Squared is close to 0 → It is Poor Model → 0.1, 0.2 → Discard the Model / Change the Algorithm.

[NOTE :- If R-Squared < 0 → It is Worst Model → In that case, we prefer mean model]

Adjusted R-Squared :-

• Limitation of R-squared keeps increases as we add more variables to model. Even though they might not be significant.
• Adjusted R-Squared eliminated this drawback because it only increases, if the newly added variables actually improves the Predictive Power.
• So Adjusted R-Squared is considered as Modify / Penalized version of R-Squared, that accounts for only significant variables in model.
• The Range of Adjusted R-Squared is 0 to 1.
  If R-Squared = 0.79 , Adjusted R-Squared = 0.78 is close to each other, then we can say that it is a Ideal Model.
  If R-Squared = 0.83 , Adjusted R-Squared = 0.79 is far away from each other, then the Model needs Tuning wrt Feature Selection.

[NOTE :- If the distance of R-Squared and Adjusted R-Squared is more than 3% , it needs Tuning.]

Mean Squared Error :-

• Mean Squared Error works by calculating the squared differences between each predicted values & its corresponding to each other.

Root Mean Squared Error :-

• There is no specific range for Root Mean Squared Error, but lower the RMSE, better the Model.
• It is useful while comparing the models that use the same data but different algorithms.
• The Model that returns the lowest Root Mean Squared Error (RMSE) is the better one.
• It is Calculated as

Mean Absolute Error :-

• Mean Absolute Error (MAE) measures the average absolute difference between the predicted values and the actual values. It provides a straightforward understanding of prediction errors without considering their direction.

Akaike Information Criteria (AIC) / Bayes Information Criteria (BIC) :-

• AIC and BIC are not standard ways to measure how well a regression model predicts outcomes, unlike simpler metrics such as Mean Absolute Error or R-squared.
• AIC and BIC help decide which model is better by balancing how well it fits the data against how complex it is, with lower values suggesting a better choice.

Conclusion :-

Regression metrics are used to evaluate the performance of Machine Learning’s that predict continuous variable. Some common regression metrics include R-Squared, Adjusted R-Squared, Root Mean Squared Error, Mean Squared Error, Mean Absolute Error and AIC/BIC.