Addressing Overfitting

How to Mitigate Overfitting with Dimensionality Reduction

Addressing the problem of overfitting — Part 3

Photo by Rene Böhmer on Unsplash

So far, we’ve completed Part 1 and Part 2 of the “Addressing the problem of overfitting” article series. You already know that there are so many options to address the problem of overfitting. Cross-validation and Regularization are two well-known techniques that we’ve already discussed. Dimensionality reduction (DR) is another useful technique that can be used to mitigate overfitting in machine learning models. Keep in mind that DR has many other use cases in addition to mitigating overfitting.

When addressing overfitting, DR deals with model complexity. A model can become more complex when there are many features in the data. Complex models tend to overfit the data. DR takes care of model complexity by reducing the number of features (dimensionality) in the data. There are two ways to reduce dimensionality in the data:

• Find a new set of features that contains different values than in the original dataset. A transformation is applied. This is the method that we discuss today.
• Keep the most important features and remove the redundant (unnecessary) features in the dataset. The original values in the dataset remain unchanged and no…