Bagging Vs Boosting In Machine Learning

Understand How Bagging And Boosting Work And Help Machine Learing Models

This article aims to provide an overview of the concepts of bagging and boosting in Machine Learning. These are both most popular ensemble techniques known.

In particular, I will explain what bagging is, then I will document what boosting is and finally, I will explain the differences between them.

I will also demonstrate the techniques in Python.

1. What Is Bagging?

• Bagging, which is also known as bootstrap aggregating sits on top of the majority voting principle.
• The samples are bootstrapped each time when the model is trained. When the samples are chosen, they are used to train and validate the predictions. The samples are then replaced back into the training set. The samples are selected at random. This technique is known as bagging.
• To sum up, base classifiers such as decision trees are fitted on random subsets of the original training set. Subsequently, the individual predictions are aggregated (voting or averaging etc). The final results are then used as predictions. It reduces the variance of a black box estimator.