Bagging: Machine Learning through visuals. #1: What is “Bagging” ensemble learning?
By Amey Naik & Arjun Jauhari
Welcome to part 1 of “Machine Learning through visuals”. In this series, I want the reader to quickly recall and more importantly retain the concepts through simple visual cues shown in this article. A large body of research indicates that visual cues help us to better retrieve and remember information. Here, the concepts are not discussed in detail. The assumption is that the reader already knows the concept but wants a quick recap of it.
Ensemble learning banks on “wisdom of crowd”. Suppose you ask a complex question to hundreds of random people, then aggregate their answers. In many cases you will find that this aggregated answer is better than an expert’s answer. An ensemble of predictors is more accurate than one best individual predictor.
Bagging is short for “Bootstrap Aggregating”.
In bagging, training instances can be sampled several times for the same predictor.
Let’s try to understand this with a visual cue shown below:
In the above example, training set has 7 samples. Size of the data set for each predictor is 4. This figure shows 4 samples picked up for the first predictor. As you can see, since we replace the ball after sampling each sample hence we might end up with same sample multiple times. In this example, orange ball ends up being picked up 3 times.
Moving on to bagging ensemble learning: the output of each predictor is aggregated as shown below.
Please note: Terms ‘classifier’ and ‘predictor’ are used interchangeably. Both mean the same.
Some pointers about bagging technique:
- Aggregation function is typically the statistical mode for classification and average for regression.
- Aggregation helps reduce both bias and variance.
- Predictors can all be trained in parallel unleashing the potential of a GPU.
- Bagging allows training instances to be sampled several times for the same classifier. (example: “orange” ball is picked up 3 times for classifier 2 as in the above figure)
