Member-only story
AdaBoost, Step-by-Step
An Introduction to Boosting and AdaBoost
AdaBoost belongs to the ensemble learning methods and imitates the principle of the “Wisdom of the Crowds”: models that individually show poor performance can form a strong model when combined.
A MIT study [Diz21] published in 2021 describes how crowds are able to identify fake news. Without background knowledge or fact-checking, individuals often find it difficult to reliably identify fake news. Nevertheless, based on our experience, we are usually able to at least give a tendency, which usually performs better than random guessing. If we want to know whether a given headline describes the truth or contains fake news, we could simply ask 100 random people. If more than 50 say that the headline contains fake news, we classify it as fake.
The prediction of several single weak learners combined can result in a strong learner, which is able to distinguish between fake and truth with high accuracy.
With Ensemble Learning, we mimic this concept
Boosting is one of the most popular ensemble learning techniques. A set of so-called weak learners, i.e., models whose performance is slightly better than random guessing, is built. The outputs of individual weak learners are combined as a weighted sum and represent the final output of the boosted…
