Road to SVM: Maximal Margin Classifier and Support Vector Classifier

Support Vector Machine is a popular Machine Learning algorithm used in classification tasks, especially for its adaptability to non-linearly separable data (thanks to the so-called Kernel trick). However, before getting to what we use today, several models with the same underlying structure were developed. In this article, I’m going to give you the intuition behind two of them, whose progressive implementation leads to modern SVM.

Those are the Maximal Margin Classifier and Support Vector Classifier. However, before dive into those, let’s first introduce the main object of all of this classifier, which is the separating hyperplane.

Getting familiar with hyperplanes

A hyperplane defined in an h-dimensional space is an object of dimension h-1 which separates the space into two halves. I’ve been talking about the mathematical interpretation of hyperplanes in my former article. To recap, in a generic h-dimensional space, we define a generic hyperplane as:

Now, each vector x* of variables can lie either on the hyperplane if:

Or on one of the two halves if:

For simplicity, let’s consider the following bivariate case: