SVM and Kernel SVM

Summary

In this article, you will learn about SVM or Support Vector Machine, which is one of the most popular AI algorithms (it’s one of the top 10 AI algorithms) and about the Kernel Trick, which deals with non-linearity and higher dimensions. We will touch topics like hyperplanes, Lagrange Multipliers, we will have visual examples and code examples (similar to the code example used in the KNN chapter) to better understand this very important algorithm.

SVM Explained

The Support Vector Machine is a supervised learning algorithm mostly used for classification but it can be used also for regression. The main idea is that based on the labeled data (training data) the algorithm tries to find the optimal hyperplane which can be used to classify new data points. In two dimensions the hyperplane is a simple line.

Usually a learning algorithm tries to learn the most common characteristics (what differentiates one class from another) of a class and the classification is based on those representative characteristics learnt (so classification is based on differences between classes). The SVM works in the other way around. It finds the most similar examples between classes. Those will be the support vectors.

As an example, lets consider two classes, apples and lemons.