This article continues from the previous: K-Nearest Neighbours (KNN).
Remember back in the section on Regression, I touched on the idea of Support Vector Regression? Well, like I said, you can think of SVR as a type of SVM as SVM is quite a broad topic.
For today’s article, I will explain a bit more about the mechanisms of SVM as a refreshment, and in the next upcoming article, I will dive a bit more deeply into SVM, with the “curved” separation boundary.
Classification is all about separating data into different classes, and as the article on SVR mentioned, SVM is all about finding the best line or separating boundary to separate the data.
Let’s begin with a quick refreshment with the following image:
The gap between the two decision boundaries are known as Maximum Margin. For the two boundaries, we will name one as positive hyperplane while the other one as negative hyperplane.
The special part of SVM
Unlike many other algorithm, SVM works in an unique way.
As we all know, if we want to train a classification model to distinguish between apples and pears, most of the algorithms go by seeing which category has the most apples, and which has the most pears. Then, classify a new object to the category that it belongs to.
However, for SVM, for it to distinguish between apples and pears, it goes by looking for the apples that look very much like oranges, and vice versa. Then, SVM will pick the one that looks the most like the oppositie.
This characteristic allows SVM to look at the extreme cases. These extreme cases are also the vectors known as support vectors that are very close to the decision boundaries.
To understand deeper about SVM, it is important to first get our head around this concept. If this sounds unfamiliar, be sure to have a read on my previous article Support Vector Regression.
In the next article, I will explain the Kernel of SVM, sit tight.
> Continue reading: Kernel SVM
