Explain Like I’m Five (Part 2) : Support Vector Machine
When Linear Regression doesn’t solve your problem, it is time for a more advanced algorithm, introducing, Support Vector Machine!
Suppose you want to divide two things clearly — for example, we have two distinctive classes, Cappuccino and Black Coffee. Whenever a new drink is similar (whatever its taste or color) to either class, then that drink belongs to that class.
We can draw it on a hyperplane (the graph below) like this:
Imagine X1 is taste, X2 is the color, the green line (H1) are unable to correctly divide the drinks into two distinctive classes (where all drinks on one side consists of only 1 type of drink), the blue line (H2) are able to divide the drinks, but just narrowly able to differentiate between each class, while the red line (H3) are able to divide both drinks with maximum margin.
This is what SVM does, it tries to find the largest separation between each class.
Even though it sounds good, what happens if the data looks like this?
There are at least 2 misclassified data (the blue square between the red circles and the red circle between blue squares), where should we put the line that divides both class?
This is what makes SVM powerful — we allow for some amount of mistakes, while keeping the margin as high as possible. This is called soft margin.
Hopefully, this gives you at least a gentle introduction of what SVM does, there are some other things you should learn such as “Kernel Trick”, which is why I encourage you to keep reading
