Support Vector Machine
Support vector machines or SVM for short is becoming very popular in machine learning because they can be very powerful and they are somewhat different from other machine learning algorithms. And we will find out how they are special.
So imagine you’re trying to teach a machine how to distinguish between apples and oranges how to classify a fruit into either an apple an orange. So you’re telling a machine that, All right I’m going to give you some train data so have a look at all of these apples and oranges. Analyze them. Look at them and see what parameters they have and then next time I’m going to give you a fruit which will be either an apple or an orange and you’re going to classify it and tell me whether it’s an apple or an orange. So that’s kind of a standard machine learning problem.
Now in our case here you can see let’s say on the right we have oranges on the left we have apples. So what machine learning would do is look at the most Apple and the most orange so they would look at the most standard common type of apples and the most standard common type of oranges. A machine would try to learn from the apples that are very much like apple so it would know what an apple and the same for orange. That’s how most of the machine learning algorithms work and then based on that it would be able to come up with some classifying for new data elements.
In the case of a support vector machine, it is a bit different. Instead of looking at the most standard common apples and most standard common oranges what this support vector machine does is actually look at apples that are very much like orange and oranges that are very much like an apple.
The support vectors you can see that they’re actually very close to the boundary. The apple or the red mark would be very close to green marks or oranges and the green mark would be very close to the red marks. SVM searches the classifier line through maximum margin which means this line is drawn equidistant from both support vectors and the sum of margin from support vectors to the line is maximum. So basically these two points are supporting this whole algorithm. We’ve got the line in the middle which is called the maximum margin hyperplane or the maximum margin classifier. In a two-dimensional space, it’s just like a classifier line but in multi-dimension, it is a hyperplane.
Therefore the support vector machine in that sense you can think of it is like a more extreme type of algorithm because it looks at a very extreme case that is very close to the boundary and it uses that to construct its analysis.
