What is Support Vector Machine (SVM)?
Hello you Machine Learning enthusiasts! This article is going to be on yet another well known classification algorithm known as ‘Support Vector Machine’.
Following topics will be covered in the upcoming sections:
(1) What is Support Vector Machine (SVM)?
(2) How does SVM work?
(3) Advantages and Disadvantages of SVM
(4) Conclusion
(1) What is Support Vector Machine (SVM)?
As we discussed earlier SVM is an algorithm that is used to solve classification problems. The objective of SVM algorithm is to find an optimal hyperplane that maximizes the margin between two classes. Hyperplane is a generalized term to identify the decision boundary in an n-dimensional space.
Figure 1 shows the SVM algorithm in action. The decision boundary is dividing the plane into 2 halves, thus the data points can fall into their respective classes.
Now that we have seen what SVM looks like, let us look at the working of this beautiful algorithm.
(2) How does SVM work?
Before getting into the nuance of this algorithm, let us understand what ‘Support Vector’ are in Support Vector Machine.
Support vector is the perpendicular line from the decision boundary to the closest points in both the classes.
What does Maximum Margin mean in SVM?
The decision boundary (bold line) in Figure 2 is what separates the plane into two halves for the two classes. Hyperplanes (Positive and negative) acts as a buffer, with respect to the decision boundary, so that when new set of data points which can fall somewhere near the decision boundary, falls into it’s respective class. This is one advantage of this algorithm, which is not available in other algorithms such as Logistic Regression, where a new data point can be misclassified if it falls somewhere near the decision boundary. Thus, SVM is considered to be one of the powerful classifiers in machine learning.
The working mentioned above is for Linear SVM (When the dataset can be classified into two classes by a linear line). But what happens when the classes are not linearly separable? This is when we use kernel tricks, where certain functions are used to transform data that are not linearly separable in n-dimensional space to a higher dimension, where it is linearly separable. The below figure is a representation of non-linear SVM.
If we observe carefully, we can see that unlike Linear SVM, the hyperplane is not straight forward. In order to make them linearly separable, a higher dimensional hyperplane had to be generated. Thus, this allows to make a generalized assumption about the data, irrespective of the feature complexity.
Now that we have understood the basic understanding of SVM algorithm, let us now look into it’s advantages and disadvantages.
(3) Advantages and Disadvantages of SVM
(i) Advantages of SVM
- Handles non-linear data efficiently: As we have seen with the non-linear SVM, thanks to the kernel tricks, it can create hyperplanes which can separate the two classes effectively and efficiently.
- Regularization capabilities: SVM are known to have L2 regularization which helps them to prevent over-fitting.
- SVMs can be used to solve both classification problem (SVC) as well as regression problems (SVR). Although SVC are more preferred and widely used than SVR.
- SVMs are known to be memory efficient.
(ii) Disadvantages of SVM
- Difficult to interpret: Unlike some of the algorithms like decision trees, SVMs are difficult to analyze and interpret.
- Choosing an appropriate kernel function: As in the case of solving a non-linear problem, we have seen that certain kernel tricks can be used to handle these kinds of problems. The problem arises in choosing the right kernel function so that a right model is fit to the data and predictions are made.
- Training these models can be time consuming if the dataset size is too large.
- Hyper parameter tuning these models can be a difficult task.
(4) Conclusion
After going through this article we now know what SVM algorithm is, it’s inner working, it’s advantages and it’s limitations. If understood and applied aptly, SVM can be a powerful tool in your arsenal.
Hope you enjoyed reading this article!!
