Support Vector Regression for Machine Learning
SVR(support vector regression) is actucally a part of SVM (support vector machine),Support Vector Machine is a supervised machine learning algorithm and it is useful in solving both classification and regression problems.
The goal of the SVM is to train a model that assigns new unseen objects into a particular category. It achieves this by creating a linear partition of the feature space into two categories. Based on the features in the new unseen objects , it places an object above or below the separation plane, leading to a categorisation.
Linear Support Vector Regression
SVR uses the principle of SVM and create regression model. In SVR, instead of a line or a hyperplane there is a Ɛ-tube and we have regression line in middle of Ɛ-tube.
This tube has a width of Epsilon and the width as Bush measured vertically is along the axis , not perpendicular to the tube but vertically and this tube itself is called the Ɛ-insensitive that means that any point in the dataset that fall inside the tube we’ll be disregarding their error.
So basically this Ɛ-tube think of it as a margin of error that means we are allowing our model to have those point and not care about any error inside it. So any discrepency between or any like distance between the point and regression line or the hyper plane , we don’t care about the area because it falls with the Ɛ-insensitive tube.
For the points that lies outside the Ɛ-insensitive tube we will consider them and take care of them by measuring the distance between the point and the tube itself and these value/point are called the Slack variables.
The formula for measuring the distance/area is :
Support vector
The points that lies outside the Ɛ-insensitive tube are dictating that what the tube will look like and how the tube will be positioned. So these point outside the tube,they are known as Support Vectors and helps in formation of the Ɛ-insensitive tube.
This method is called Support Vector Regression because effectively all of the points can be represented as a vector in the two dimensional space or a multi-dimensional if we have more feature in the dataset.
Non-Linear Support Vector Regression
In Non-Linear Support Vector Regression , the data is not seperable by using straight line or regression lines. So , to solve this problem we are going to higher the dimensional space and make the data linearly seperable.
Here we increase the dimensional space of the data from 2D space to 3D sapce and now it is clearly visible for us to and now we can easily seperate the data.
Here we used a mapping function(a function that maps the lower dimensional data points in a higher dimensional space), that elevates our data points into a higher dimensional space where they become linearly separable. And we find a hyperplane that classifies the data points into two distinctive classes.
In SVR we increase the dimensional space of data by using Kernel Trick. There are different types of Kernal . Some most used kernels are- the Gaussian RBF Kernel, Polynomial Kernel, Sigmoid Kernel etc.
Here we choose the Gaussian RBF Kern
Kernel
We can assume Kernel as a similarity function which implicitly maps the data to an infinite dimension plane and then find the support vectors in that plane and map the vectors back to the original dimension.
Kernel parameters select the type of hyperplane used to separate the data. Using ‘linear’ will use a linear hyperplane (a line in the case of 2D data). ‘rbf’ and ‘poly’ uses a non-linear hyper-plane.
That’s all for SVR Machine learning Algorithm. Stay tuned for further blogs.
Thank you
Implementation of SVR algorithm on Salary dataset ~
Dataset : Salary dataset
