# Support Vector Regression Or SVR

This post is about SUPPORT VECTOR REGRESSION. Those who are in Machine Learning or Data Science are quite familiar with the term SVM or Support Vector Machine. But SVR is a bit different from SVM. As the name suggest the SVR is an regression algorithm , so we can use SVR for working with continuous Values instead of Classification which is SVM.

The terms that we are going to be using frequently in this post

**Kernel**: The function used to map a lower dimensional data into a higher dimensional data.**Hyper Plane**: In SVM this is basically the separation line between the data classes. Although in SVR we are going to define it as the line that will will help us predict the continuous value or target value**Boundary line**: In SVM there are two lines other than Hyper Plane which creates a margin . The support vectors can be on the Boundary lines or outside it. This boundary line separates the two classes. In SVR the concept is same.**Support vectors**: This are the data points which are closest to the boundary. The distance of the points is minimum or least.

Why SVR ? Whats the main difference between SVR and a simple regression model?In simple regression we try to minimise the error rate. While in SVR we try to fit the error within a certain threshold. This might be a bit confusing but let me explain.

*Blue line: Hyper Plane; Red Line: Boundary Line*

See fig 2 see how all the points are within the boundary line(Red Line). Our objective when we are moving on with SVR is to basically consider the points that are within the boundary line. Our best fit line is the line hyperplane that has maximum number of points.

*So Let’s Start:*

So the first thing we have to understand is ** what is this boundary line ?(yes! that red line). **Think of it as to lines which are at a distance of ‘e’ (though not e its basically epsilon) but for simplicity lets say its

**‘e’.**

So

the lines that we draw are at‘+e’and‘-e ’distance from Hyper Plane.

Assuming our hyper plane is a straight line going through the Y axis

We can say that the Equation of the hyper plane is

wx+b=0

So we can state that the two the equation of the boundary lines are

Wx+b=+e

Wx+b=-e

respectively

Thus coming in terms with the fact that for any linear hyper plane the equation that satisfy our SVR is:

e≤y-Wx-b≤+e

stating the fact that y=Wx+b

y-Wx-b=0

*This applies for all other type of regression (non-linear,polynomial)*

RECAP

What we are trying to do here is basically trying to decide a decision boundary at ‘e’ distance from the original hyper plane such that data points closest to the hyper plane or the support vectors are within that boundary line

Thus the decision boundary is our Margin of tolerance that is We are going to take only those points who are within this boundary.

Or in simple terms that we are going to take only those those points which have least error rate. Thus giving us a better fitting model.

If my explanation is not sufficient You can check:

Ok let now do what we actually wanna do. Coding Part of course

**DOWNLOAD THE CODE AND CSV FROM HERE:**

*DATA SET DESCRIPTION:*

Predicting the age of abalone from physical measurements.

Name Data Type Meas. Description

---- --------- ----- -----------

Sex nominal M, F, and I (infant)

Length continuous mm Longest shell measurement

Diameter continuous mm perpendicular to length

Height continuous mm with meat in shell

Whole weight continuous grams whole abalone

Shucked weight continuous grams weight of meat

Viscera weight continuous grams gut weight (after bleeding)

Shell weight continuous grams after being dried

Rings integer +1.5 gives the age in years

Link:

https://archive.ics.uci.edu/ml/machine-learning-databases/abalone/abalone.names

**This time lets take it all the way! From Scaling to Feature Selection What say!**

# coding: utf-8

import pandas as pd

df=pd.read_csv(‘./age_mod.csv’)

df.head()

df=df.drop([‘Sex’],axis=1)

from sklearn.svm import SVR

regressor=SVR(kernel=’linear’,degree=1)

import matplotlib.pyplot as plt

plt.scatter(df[‘Shucked weight’],df[‘Age’])

from sklearn.model_selection import train_test_split

xtrain,xtest,ytrain,ytest=train_test_split(x,y)

regressor.fit(xtrain,ytrain)

pred=regressor.predict(xtest)

print(regressor.score(xtest,ytest))

from sklearn.metrics import r2_score

print(r2_score(ytest,pred))

regressor=SVR(kernel=’rbf’,epsilon=1.0)

regressor.fit(xtrain,ytrain)

pred=regressor.predict(xtest)

print(regressor.score(xtest,ytest))

print(r2_score(ytest,pred))

*READ THE CSV FILE*

import pandas as pd

df=pd.read_csv(‘./age_mod.csv’)

df.head()

*WE DON’T NEED THE ‘Sex’ COLUMN SO DELETE*

df=df.drop([‘Sex’],axis=1)

*LOADING THE SVR MODEL FROM sklearn.svm*

from sklearn.svm import SVR

regressor=SVR(kernel=’linear’)

**#NOTE: kernel=’linear’ → we are setting the kernel to a linear kernel**

**#DEFAULT: kernel=’rbf’**

*PLOT THE RELATION:*

import matplotlib.pyplot as plt

plt.scatter(df[‘Shucked weight’],df[‘Age’])

**#try it for other parameters**

**SPLIT INTO TRAIN AND TEST SET**

from sklearn.model_selection import train_test_split

xtrain,xtest,ytrain,ytest=train_test_split(x,y)

**FIT THE MODEL DO THE PREDICTION**

regressor.fit(xtrain,ytrain)

pred=regressor.predict(xtest

)

**CHECK THE ACCURACY**

print(regressor.score(xtest,ytest))

from sklearn.metrics import r2_score

print(r2_score(ytest,pred))

*#NOTE: Both .score() and r2_score gives us a accuracy score prediction*

*LET’S TUNE SO PARAMETERS TO SEE IF WE CAN GET BETTER SCORE:*

regressor=SVR(kernel=’rbf’,epsilon=1.0)

regressor.fit(xtrain,ytrain)

pred=regressor.predict(xtest)

print(regressor.score(xtest,ytest))

print(r2_score(ytest,pred))\

*LOOK HERE:*

SVR(kernel=’rbf’,epsilon=1.0,degree=3)

#here we set the kernel to ‘rbf’ of degree 3 and a epsilon value of 1.0

#by default the kernel is ‘rbf’ degree is 3 and epsilon is 0.1

#other kernels are → ‘linear’,’poly’(for polynomial),’ rbf’

Thanks for Reading!

*Follow my blog at:*

*Polynomial regression:*

*Multiple regression*

*Linear Regression*