In Depth: Parameter tuning for SVC

Mohtadi Ben Fraj
Jan 5, 2018 · 4 min read

In this post we will explore the most important parameters of Sklearn SVC classifier and how they impact our model in term of overfitting.

Support Vectors Classifier tries to find the best hyperplane to separate the different classes by maximizing the distance between sample points and the hyperplane.

We will use the Iris data from sklearn.

We start by loading the libraries and SVC

import numpy as np
import matplotlib.pyplot as plt
from sklearn import svm, datasets
%matplotlib inline
# import some data to play with
iris = datasets.load_iris()
X =[:, :2] # we only take the first two features. We could
# avoid this ugly slicing by using a two-dim dataset
y =

We use the same plotting script for our plots.

def plotSVC(title):
# create a mesh to plot in
x_min, x_max = X[:, 0].min() — 1, X[:, 0].max() + 1
y_min, y_max = X[:, 1].min() — 1, X[:, 1].max() + 1
h = (x_max / x_min)/100
xx, yy = np.meshgrid(np.arange(x_min, x_max, h),
np.arange(y_min, y_max, h))
plt.subplot(1, 1, 1)
Z = svc.predict(np.c_[xx.ravel(), yy.ravel()])
Z = Z.reshape(xx.shape)
plt.contourf(xx, yy, Z,, alpha=0.8)
plt.scatter(X[:, 0], X[:, 1], c=y,
plt.xlabel(‘Sepal length’)
plt.ylabel(‘Sepal width’)
plt.xlim(xx.min(), xx.max())


kernel parameters selects the type of hyperplane used to separate the data. Using ‘linear’ will use a linear hyperplane (a line in the case of 2D data). ‘rbf’ and ‘poly’ uses a non linear hyper-plane

kernels = [‘linear’, ‘rbf’, ‘poly’]for kernel in kernels:
svc = svm.SVC(kernel=kernel).fit(X, y)
plotSVC(‘kernel=’ + str(kernel))


gamma is a parameter for non linear hyperplanes. The higher the gamma value it tries to exactly fit the training data set

gammas = [0.1, 1, 10, 100]for gamma in gammas:
svc = svm.SVC(kernel=’rbf’, gamma=gamma).fit(X, y)
plotSVC(‘gamma=’ + str(gamma))

We can see that increasing gamma leads to overfitting as the classifier tries to perfectly fit the training data


C is the penalty parameter of the error term. It controls the trade off between smooth decision boundary and classifying the training points correctly.

cs = [0.1, 1, 10, 100, 1000]for c in cs:
svc = svm.SVC(kernel=’rbf’, C=c).fit(X, y)
plotSVC(‘C=’ + str(c))

Increasing C values may lead to overfitting the training data.


degree is a parameter used when kernel is set to ‘poly’. It’s basically the degree of the polynomial used to find the hyperplane to split the data.

degrees = [0, 1, 2, 3, 4, 5, 6]for degree in degrees:
svc = svm.SVC(kernel=’poly’, degree=degree).fit(X, y)
plotSVC(‘degree=’ + str(degree))

Using degree=1 is the same as using a ‘linear’ kernel. Also, increasing this parameters leads to higher training times.

The inDepth series investigates how model parameters affect performance in term of overfitting and underfitting. You can check parameter tuning for tree based models like Decision Tree , Random Forest, Gradient Boosting and KNN.

All things AI

This publication is dedicated to all things AI.

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store