The conundrum of ‘C’: SVM hyperparameters
A quick and brief explanation of the effect of C when training an SVM model
SVM’s or Support Vector Machines are supervised learning models used for classification and regression. When using an SVM our preliminary objective is to arrive at a hyperplane that helps us
- Maximize the margin
- Classify the training points accurately
One of the key regularization parameters helping us control or influence the outcome of the model is C also known as penalty parameter or cost parameter for misclassification.
Below are a few values of C and its effects while training a model that is of interest to many enthusiasts when learning SVMs.
a. A low value of C means a lower penalty and allows a little freedom, as a result, the training model has some misclassifications and hence creates a larger margin hyperplane also known as Soft Margin. Lower values of C lead to Underfitting resulting in a high bias and low variance.
b. A high value of C means a higher penalty and is focused on achieving better accuracy by avoiding errors or misclassification and hence opts for a hyperplane with a better classification accuracy resulting in a smaller margin hyperplane also known as Hard Margin. Higher values of C lead to Overfitting resulting in a low bias and high variance.
c. As the value of C approaches positive infinity there is no room for error as the penalty for misclassification is enormous and it results in heavy overfitting as the model pushes for the best possible accuracy.
d. As the value of C approaches zero the model is poorly underfitting and there is potential overlap of the data points and it's most likely the model might be unable to find an appropriate hyperplane.
Note: C must be strictly positive as mentioned insklearn.svm.SVC
This is my first article and I believe there is always scope for improvement. Please feel free to share your thoughts!
Image credits:https://amitranga.files.wordpress.com/2014/03/image44.png
