What is the Significance of C value in Support Vector Machine?

Hello,

Today, I am covering a simple answer to a complicated question that is “what C represents in Support Vector Machine” Here is just the overview, I explained it in detail in part 1 of Support Vector Machine Series.

Note: Here I am assuming that you know the basic fundamentals of SVM.

Fundamental under the hood:

As we know, in Support Vector Machine we always look for 2 things:

1. Setting a larger margin
2. lowering misclassification rate(how much a model misqualifies a data)

Now the problem is above said 2 things are kind of contradictory. If we increase margin, we will end up getting a high misclassfication rate on the other hand if we decrease a margin, we will end up getting a lower misclassification rate.

You must be thinking then why we want a larger margin, our priority should be getting a lower misclassfication rate then let me tell you above quoted things are for training dataset. Lower misclassification on training dataset doesn’t mean lower misclassification on validation/testing data. To get a better a result of testing data, SVM looks for a higher margin.

So I finally confused you in how to set up this 2 contradictory things now? The answer is parameter C.

Paramerter C:

Large Value of parameter C => small margin

Small Value of paramerter C => Large margin

Diagram below will give what exactly I am trying to say

Change in margin with change in C

How should you choose the value of C?

There is no rule of thumb to choose a C value, it totally depends on your testing data. The only option I see is trying bunch of different values and choose the value which gives you lowest misclassification rate on testing data. I would suggest you to use gridsearchCV, in which you can directly give a list of different values parameter and it will tell you which value is best.

Optional(In more depth)

What exactly happening under the hood?

Visit highlighted portion in this blog

End Notes:

I hope this blog solved your doubt Please upvote. Enjoy Machine Learning.

Thanks,

Pushkar Mandot

Source: Images here comes from google images