ML-Bias/Variance

Diagnosing bias vs variance

We will continue to look at some methodologies to diagnose our hypothesis function. This time, it will be about bias and variance.

Bias and Variance

So what does bias and variance represent?

Bias represents the distance of our predicted values with the original values.

Variance represents the distance of the predicted values, which means how much the predicted values are scattered from each other.

Let’s see this intuitively through a picture with my explanations.

Errors due to the bias means that the model is too simple to represent a certain situation, it cannot catch the details. If the bias is high, it will cause underfitting. This means that our model is missing something important.

Errors due to the variance means that the model is not generalized enough to apply to other situations or datasets other than our training set. If the model has high variance, this means that the model is overfitted.

High bias & low variance, Good compromise, Low bias & high variance

Bias & Variance trade-off

Bias and variances have a trade-off relationship.

If our model gets simple, the bias would get higher, and the variance would get lower.

If our model gets complicated, the bias would get lower, and the variance would get higher.

So we must find the appropriate point where the bias and the variance values are well compromised with each other. We cannot just lower both at once.