Intuition behind Covariance
We have studied various terms like mean, median, mode, co-variance etc in our schools, mugged up their formulas. And we are doing the pretty same thing today when we want to become ML engineer or Data Scientist.
Knowing just the formulae isn’t enough, if that was the case then what is the difference between you and a school going child.
Ever wondered why co-variance is positive when variables vary in the same direction and negative when they vary in negative direction.
We all just know the formula so we and the school going child are at the same level. This hit me hard and then I decided to get an intuition about things and stop mugging up just their formulas.
I guarantee you that by the end of this article you would be confident enough to say that “Yes there’s a difference between me and school going child because the child only knows the formula and I know the intuition behind the formula”
Let’s get started.
So we all know that mean is nothing but the average value of the data. Variance is the measure of the spread of the data, meaning how far/near are our points from the mean.
Let’s have a look at their formulas:
1. Mean
This formula is self-explanatory, it is the sum of values divided by the number of values.
2. Variance
As the definition says the formula calculates on an average, how far the data points are from the mean. The subtraction term is squared to avoid nullification of +ve and -ve values.
Some might think why don’t we take the absolute value. Taking the absolute value does not capture the spread correctly. I suggest you to try some examples with both absolute value and squared value, you yourself will find the answer. Or if you are still not convinced read this.
Now coming to the most important topic, for which this article is meant.
3. Co-variance
The word Co-variance has the term VARIANCE in it , so it has to be related to Variance or else they could have named it anything like Shahrukh Khan.
Keeping that thing in mind (Variance and not SRK) let’s have a look at its formula.
This formula looks similar to formula for variance. If we replace y by x then it would become the formula for Variance.
Hence the term Variance in Co-variance and not Shahrukh Khan.
Co-variance also has the term CO meaning pair. This is similar to the term co-passenger, co-rider similarly co-variance meaning variance between two entities.
Now we have also heard that if both x and y change in the same direction then co-variance is positive (+ve), if they vary in the opposite direction then co-variance is negative (-ve).
Let’s try to find out the reason behind this.
Intuition
Let’s consider an example where both x and y change in the same direction, say both are increasing.
Here we are assuming both variables are increasing.
Red data point is the mean, the brown data point is below the mean, and green is above the mean. We want to cover both scenarios where the point is above and below the mean.
We will divide the co-variance formula into 2 parts. Part one is the first () bracket related to x and second part is the second bracket related to y.
This makes us understand why co-variance is positive if both variables are increasing. Similarly it can be proved if both variables are decreasing.
If x and y both change in the opposite direction
Here we are assuming that x increases and y decreases.
Red data point is the mean, the brown data point is below the mean, and green is above the mean.
As we have done above, similarly we will divide the co-variance equation into two parts.
Similarly we have understood why the co-variance is negative when variables vary in opposite direction.
Now we can say that we are different from that school going child because we have some intuition behind why things are they way they are.
But still, there are pretty many things we have accepted without asking WHY?
Why do we need correlation if co-variance can be used to check whether or not 2 variables are related?
The reason for that is co-variance only gives us the direction of whether the variables vary in the same direction or opposite direction. But to Quantify this, correlation comes to our rescue.
I would be coming up with many such intuitive articles so Stay Tuned!
Feel free to drop comments or questions below, you can find me on Linkedin. Suggestions on what should I write next are welcomed.
