An introduction to Markov’s and Chebyshev’s Inequality.

It’s normal as a living creature to encounter inequalities, one might overpower another, one might outwit another, there are numerous examples to explain so. Well, guess what, there are inequalities in data too. Although, these are a little different than the ones we are used to. This is a gentle introduction to two of those inequalities.

Let’s consider an event of going to a store to buy chocolates, there are “n” number of stores that I can go to and for each store, I can have two possibilities:
1. There is chocolate available.
2. There is no chocolate available.

In probability theory, the probability of any event occurring can be mapped to the range of [0,1]. So, if I say that the probability that I visit a store that has chocolates is 0.5, it means that out of all the shops I visit there is a 50% chance of chocolate being available on it and likewise a (1–0.5) or a 50% chance of chocolate not being available on it. We denote these events of going to a store by something called a random variable. In our case let’s consider it to be X.

Since I am visiting “n” number of stores and for some reason, if I have infinite stamina the number of stores I visit can range from {0 → infinity}, that means I can visit {0 shops, 1 shops, 2 shops, …, infinite shops}.

These probabilities are easy to calculate for a small value of “n” and can be modeled by using a Binomial Distribution. But as the value of “n” increases, it’s nicer to have a “bound” of these probabilities. For example, if I wanted to know what is the probability that if I visit 200 stores out of those 200 stores what is the probability that I will get chocolates on at least 145 stores, that would be P(X is greater than or equal to 145) which will come out to be 0.689, This means that there is at least 68.9% chance that I will get chocolate on 145 or more number of stores.

Markov’s Inequality
The example above was a demonstration of how we can use Markov’s Inequality to calculate certain “Bounds” on probabilities. Bounds can be thought of as barriers to the probabilities of certain events occurring.
Markov’s inequality can be calculated using the formula given below:

Markov’s Inequality.

“Bounds on Values our Random Variable takes”

Here, E[X] is the Expected value of our random variable X. Expected value can be described as a long-term average of a random variable. In other words what value would our random variable “tend” to if we repeat the experiment “n” number of times. In our chocolate example E[X] of a binomial random variable is given by the formula “np”, where “n” is the number of repeats and p is the probability of success. That will come out to be 100. The denominator here is the value on which we want to find our bound, in our case 145, which gives us the probability as 0.689.

Probability Bound of the chocolate example.

This is also called the “Upper Bound” on the probability which means that this event would occur with a probability of at most 68.9%.
Tho being very useful there are some downsides to this inequality too. Markov’s inequality cannot be applied if our random variable X takes a negative value. For example, if X denotes the difference of the numbers one would get but throwing a fair die twice, there would be cases when that turns out to be negative. To overcome this we define something called Chebyshev’s Inequality.

Chebyshev’s Inequality
The main idea behind Chebyshev’s inequality relies on the Expected value E[X] and the standard deviation SD[X]. The standard deviation is a measure of spread in statistics, it defines how much our data is “deviated” away from the mean value, in our case that mean value is E[X].

This inequality tells us that certain values in our data can be no more than “k” standard deviations away from our expected value E[X]. The expectation of X is also known as the mean value of our random variable. This, in a sense, gives us a symmetrical point around which X would be spread. This spread can be measured by the standard deviation and multiplying the standard deviation by “k” just gives us the bounds of how far can certain values lie from our expected value. Chebyshev’s Inequality can be calculated using the formula:

Chebyshev’s Inequality.

This inequality also defines some bounds that are applicable to a lot of probability distributions. It tells us that 100% of our data values would lie 1 standard deviation away from the mean, 25% of our data would lie 2 standard deviations away from the mean, and so on.

General Probability Bounds by Chebyshev’s Inequality.

Bounds in Chebyshev’s Inequality.

To demonstrate this let’s go back to our chocolate example. Let’s say we wanted to know that what will be the upper bound on my probability if we visit at least 90 stores and at most 110 stores. This can be calculated using the method below. We subtract our mean value from both of our bounds. This gives us the value of k*SD[X] as 10. From that, we calculate the value of k and put it on the right-hand side of the inequality.

Using Chebyshev’s Inequality to calculate upper bound.

Chebyshev’s Inequality can be applied to any probability distribution on which mean and variance are defined. This is of great help when we have no idea how to describe our random variable. This inequality helps us calculate upper and lower bounds by which those certain events can occur.