Introduction :-
This theorem is provided by “Thomas Byes” (alternatively Bayes’ law or Bayes’ rule). This theorem describes the “Probability of an Event”. Thomas Byes also write an essay on probability. His friend “Richard Price” edited and published it after he died. One of the many applications of Bayes’ theorem is “Bayesian Inference”.
What dose “Bayesian Inference” mean ?
Inference = Educated guessing
Bayesian inference = Guessing in the style of Bayes
Let’s see it with live example…
Explanation :-
“Dilemma” at the movie. This person dropped their ticket in the hallway. Do you call out
“Excuse me, ma’am!”
or
“Excuse me, sir!”
You have to make a guess.
Bayesian inference is a way to capture common sense.
It helps you use what you know to make better guesses.
Put numbers to our dilemma
But there are 98 men and 2 women in line for the men’s restroom.
In the line, 4 times more men have long hair than women.
Translate this to Math…
P(something) = something / everything
P(woman) = Probability that a person is a woman
P(woman) = women / people
P(woman) = 50 / 100 = .5
P(man) = Probability that a person is a man
P(man) = men / people
P(man) = 50 / 100 = .5
Conditional Probabilities :-
P(long hair | woman)
If I know that a person is a woman, what is the probability that person has long hair?
P(long hair | woman) = women with long hair / women
P(long hair | woman) = 25 / 50 = .5
If I know that a person is a man, what is the probability that person has long hair?
P(long hair | man) = men with long hair / men
P(long hair | man) = 2 / 50 = .04
Whether in line or not.
P(A | B) is the probability of A, given B.
“If I know B is the case, what is the probability that A is also the case?”
P(A | B) is not the same as P(B | A).
P(cute | puppy) is not the same as P(puppy | cute)
If I know the thing I’m holding is a puppy, what is the probability that it is cute?
If I know the the thing I’m holding is cute, what is the probability that it is a puppy?
Joint Probabilities :-
What is the probability that a person is both a woman and has short hair?
P(woman with short hair) = P(woman) * P(short hair | woman)
P(woman with short hair) = .5 * .5 = .25
P(woman with long hair) = P(woman) * P(long hair | woman)
P(woman with long hair) = .5 * .5 = .25
P(man with short hair) = P(man) * P(short hair | man)
P(man with short hair) = .5 * .96 = .48
P(man with long hair) = P(man) * P(long hair | man)
P(man with long hair) = .5 * .04 = .02
P(A and B) is the probability that both A and B are the case.
Also written P(A, B) or P(A ∩ B)
P(A and B) is the same as P(B and A)
The probability that I am having a jelly donut with my milk is the same as the probability that I am having milk with my jelly donut.
P(donut and milk) = P(milk and donut)
What we really care about….
We know the person has long hair. Are they a man or a woman?
P(man | long hair)
We don’t know this answer yet.
Thomas Bayes noticed something cool
P(man with long hair) = P(long hair) * P(man | long hair)
P(long hair and man) = P(man) * P(long hair | man)
Because P(man and long hair) = P(long hair and man)
P(long hair) * P(man | long hair) = P(man) * P(long hair | man)
P(man | long hair) = P(man) * P(long hair | man) / P(long hair)
P(A | B) = P(B | A) * P(A) / P(B)
Bayes’ Theorem :-
P(A | B) = P(B | A) * P(A) / P(B)
Back to the movie theatre, this time with Bayes
P(man | long hair) = P(man) * P(long hair | man) / P(long hair)
P(man | long hair) = P(man) * P(long hair | man) / P(woman with long hair) + P(man with long hair)
P(man | long hair) = .5 * .04 /.25 + .02 = .02 / .27 = .07
This is the practical Working approach of Bayes Theorem.
