What is Bayesian approach and the value it reveals for life?
In this blog, let’s see about what is Bayes theorem in statistics and what does that means. Bayesian is the one who is the follower of this statistical approach. Before this, we will see about its basics like conditional probability, type of events, etc that is needed to understand the Bayes theorem.
This image will make sense after knowing this theorem at the end of this blog.
As we all studied in our high school about probability. It tells the likelihood of a specific event can occur. This likelihood is called probability.
Eg: Probability(roll an even no in a six-faced die) = 3/6 = 1/2
As there are three possible outcomes for this event {2,4,6}
and it’s sample space is {1,2,3,4,5,6}
We saw what is probability. Now let’s move on to what is conditional probability.
Conditional probability
This tells the probability of an event that another event has already occurred (or) the probability of an event based on the previous event that occurred. It is denoted by P(B/A). It is called as B given A. Where A is the first event and B is the second event
Let’s see some example that explains this definition.
What is the probability of a second card from a deck is a queen that the first card taken is a king?
Here taking king is the first event so P(A) = 4/52
as there are 52 cards in a deck and 4 kings are viable.
The probability of a queen is denoted by P(B/A) as the event already happened. Now
P(B/A) = 4/51
Note that the sample space is 51 as one king is already drawn from the 52 cards. The queen has to take from the remaining 51 cards.
Independent and dependent events
In an independent event, the outcome of one event does not affect the outcome of the other event.
Eg: Rolling a die
when rolling a die gives 1, then the next roll of the die does not depend on the previous outcome (ie 1 here)
In a dependent event, the outcome of an event depends upon the outcome of another event. As the example, we saw in the conditional probability.
Eg: Choosing a queen that king is already drawn from the deck.
Here the probability of drawing queen depends upon the previous outcome of the event.
Consider this die rolling example,
first event- Probability of getting 1
second event — Probability of getting 2
P(A) = 1/6
P(B) = 1/6
Here P(B/A) also 1/6
therefore, P(B/A) = P(B)
In the second example of getting a queen after a king,
Probability of getting a king P(A) = 4/52
Probability of getting a queen P(B) = 4/ 52
Here, probability of getting a queen after a king
P(B/A) = 4/51
So, now P(B) ≠ P(B/A)
So, this tells that
when P(B) = P(B/A), the events A and B are independent
and
when P(B) ≠ P(B/A), the events A and B are dependent
Thus conditional probability is used to tell, whether the events are dependent or independent.
With this, we can say that the probability of two independent events occurring in the sequence is
P( A and B ) = P(A). P(B)
For two dependent events
P(A and B) = P(A).P(B/A)
Now, with these understandings of the basic probabilities, let’s move on to the main ingredient “Bayes Theorem”
Bayes Theorem
The Bayes theorem is defined as
This is used to find the probability of event A given that event B has occurred.
But, why on earth we are using this? Let’s see with an example
Consider an example of an author publishing a book. From the reader’s feedback, the author gets two kinds of feedback, one kind is the reader’s beliefs what the author says in the book, and the other group objects to the author’s contents. And the author got great feedback from their friends and family. So, he believes that his book is going to be his big success.
So, our work is to find the probability of the author’s content is true given that the feedback from the audience.
In statistical terms, let’s consider
the authors content is true as a hypothesis (H)
and the reader’s feedback as evidence (E),
we have to find P(H/E)
Consider this grey box contains the peoples who gave their feedbacks. It contains the peoples who believe the content and the ones who reject the content. It gives the P(E) ( which contains P(H) and P(~H).
The left vertical bar (includes pink and blue area) gives the P(H) (ie) peoples who are believing the authors content are true. The remaining area (green and grey) gives the people they are rejecting the content P(~H). We are considering a small portion from the box to find P(H/E) that his hypothesis is true based on the evidence.
The P(E/H) (blue area) takes the evidence of the people given that the hypothesis is true and P(E/~H) tells the evidence of the people given that the hypothesis is false (ie) it considers the people who don’t believe his content.
This tells the number of peoples who accepts the content by the considered or certain amount of people who gave the evidence (including accepted and rejected audience). If this formula gives 16.5 %, it tells around sixteen percent of peoples who likes author’s content.
In long run, it will be like
So, considering more area comes into the picture, the numbers will get updated. It may be increase or decrease according to the area which consists of the evidence that may or may not accept the content.
From these pictures, the formula will be constructed as
The numerator tells the amount of people who believes the hypothesis is true and the evidence given that that the hypothesis is true. The denominator considers the total amount of evidence that considers the hypothesis is true and false.
It can be simplified as
Thus we got the Bayes theorem from our example.🎉 🎉 🎉
P(H)- prior (tells the hypothesis is true before any evidence)
P(E/H)- likelihood( seeing the evidence that hypothesis is true)
P(E)- evidence(including the hypothesis is true and false)
P(H/E)- posterior (probability of the hypothesis given some evidence)
The term posterior means “coming after”. So, this formula tells the probability of the hypothesis after seeing the evidence.
Thus, the final result will tell the probability of the author’s hypothesis is true based on the evidence.
Now, when we modify this theorem a bit,
using A and B instead of H and E
P(B)P(A/B) = P(A)P(B/A)
When we consider the example we took in our dependent events (ie) probability of queen after getting a king
P(K) = 4/52 and P(Q/K) = 4/51
P(Q) = 4/52 (taking queen from deck first)
P(K/Q) = 4/51
therefore, this also proves the Bayes theorem
P(K)P(Q/K) = P(Q)P(K/Q)
So, the Bayes theorem is used to tell how one event depends on the other event. (ie) dependent events
In real life example, if a person is dependent on another one, the probability of one is going to school if the other is absent using Bayes theorem instead of using the probability of individuals for calculation. As Bayes theorem is using evidence rather than prior knowledge.
Life value from Bayes theorem
Ultimately what this Bayes theorem tells us is, we should not believe in the prior thoughts that we have. We should update our beliefs based on the learning we are taking in our life.
Now scroll back to the top, that doodle will make sense now. 😃
PS: Feedbacks are welcome!! And apologies for the bad diagrams
Cheers,
Bagavathy Priya N
Reference:
https://www.coursera.org/learn/bayesian-methods-in-machine-learning/
https://www.youtube.com/watch?v=HZGCoVF3YvM This is a gem
