What’s so naive ‘bout Naive Bayes Classifier?

Naive, Yet it is one of the very simple yet powerful and easy to implement algorithm used in Supervised learning mainly for classification problems.

Through this blog, I intend to make y’all have a basic understanding of the Naive Bayes classifier and its applications and why is it called naive.

Introduction

Naive Bayes classifier is built upon three main fundamental theories which are Probability, Conditional Probability and Bayes theorem. I assume that my readers have knowledge of these if not you can find my blog on these over here.

I will just try to give an overview of what is Bayes theorem because that is important to understand Naive Bayes theorem.

I’m using the above image from Pathmind. Though the above image is self-explanatory, I will explain it in brief :

Here, P(A) and P(B) are probabilities of A and B given that they are independent of each other,which is why they are called as marginal probabilities. P(B|A) and P(A|B) are conditional probabilities.
P(A) is called Prior probability and P(B) is called Evidence.
P(B|A) is called Likelihood and P(A|B) is called Posterior probability.

Bayes Theorem in simple terms :
posterior = likelihood * prior / evidence

Naive Bayes classifier shows you the power of Bayes theorem in real-life data.
Naive Bayes classifier works by separating data into two different categories according to Bayes Theorem. It also assumes that all of the predictors are independent of each other. It predicts the probabilities of each group. The group with the highest probability is selected as the most likely group.

Consider an example of spam email filtering : In this case we are trying to figure out which email should be classified as spam and which one to be as normal email.
We first separate our data into normal and spam and use the keywords (Majorly used) in both the groups.Then based on the Bayes theorem we calculate he probability of each word in all the groups.

Steps Included :
1. Calculate the prior probability for separated groups.
2. Find Likelihood probability with each feature for every group.
3. With the help of these values we can and Bayes formula we can find out the posterior probability.
4. Choose the group with the higher probability.

Applications of Naive Bayes Classifier

1. Spam Filtering: Naive Bayes filters out spam using keywords. For example, in spam, you can see the word ‘money much more often than in regular mail. The algorithm must be trained to recognize such probabilities and, then, it can efficiently apply them for spam filtering.
2. Sentiment analysis: Based on what emotions the words in a text express, Naive Bayes can calculate the probability of it being positive or negative.
3. Document classification: Naive Bayes algorithm can help us to determine to which category a given document belongs. It can be used to classify texts into different languages, topics etc.
4. Real-time Predictions: Naive Bayes is fast and efficient which helps us to make real-time predictions.

Now let us understand why is Naive Bayes algorithm called naive.

Naive Bayes is naive because it makes a naive assumption on the predictor or input variables that they are independent of each other. It is a strong assumption but it is very unrealistic for real-life data. but helpul in solving larger and complex problems.
The one good thing about Naive Bayes is that the naive assumption actually tends to help the classification.

For those who want to see a video on exactly how Naive Bayes works, here is one from StatQuest:

I hope my blog gives you a basic understanding of Naïve Bayes algorithm.
Happy reading Y’all :)