Naive Bayes & its Mathematical Implementation

Let’s continue our learning

In this blog we will cover the following topics:-

• What is Bayes Theorem?
• What is Naive Bayes algorithm?
• Mathematical Implementation of Naive Bayes algorithm or How this algorithm works.
• Types of Naive Bayes Classifier.
• Advantages and Disadvantages of this algorithm.

Before doing deep dive in Naive Bayes Algorithm , first understand the significance of naive and bayes .

The algorithm is called “Naive” because it assumes that every pair of features being classified is independent of each other which is not possible in real world .

This algorithm is given by statistician Thomas Bayes and the theorem named after him, “Bayes’ theorem”.

What is Bayes theorem?

Bayes’ Theorem is a way of finding a probability when we know certain other probabilities.

The formula is:

Bayes Theorem formula

where:

P(A)= The probability of A occurring

P(B)= The probability of B occurring

P(A∣B)=The probability of A given B

P(B∣A)= The probability of B given A

What is Naive Bayes algorithm?

Naive Bayes is a classification algorithm for binary (two-class) and multi-class classification problems based on Bayes’ Theorem with an assumption of independence among predictors. In simple terms, a Naive Bayes classifier assumes that the presence of a particular feature in a class is unrelated to the presence of any other feature.

How Naive Bayes algorithm works? or Mathematical Implementation of Naive Bayes algorithm.

Let’s understand the working of this algorithm from the given example:

Imagine 100 people at a party, and you tally how many wear pink or not, and if a man or not, and get these numbers:

Let us do some totals:

And calculate some probabilities:

• the probability of being a man is P(Man) = 40/100 = 0.4
• the probability of wearing pink is P(Pink) = 25/100 = 0.25
• the probability that a man wears pink is P(Pink|Man) = 5/40 = 0.125
• the probability that a person wearing pink is a man P(Man|Pink) =?

P(Man|Pink) = (P(Man). P(Pink|Man))/P(Pink)

P(Man|Pink) = (0.4 × 0.125)/0.25 = 0.2

Therefore , the probability that a person wearing pink is a man is 20%.In this way Naive Bayes algorithm works.

Types of Naive Bayes Classifier:

• Multinomial Naive Bayes:

This is mostly used for document classification problem, i.e whether a document belongs to the category of sports, politics, technology etc. The features/predictors used by the classifier are the frequency of the words present in the document.

• Bernoulli Naive Bayes:

This is similar to the multinomial naive bayes but the predictors are boolean variables. The parameters that we use to predict the class variable take up only values yes or no, for example if a word occurs in the text or not.

• Gaussian Naive Bayes:

When the predictors take up a continuous value and are not discrete, we assume that these values are sampled from a gaussian distribution.

Advantages and Disadvantages of Naive Bayes:

Advantages-

• Naive Bayes algorithms are mostly used in sentiment analysis, spam filtering, recommendation systems etc.
• They are fast and easy to implement
• It is easy and fast to predict class of test data set. It also perform well in multi class prediction.

Disadvantages-

• It’s biggest disadvantage is that the requirement of predictors to be independent. In most of the real life cases, the predictors are dependent, this hinders the performance of the classifier.

Here comes the end of this blog.

THANKS FOR YOUR VALUABLE TIME