CODEX

Naïve Bayes Classifier

Probabilistic machine learning model

A Naive Bayes classifier belongs to family of probabilistic machine learning model which is used for classification task. The groundwork of this classifier is based on the Bayes theorem.

So lets see what is Bayes Theorem

Bayes Theorem

Following formula depicts Bayes Theorem where we find probability of A given B has occurred.

Journey from Bayes Theorem To Naive Bayes

This is the formula for Bayes Theorem where X is a vector of features

Now here we make an assumption an assumption that occurrence of feature is totally independent from occurrence of other features. There will be no influence of one on other. That’s why it is called naive. It means if we have 2 features [‘good’ , ‘awesome’] and we seen probability of ‘good’ , you can say nothing about probability of ‘awesome’.

Another assumption here is that all the predictors (features) have an equal effect on the outcome.

Now expanding our X and applying Chain rule we get

We can see the denominator (the normalization factor ) is inert, so we can remove it and rewrite our equation as

Predictions will be made by judging which class gives us the maximum probability-

Types of Naive Bayes Classifiers

• Gaussian Naive Bayes:In Gaussian Naive Bayes, continuous values associated with each feature are assumed to be distributed according to a Gaussian distribution. It gives us bell-shaped curve.

Formula for finding probability-

• Multinomial Naive Bayes: This is mostly used for document classification problem, i.e whether a document belongs to the category of sports, politics, technology etc. The features/predictors used by the classifier are the frequency of the words present in the document.
• Multivariate Bernoulli Naive Bayes: This is similar to the multinomial naive bayes but the predictors are boolean variables. For instance if a word occurs in the text or not.

Naive Bayes Theorem is fast but due to its assumption that every predictor is independent it doesn’t fits with real world problems many times.

For complete implementation refer to following GitHub link

ads-22/ML-Practice

Dataset Link- https://www.kaggle.com/uciml/mushroom-classification