Naïve Bayes Classifier with Practical Implementation

In this chapter, we will discuss Naïve Bayes Classifier which is used for classification problem and it’s supervised machine learning algorithm.

This chapter spans in three parts:

1. What is Naïve Bayes?
2. How does Naïve Bayes work?
3. Practical Implementation of Naïve Bayes in Scikit Learn.

1. What is Naïve Bayes?

Bayes theorem provides a way of calculating the posterior probability, P(c|x), from P(c), P(x), and P(x|c). Naive Bayes classifier assumes that the effect of the value of a predictor (x) on a given class © is independent of the values of other predictors. This assumption is called class conditional independence.

Naive Bayes Equation

· P(c|x) is the posterior probability of class (target) given predictor(attribute).

· P(c) is the prior probability of class.

· P(x|c) is the likelihood which is the probability of predictor given class.

· P(x) is the prior probability of predictor.

2. How does Naïve Bayes work?

This dataset has 14 instances and five numbers of attributes. Here first four attributes are predictor and the last attribute is the target attribute.

Solution:

First, we find the Probability for yes and no in the target attribute which is play.

P(C) or P(Class)

For Yes: Total 14 and 9 Yes

For No: Total 14 and 5 No

Now I can also calculate the individual property for Outlook, Temperature, Humidity, Wind, Play on based on ability.

Outlook:

In the first attribute which is outlook we have three categorical values which are (sunny, overcast, rainy ) and our target is that we find each of the probability for Sunny, Overcast and Rainy based on target attribute which is play (yes, no).

Temperature:

In the second attribute which is Temperature we have three categorical values which are (Hot, mild, cool) and our target is that we find each of the probability for Hot, mild and cool based on target attribute which is play (yes, no).

Humidity:

In the third attribute which is Humidity we have two categorical values which are ( High, Normal) and our target is that we find each of the probability for High, Normal based on target attribute which is play (yes, no).

Wind:

In the fourth attribute which is Wind we have two categorical values which are ( False, True) and our target is that we find each of the probability for False, True based on target attribute which is play (yes, no).

Play:

In the Last attribute which is predicted or Target, we have binary value (Yes, No) yes mean play and no mean not play and we find the probability for yes from the total instance which is 14 and no also.

Now I calculate each of the probability for Outlook, Temperature, Humidity, Wind, based on Target attribute which is Play.

Our Target:

We need to classify the following new instance

What is the Probability for Play (Yes or No)?

If probability yes then should be play Golf.

If probability no then should be watching movies.

We find this predicted result using the Naïve Bayes Model.

Based on yes probability:

P(X/C) P(C) or P(X/play= yes)P(play=yes)

Based on No Probability

P(X/C) P(C) or P(X/play= yes)P(play=yes)

Now Find Evidence:

So Finally

For yes (Should play a Golf)

For no (Should watch movies)

In the End

0.9421 > 0.2424

So the probability for no is highest as compare to yes.

Our predicted result is

So in this way Naïve Bayes work.

Note: If you want this article check out my academia.edu profile.

3. Practical Implementation of Naïve Bayes in Scikit Learn?

Dataset Description:

This Dataset has 400 instances and 5 attributes which is a User ID, Gender, Age, Estimate Salary and last is Purchased which Target attributes. First Four Attribute which is ID, Gender, Age, Estimate Salary is predictor and the last attribute is the target attribute.

Part 1: Data Preprocessing:

1.1 import the Libraries

In this step, we import three Libraries in Data Preprocessing part. A library is a tool that you can use to make a specific job. First of all, we import the numpy library used for multidimensional array then import the pandas library used to import the dataset and in last we import matplotlib library used for plotting the graph.

1.2 Import the dataset

In this step, we import the dataset to do that we use the pandas library. After import our dataset we define our Predictor and target attribute. Our Predictor attributes are a User ID, Gender, Age and Estimated Salary as you can see in the sample dataset which we call ‘X’ here and Purchased is a target attribute which we call ‘y’ here.

1.3 Split the dataset for test and train

In this step, we split our dataset into a test set and train set and a 75% dataset split for training and the remaining 25% for tests.

1.4 Feature Scaling

Feature Scaling is the most important part of data preprocessing. If we see our dataset then some attribute contains information in Numeric value some value very high and some are very low if we see the age and estimated salary. This will cause some issues in our machinery model to solve that problem we set all values on the same scale there are two methods to solve that problem first one is Normalize and Second is Standard Scaler.

Here we use standard Scaler import from Sklearn Library.

Part 2: Building the Naïve Bayes model:

In this part, we import our Naïve Bayes Classifier model using Scikit Learn Library.

2.1 Import the Libraries

In this step, we are building our Naïve Bayes model to do this first we import a Naïve Bayes model from Scikit Learn Library.

2.2 Initialize our Naïve Bayes model

In this step, we initialize our Naïve Bayes model.

2.3 Fitting the Naïve Bayes Model

In this step, we fit the training data into our model X_train, y_train is our training data.

Part 3: Making the Prediction and Visualizing the result:

In this Part, we make a prediction of our test set dataset and visualizing the result using the matplotlib library.

3.1 Predict the test set Result

In this step, we predict our test set result.

3.2 Confusion Metrics

In this step we make a confusion metric of our test set result to do that we import confusion matrix from sklearn.metrics then in confusion matrix, we pass two parameters first is y_test which is the actual test set result and second is y_pred which predicted result.

3.3 Accuracy Score

In this step, we calculate the accuracy score based on the actual test result and predict test results.

3.4 Visualize our Test Set Result

In the step, we visualize our test set result to do this we use a matplotlib library and we can see only 10 points are the incorrect map and the remaining 110 are the correct map in the graph according to the model test set result.

If you want dataset and code you also check my Github Profile.

End Notes

If you liked this article, be sure to click ❤ below to recommend it and if you have any questions, leave a comment and I will do my best to answer.

For being more aware of the world of machine learning, follow me. It’s the best way to find out when I write more articles like this.

You can also follow me on Github for code & dataset follow on Aacademia.edu for this article, Twitter and Email me directly or find me on LinkedIn. I’d love to hear from you.

That’s all folks, Have a nice day :)