6 Easy Steps to learn Naive Bayes algorithm (with code in Python)

6 Easy Steps to learn Naive Bayes algorithm (with code in Python) Introduction

Here's a situation you ' ve got into:

You is working on a classification problem and you have generated your set of hypothesis, created features and discussed The importance of variables. Within an hour, stakeholders want to see the first cut of the model.

What'll do? You are hunderds of thousands of data points and quite a few variables in your training data set. In such situation, if I were at your place, I would has used 'Naive Bayes', which can extremely fast relative to other classification algorithms. It works on Bayes theorem of probability to predict the class of unknown data set.

In this article, I'll explain the basics of this algorithm, so that next time if you come across large data sets, you CA n bring this algorithm to action. In addition, if you is a newbie in Python, you should is overwhelmed by the presence of available in this codes.

Table of Contents

1. What is Naive Bayes algorithm?
2. How Naive Bayes algorithms works?
3. What is the Pros and Cons of using Naive Bayes?
4. 4 Applications of Naive Bayes algorithm
5. Steps to build a basic Naive Bayes Model in Python
6. Tips to improve the power of Naive Bayes Model

What is Naive Bayes algorithm?

It is a classification technique based on the Bayes ' theorem with an assumption of independence among predictors. In simple terms, a Naive Bayes classifier assumes so the presence of a particular feature in a class are unrelated to the Presence of any other feature. For example, a fruit could be considered to being an apple if it was red, round, and about 3 inches in diameter. Even if these features depend on each other or upon the existence of the other features, all of these properties Independe ntly contribute to the probability, this fruit is a apple and that's why it's known as ' Naive '.

Naive Bayes model is easy-to-build and particularly useful for very large data sets. Along with simplicity, Naive Bayes are known to outperform even highly sophisticated classification methods.

Bayes theorem provides a by calculating posterior probability P (c|x) from P (c), p (x) and P (x|c). Look at the equation below:

Above,

• P (c|x) is the posterior probability of class (c, target) given Predictor (x, attrib Utes).
• P (c) is the prior probability of class.
• P (x|c) is the likelihood which is the probability of Predictor given class.
• P (x) is the prior probability of predictor.

How Naive Bayes algorithm works?

Let's understand it using an example. Below I have a training data set of weather and corresponding target variable ' Play ' (suggesting possibilities of playing) . Now, we need to classify whether players would play or not based on weather condition. Let's follow the below steps to perform it.

Step 1:convert The data set into a frequency table

Step 2:create Likelihood table by finding the probabilities like overcast probability = 0.29 and probability of playing I S 0.64.

Step 3:now, use Naive Bayesian equation to calculate the posterior probability for each class. The class with the highest posterior probability is the outcome of prediction.

problem: Players would play if weather is sunny. Is this statement is correct?

We can solve it using above discussed method of posterior probability.

P (Yes | Sunny) = P (Sunny | YES) * P (yes)/P (Sunny)

Here we have P (Sunny | Yes) = 3/9 = 0.33, p (Sunny) = 5/14 = 0.36, p (yes) = 9/14 = 0.64

Now, P (Yes | Sunny) = 0.33 * 0.64/0.36 = 0.60, which has higher probability.

Naive Bayes uses a similar method to predict the probability of different class based on various attributes. This algorithm was mostly used in the text classification and with problems have multiple classes.

What is the Pros and Cons of Naive Bayes?

Pros:

• It is easy and fast to predict class of test data set. It also perform well in multi class prediction
• When assumption of independence holds, a Naive Bayes classifier performs better compare to other models like logistic Regr Ession and you need less training data.
• It perform well in case of categorical input variables compared to numerical variable (s). For numerical variable, normal distribution are assumed (bell curve, which is a strong assumption).

Cons:

• IF categorical variable have a category (in test data set), which is not observed in training data set, then model would as Sign a 0 (zero) probability and'll is unable to make a prediction. This is often known as "Zero Frequency". To solve this, we can use the smoothing technique. One of the simplest smoothing techniques is called Laplace estimation.
• On the other side naive Bayes was also known as a bad estimator, so the probability outputs fromPredict_proba was Not to is taken too seriously.
• Another limitation of Naive Bayes is the assumption of independent predictors. In real life, it's almost impossible that we get a set of predictors which be completely independent.

4 Applications of Naive Bayes algorithms

• Real Time Prediction: Naive Bayes is a eager learning classifier and it is sure fast. Thus, it could is used for making predictions in real time.
• Multi class Prediction: This algorithm are also well known for multi class prediction feature. Here we can predict the probability of multiple classes of target variable.
• Text classification/spam filtering/sentiment Analysis: Naive Bayes classifiers mostly used in text classification (due to better result in multi class problems and independence Rule) has higher success rate as compared to other algorithms. As a result, it is widely used in Spam filtering (identify Spam e-mail) and sentiment analysis (in social media analysis, To identify positive and negative customer sentiments)
• recommendation System: Naive Bayes Classifier and collaborative Filtering together builds a recommendation System, uses machine learning and Data mining techniques to filter unseen information and predict whether a user would like a given resource or not

How to build a basic model using the Naive Bayes in Python?

Again, Scikit learn (Python library) would help the here to build a Naive the Bayes model in Python. There is three types of Naive Bayes model under Scikit Learn library:

• gaussian:  It is used-classification and it assumes that features follow a Norma L Distribution.

• multinomial:  It is used for discrete counts. For example, let ' s say,  we has a text classification problem. Here we can consider bernoulli trials which are one step further and instead of "word occurring in the document", we H Ave "Count how often word occurs in the document", you can think of it as "number of times outcome number x_i is observed Over the N trials ".

• bernoulli:  The binomial model is useful if your feature vectors be binary (i.e. zeros and on ES). One application would be text classification with ' bag of words ' model where the 1s & 0s is "word occurs in the Docum Ent "and" Word does not occur in the document "respectively.

Based on your data set, you can choose any of the above discussed model. Below is the example of Gaussian model.

Python Code

#Import Library of Gaussian Naive Bayes modelnp.  Array([[ -3,7],[1,5], [up], [ -2,0], [2,3], [ -4,0], [ -1,1], [up], [ -2,2], [2,7], [ -4,1], [ -2,7]])NP.  Array([3, 3, 3, 3, 4, 3, 3, 4, 3, 4, 4, 4])        

  #Create a Gaussian classifiermodel =  Gaussiannb () # Train the model using the training sets modelfit (Xy) # Predict Output predicted= model. Predict ([[1,2],[3,4]]) print predicted  Output: ([3,4])    

Above, we looked at the basic Naive Bayes model, you can improve the power of this basic model by tuning parameters and ha Ndle assumption intelligently. Let's look at the methods to improve the performance of the Naive Bayes Model. I ' d recommend you to go through this document for more details on Text classification using Naive Bayes.

Tips to improve the power of Naive Bayes Model

Here is some tips for improving power of Naive Bayes Model:

• If continuous features does not have the normal distribution, we should use transformation or different methods to convert it in Normal distribution.
• IF test Data set has zero frequency issue, apply smoothing techniques "Laplace Correction" to predict the class of Test da Ta set.
• Remove correlated features, as the highly correlated features is voted twice in the model and it can leads to over Inflati ng importance.
• Naive Bayes classifiers have limited options for parameter tuning like Alpha=1 for smoothing, fit_prior=[true| False] to learn class prior probabilities or not and some other options (look at detail here). I would recommend to focus on your pre-processing of data and the feature selection.
• Might think to apply some classifier combination technique like ensembling, bagging and boosting but these me Thods would not help. Actually, "ensembling, boosting, bagging" won ' t help since their purpose are to reduce variance. Naive Bayes have no variance to minimize.

End Notes

In this article, we looked on one of the supervised machine learning algorithm "Naive Bayes" mainly used for Classificatio N. Congrats, if you've thoroughly & understood this article, you ' ve already taken your first step to master this Algori tHM. Need is practice.

Further, I would suggest to focus more on data pre-processing and feature selection prior to applying Naive Bayes Algo Rithm.0 in the future post, I'll discuss about the text and document classification using Naive Bayes in more detail.

Did you find this article helpful? Please share your opinions/thoughts in the comments section below.

6 Easy Steps to learn Naive Bayes algorithm (with code in Python)