Bayesian classification algorithm of Scikit-learn Learning

Copyright NOTICE: <--Carefully write your every article, reprint please indicate the source @http://blog.csdn.net/gamer_gyt <--

Directory (?) [+]

======================================================================

This series of blogs mainly refer to the Scikit-learn official website for each algorithm, and to do some translation, if there are errors, please correct me

Reprint please indicate the source, thank you

======================================================================

In addition, the naive Bayesian classification algorithm in "machine learning combat" has deeply interpreted the principle of naive Bayesian classification algorithm, and the Python implementation based on naive Bayesian classification algorithm is mainly used to implement Bayesian classification in Python, and a simple text classifier model is constructed. Next, we will focus on some explanations of Bayesian algorithm in Scikit-learn

1: Naive Bayes algorithm description

2: Gaussian naive Bayes (Gaussian Naive Bayes)

3: Polynomial Naive Bayes (multinomial Naive Bayes)

4: Bernoulli naive Bayes (Bernoulli Naive Bayes)

One: Naive Bayesian algorithm description

Naive Bayes algorithm is a supervised learning classification algorithm based on the independent of each eigenvalue, and this is also called "naive" Bayesian reason, in the real environment, it is difficult to achieve the absolute mutual independence between two eigenvalues. Given a class variable Y and a dependent eigenvector x_1 through X_n, the state of the Bayes theorem follows the relationship:

Assuming that two eigenvalues are independent of each other

For all I values, this relationship is simplified to

Since P (x_1, ..., x_n) is a constant given input, we can use the following classification rules (the former proportional to the latter):

We can use the maximum posterior probability (MAP) to estimate P (Y) and P (xi|y), and P (y) is the probability of the occurrence of Y in the training set

The different naive Bayes classification algorithms are because they make different assumptions about P (xi|y).

Although the naïve Bayes hypothesis is simplistic, he has shown quite good results in existing applications such as document classification and Spam classification (for theoretical reasons, why naive Bayes works well, and the type of data he is suitable for processing, please continue reading)

Compared to other more advanced methods, naive Bayesian algorithms are more efficient at learning and classifying, and the independent distribution of each class of conditional features means that each class distribution can be independently estimated as a one-dimensional distribution, which in turn helps mitigate the hassle of data dimensionality reduction

On the other hand, although the naive Bayes classification is summed up as an efficient classifier, he has a bad estimate because the predictions for his output cannot be taken seriously.

The different Bayesian classification algorithms are mentioned above because they make different assumptions about P (xi|y), so let's take a look at the common p (xi|y) hypothesis and the implementation method in Scikit-learn.

Two: Gaussian naive Bayes

GaussianNBInheriting Gaussian naive Bayes, the characteristic probability is assumed to be Gauss:

The code examples are as follows:

[Python]View PlainCopy

2. #高斯朴素贝叶斯
4. Import NumPy as NP
6. X = Np.array ([-1,-1], [-2,-1], [-3,-2], [1, 1], [2, 1] , [3, 2]])
8. Y = Np.array ([1, 1, 1, 2, 2, 2])
10. From Sklearn.naive_bayes import gaussiannb
12. CLF = GAUSSIANNB (). Fit (X, Y)
14. Print Clf.predict ([[-0.8,-1]])
18. "' "
20. Partial_fit Description: Incremental training of a batch of samples
22. This method is known to be used several times in different datasets to achieve core and online learning, which is particularly useful when datasets are large and are not suitable for in-memory operations
24. This method has some performance and numerical stability overhead, so it is best to act on as large a chunk as possible (as long as the budgeted cost of memory is met)
26. ‘‘‘
28. CLF_PF = GAUSSIANNB (). Partial_fit (X, y, Np.unique (y))
30. Print Clf_pf.predict ([[-0.8,-1]])

The output is:

For more information on Gausbeyes distribution please refer to: Click to read

Three: Polynomial distribution

MultinomialNBThe Bayesian algorithm that implements the multinomially distribution data is a VARIANT used in the classic naive Bayesian text classification (where the data is usually expressed as the number of word vectors, although the TF-IDF vector behaves well in the actual project), for each y, the distribution is parameterized by vectors, n is the number of categories (in the text category, which represents the length of the vocabulary) indicating the probability that the sample appearing in label I belongs to category Y

The parameter is a smoothed maximum likelihood estimate, which is the relative frequency count:

Indicates the number of label I belonging to category y in the sample set T

Represents the number of category Y occurrences in all labels

A smoothing prior a >=0 represents a feature that does not exist in the learning sample and prevents a probability of 0 in the calculation, and the setting of Alpha = 1 is called Laplace smoothing, when α<1 is called Lidstone smoothing

The code examples are as follows:

[Python]View PlainCopy

1. #多项式分布   
2. import numpy as np  
3. x =  Np.random.randint (5, size= (6, 100))   
4. y = np.array ([1,  2, 3, 4, 5,  6])   
5. from sklearn.naive_bayes import multinomialnb  
6. CLF&NBSP;=&NBSP;MULTINOMIALNB (). Fit (x, y)   
7. print clf.predict (x[2:3])   

Output = [3]

For more information on polynomial distribution please refer to: Click to read

Four: Bernoulli naive Bayes

BernoulliNBThe implementation of naive Bayesian training and classification algorithm is based on multivariate Bernoulli distribution data; For example, there may be multiple characteristics, but each one is assumed to be a binary value (Bernoulli, Boolean) variable. Thus, samples of such requirements are represented as eigenvectors of binary values, and if given any other type of data, an BERNOULLINB instance can be entered (depending on the two value parameter)

On the basis of Bernoulli's naive Bayesian decision-making rules

In the case of text categorization, the occurrence vector of a word (rather than the word count vector) can be used to train and use the classification. BERNOULLINB may perform better on some datasets, especially those with short files. If time permits, it is advisable to evaluate the two models.

The sample code is as follows:

[Python]View PlainCopy

2. #伯努利分布
4. Import NumPy as NP
6. X = Np.random.randint (2, size= (6, + ))
8. Y = Np.array ([1, 2, 3, 4, 4, 5])
10. From Sklearn.naive_bayes import bernoullinb
12. CLF = BERNOULLINB ()
14. Clf.fit (X, Y)
16. BERNOULLINB (alpha=1.0, binarize=0.0, class_prior=None, fit_prior=True)
18. Print (Clf.predict (x[2:3))

The output is [3]

More about Bernoulli Naive Bayes please refer to: Click to read

Bayesian classification algorithm of Scikit-learn Learning