Naive Bayesian classification __ AI

Today, I learned about the naive Bayesian classification , and next, I'll cover the principles and applications in text categorization .

Contents

1. Definition of classification problem

2. Bayes theorem

3. Bayes Classification Principle

4. Conditional probability and Laplace calibration of feature attribute Division

5. Bayes Text Classification Example

1. Definition of classification problem

Known sets and sets, determines the mapping rules so that arbitrary, there

And there is only one that makes it true. It's called a collection of categories , and each element is a category; it's called a collection of items , each

An element is an item to be sorted, called a classifier . The task of the classification algorithm is to construct the classifier.

2. Bayes theorem

Naive Bayesian classification is based on the Bayesian theorem , and the theorem is expressed as follows

This formula is proved as follows, because

Joint two-way elimination, get

Bayes theorem for us to get through the road from the getting.

3. Bayes Classification Principle

The principle of Naive Bayes classification is this: for a given term, the probability of the occurrence of each category under the condition of this item is solved, which is the largest

Consider the category to which this item is categorized.

The formal definition of naive Bayesian classification is as follows

(1) set to a category item, which is a feature attribute, which has a feature attribute altogether.

(2) the collection of categories is

(3) calculated separately

(4) if, then.

It can be seen that the most critical question is how to find each conditional probability in (3) . On this critical issue, you can do this

(1) to find a set of sorted items known to be categorized, which is called the Training sample set .

(2) the conditional probability estimation of each characteristic attribute under each category is counted. That gets

(3) if each characteristic attribute is condition independent , then according to the Bayes theorem has the following derivation

The denominator is constant for all categories, only the maximum of the molecule can be considered, and because each feature attribute is conditionally Independent,

Therefore, the simple Bayesian classification process can be described in the following diagram

4. Conditional probability and Laplace calibration of feature attribute Division

Now let's discuss how to calculate

When the feature attribute is a discrete value , the frequency of each attribute in the training sample can be estimated as long as it appears in each category.

What happens when the feature attribute is continuous ? When the attribute is continuous, it is assumed that its value obeys the Gaussian distribution , i.e.

Then, as long as the average and standard deviation of the features in the training samples is calculated, the estimated value of the conditional probability can be obtained with the upper formula.

For example, the feature attribute is height or weight, and so on, it is not simple to count the number of occurrences to estimate the probability. The mean and standard deviations need to be computed to estimate the value.

Next, consider what to do. This occurs because a feature attribute is not present under a category, and the

Appearance can cause the quality of the classifier to be greatly reduced. To solve this problem, a Laplace calibration is introduced, and the core idea is that if a category

The next feature property does not appear, then let its counter add 1, that is, let it appear once. This avoids the situation.

5. Bayes Text Classification Example

There are many examples of naive Bayesian classification, such as text categorization, detecting unreal accounts in SNS community, predicting sex according to human characteristics and so on.

Before you categorize text, you need to select the feature attributes of the text, which is critical, as you will see the text before the feature attribute selection reference

Chapter "text feature attribute Selection".

In naive Bayesian text classification problem, with two important models, namely polynomial model and Bernoulli model . In fact, these two models are not advanced, only

is to calculate the priori probability and the conditional probability difference, the polynomial model will count the word frequency, and the Bernoulli model is only labeled as 1for the appearing words, without

has been labeled as 0, you can see that one is based on the word frequency, one is based on document frequencies . When calculating the conditional probability, when the text in the category

When a word does not appear in the thesaurus, the probability is zero, which can cause serious problems, which need to consider the Laplace smoothing and add 1times to the occurrence of all the words. Again

For statistics. If the probability is too small and the number of words is too large to double, you can take the logarithm and change the multiplication to an additive calculation.

Of course, in naive Bayesian classification, if the feature attribute is a continuous value, such as height, then the Gaussian distribution to deal with. On the whole, the plain

Bayesian algorithm is relatively good simple, so do not use C + + implementation.