Generate learning algorithms, introduction to Naive Bayes

PART0 discriminant Learning Algorithm

Introduced: Two-dollar classification problem

Modeling: discriminant Learning Algorithm (discriminative learning algorithm) directly based on p (y|x) "That is, the classification result y under given feature X" model

The algorithm we used before (such as logistic regression) is the discriminant learning algorithm

PART1 Generation Learning Algorithm PART1.1 Definition

Introduction: OR two-dollar classification problem

Modeling: The Generative Learning Algorithm (generative Learning algorithm) is modeled on P (x|y) "As a feature X", p (y), given a specific class of Y, and then by the Bayes formula, p (y|x) is calculated:

, note where P (X) = P (x|y=1) *p (Y=1) + P (x|y=0) *p (y=0)

However, the actual solution is not used to ask P (x). Because the above can also be introduced this:

which represents the value of Y when P (y|x) takes the maximum value.

PART1.2 a chestnut: Gaussian discriminant analysis modelPART1.2.0 The concept of multivariate normal distribution

The multivariate positive distribution is no different from the normal, except that the parameter becomes the mean vector μ (mean vector) and the covariance matrix σ (Convariance matrix) .

PART1.2.1 GDA Model

In the GDA model, we modeled P (x|y) with a multivariate normal distribution:

, i.e.

Or the same as the original analysis method, the maximum likelihood-----log----to find the extremum. Finally have to

Notice the meaning of some symbols in this area:

Indicates that all of the X (i) and "1" of the classification result is 0, which can be understood as a indicator function, the expression in curly braces is true for a value of 1, otherwise 0 "

Total number of samples representing 1 of the classification result

This model is actually doing one thing:

For example, two loaves represent a normal distribution model for y=0 and Y=1 samples, and a slash is the boundary of the sample classification. Two loaves roughly tangent around the slash

PART2 Naive Bayes (Naive Bayes)

Under construction

Generate learning algorithms, introduction to Naive Bayes