Generative learning algorithm Stanford machine learning notes

Generative learning algorithm corresponds to discriminative learning algorithm.AlgorithmAll belong to supervised learning (Supervised Learning Algorithm). The following describes discriminative learning algorithm:

We define {Xi, Yi} as a training sample. The discriminative learning algorithm is called by directly learning P (Y | x) or learning the ing from X to label {0, 1.

However, we can also create a model for P (x | Y) to obtain P (Y | X ). This is because, according to the Bayesian formula, P (Y | X) = p (x | y) * P (y)/P (x ). Therefore, modeling P (Y | X) to maximize P (Y | X) is equivalent to modeling the latter to maximize it. At the same time, because X is given, p (x) is fixed, so P (x) has no impact on modeling, so Arg Max P (Y | X) = Arg Max p (x | y) * P (y ).

First, we will introduce Gaussian discriminant analysis, which is an algorithm used for classification and corresponds to the situation where x obtains continuous values.

The model is Y following the bernuoli distribution with the following parameters: X | Y = 0 following the multivariate Gaussian distribution with the mean value of μ 0 and the covariance matrix of Σ, X | y = 1 follows the multivariate Gaussian distribution where the mean value is μ1 and the covariance matrix is Σ (This will be discussed later ).

The log function for maximum likelihood estimation is recorded as L (ø, μ 0, μ 1, Σ) = Log 1_mi = 1 p (x (I) | Y (I); μ 0, μ 1, Σ) P (Y (I); ø), our goal is to obtain the parameter ø, μ 0, μ 1, Σ to make L (ø, μ 0, 1, Σ) to obtain the maximum value.

The values of the four parameters that maximize the values of L (ø, μ 0, μ 1, and Σ can be calculated using a formula. We will not describe them here, for more information, see the notes for the Stanford University cs229 machine learning course.

With these four parameters, you can calculate and compare the values of P (y = 1 | X) and P (y = 0 | X), and then perform classification.

Write this first, and the formula is too difficult to write!