July algorithm-December machine learning Online Class-14th lesson Note-em algorithm

July Algorithm-December machine Learning online Class -14th lesson Note-em Algorithm

July algorithm (julyedu.com) December machine Learning Online class study note http://www.julyedu.com

?

EM expection Maxium Desired Maximum

1 cited examples

1000 people, Statistics height, 1.75,1.62,1.94, how many men and women, each height corresponds to the male and female

?

1.1 How to calculate? Estimating mean and variance using maximum likelihood estimation

The results of the above conclusions and moment estimates are consistent,

Namely: The mean value of the sample is the mean value of the Gaussian distribution, the pseudo-variance of the sample is the variance of the Gaussian distribution.

If it's a Gaussian distribution, you can use this calculation, the mean value and the variance

?

setting, the height of men and women obeys two Gaussian distributions

The random variable x is a mixture of K Gaussian distributions, if a series of samples of random variable x are observed x1,x2,..., xn,

The target function is: logarithmic likelihood function

Because of the addition of the logarithm function, it is impossible to directly derive the maximal value directly by the derivation solution equation. Divided into two steps

?

1.2 Intuitive understanding of EM

STEP1: estimated data from which group

STEP2: estimate parameters for each component

1.3 Gauss-Proof

Some of the pictures can be learned: To do the great expectations, and constantly do the great Expectations

?

Using Jensen inequalities (convex optimization can be used directly)

Qi is a certain distribution of Z, Qi≥0, with:

Further analysis: There is the above equation can be learned, p,q in direct proportion;

You can get the overall frame diagram of EM.

?

1.4 Derivation of GMM from theoretical formulae

Random variable x is a K Gaussian distribution mixed with the probability of each Gaussian distribution is φ1φ2 ... Φk, the mean value of the first Gaussian distribution is μi and the variance is ōi. If a series of samples of random variable x are observed x1,x2,..., xn, the parameter π,μ,σ is estimated.

E-step, and M-step, respectively.

E-step:

M-step: (First write the expectation, the maximum value)

The mean value is biased, so that the equation equals 0, the mean value of the solution:

Similarly, the value of the variance can be obtained.

After the variance and the mean are obtained, the φ1+φ2+ of φ is biased, and the constraint condition is ... Φk=1, which is the constrained condition with the equation to find the extremum, the Lagrange multiplier method

The final formula is the same as that of the original Euler-like interpretation.

For all data points, it can be considered that the component K generates these points. Component k is a standard Gaussian distribution.

Methods with implicit variables: em+ variational

?

July algorithm-December machine learning Online Class-14th lesson Note-em algorithm