Again, the EM algorithm, and the Gaussian mixture model, the maximum likelihood estimation

First Listed materials:

Derivation calculation of Gaussian mixture model (English version):

Http://www.seanborman.com/publications/EM_algorithm.pdf

This translation is written in Chinese:

Http://www.cnblogs.com/jerrylead/archive/2011/04/06/2006936.html

Flow of Gaussian mixture model:

Http://www.cnblogs.com/jerrylead/archive/2011/04/06/2006924.html

Maximum likelihood estimate:

http://blog.csdn.net/yanqingan/article/details/6125812

A Gaussian mixture model (GMM) contains a plurality of one-dimensional Gaussian distribution (GM), as the case may be, if the class's sex is known, height is also known, then the class of students to build the GMM can be so, the number of GM to take 2, respectively, the height of the boy's Gaussian distribution and the height of the female Gaussian distribution, First through the boys and girls to divide the weight of a single GM, that is, boys \ Girls occupy the proportion of the class, and then the boy's height data to calculate the Gaussian distribution, the girl is also, so that the construction of good one GMM.

Of course the above example is an idealized calculation, if a group of sample n-by-d is clustered, n represents the number of samples, and D represents the dimension of the sample .

The training of a GMM is calculated by EM and maximum likelihood estimates, which need to be determined by the weight W of each GM in GMM, and the sample mean u in each GM and the covariance matrix ∑ (d-by-d matrix), which is the two parameters of GM distribution. A sample of a GM is the same class, so the number of classes changes the number of GM.

By using maximum likelihood estimation as the end judgment of GMM training, maximum likelihood estimation is judged according to the expectation of X in GMM distribution (this sentence is inaccurate, see GMM derivation above).

M-step:

Enter the M-step, the Q as known (fixed q), that is, fixed the sample belongs to which GM, indirectly obtained W, and then by updating the GM in the U, ∑.

E-step

M-step after the end of the new U, ∑, update the destination of the sample, such as the i-th sample, calculate he again belong to the probability of each GM, take the highest value of index, so update the class designator, will [] See left constant (fixed part of the parameter), then the main effect of this expectation is Q, Then judging the loglikelihood of the previous step and the current step can make an end judgment (less than a threshold convergence), which is the e-step in EM.

Algorithm Flow:

Initialization

Class labels for sample points can be initialized randomly (how many class K values need to be specified), or class labels can be assigned.

Maximization:

To maximize the steps, for an existing class designator (initialized with the latter iteration is updated), to build or update the K-GM, for each GM has its own weight w, the sample mean U-covariance matrix ∑.

Calculation formula:

The weight formula is as follows, J means j-th a gm,m as the number of samples, that is, the above n, the reason here is φ and W, is a fixed part of the update another part of the update value of W bit e-step.

Sample average:

Covariance:

Expectation:

Using the M-step updated GM, calculate the probability of the sample distribution to each GM, that is P (x|z), select the largest as its allocation of results, and then calculate the following L, as a convergence judgment, pick out the failure of repetition M-step

P (x|z) in the formula: N (X|PMIU,PSIGMA) = 1/((2PI) ^ (D/2)) * (1/(ABS (sigma)) ^0.5) *exp ( -1/2* (X-pmiu) ' psigma^ ( -1) * (X-pmiu))

P (z): This is the weight of a GM after the class label has been updated.

The

also looked at the EM algorithm again, and the Gaussian mixture model, maximum likelihood estimation