Real-combat EM algorithm and image segmentation

EM algorithm is a method to estimate the maximum likelihood of parameters, it can estimate the parameters from the incomplete data set, it is a very simple and practical learning algorithm. This method can be widely used to deal with the defect data, censored data and the so-called incomplete data with noise , we can use EM algorithm to fill the missing data in the sample, find the value of hidden variables, estimate the parameters in Hmm, Estimating parameters in a finite mixed distribution and unsupervised clustering can be done.

Post a few Good articles: from the maximum likelihood to the EM algorithm shallow solution

Mixed Gaussian model (mixtures of Gaussians) and EM algorithm

Stanford University machine learning--em algorithm to solve Gaussian mixture model

About the principle of the algorithm derivation of the above article is described in detail, here again in the vernacular of the key points of personal understanding:

(1) What do we want to estimate? An estimate of the parameters of the unknown, whose parameters? What do you think of yourparameters of the model that the data conforms to, then you need to recognize what model our data conforms to. See a lot of the data is used to conform to the Gaussian (mixed) model, the formula is the normal distribution of the formula:
P (x)= 1 < Span class= "Mrow" id= "mathjax-span-560" > 2 π Span style= "Display:inline-block; width:0px; Height:2.723em; " >?? √ σ exP(?(x?u ) 2 2σ 2 )
So before estimating the parameter, it is considered that the data conforms to the model, and the model requires both the mean and the variance, so it is needed so that a new data can be calculated to calculate the probability that it belongs to this model.

(2) about the initialization parameters of the EM algorithm , read a lot of articles, feel no how to say in the EM algorithm inside, parameter initialization is how important, until their own in the implementation of the EM algorithm, find out how bad the results, check a lot of reasons to this problem. Although the EM algorithm is an egg problem, but this egg problem how to feel a bit special, in the initialization must give a set of decent away from the egg, and finally in the process of repeated evolution to grow into a complete chicken, otherwise you in the initialization of the time to give a big stone, The results are generally less likely to evolve into chickens.

Well, as far as the detailed principle, the network resource is not a lot of exhaustion. However, many principles but how to implement the code is very few, here a simple example (image segmentation, is actually unsupervised clustering) to illustrate the implementation process. When clustering, set the number of clusters, then each class of data is considered to obey a Gaussian distribution, then we use the EM algorithm to estimate the Gaussian parameters in each class. About Gray-scale images, in the cluster is actually a one-dimensional sample, each pixel as a sample point, it is worth the size of its characteristics, so that you can, after the cluster and then transform it into an image display, MATLAB code as follows:

Clcclearclose allimg = double (Imread (' lena.jpg '));% set number of clustersCluster_num =4;% Random initialization parameters--critical% i choose to be more close to the real mean value% so the effect will be betterMu = (1: Cluster_num)./(Cluster_num +1) * MAX (Max (IMG)); sigma = MU;PW =Zeros(Cluster_num,size(IMG,1)*size(IMG,2));p C =Rand(1, cluster_num);p C = pc/sum (PC);% normalized the probability of a classMax_iter = -;% as the condition of the stop as the number of iterationsITER =1; whileITER <= Max_iter%----------E-------------------     for I=1: Cluster_num% matrix operation-fast speedMU =Repmat(Mu (I),size(IMG,1)*size(IMG,2),1);% Gauss Modeltemp =1/sqrt(2*Pi*sigma (I))*Exp(-(IMG (:)-mu). ^2/2/sigma (I)); Temp (temp<0.000001) =0.000001;% prevents the appearance of 0% pw (i,:) = log (PC (i)) + log (temp);pwI,:) = PC (I) * TEMP;EndPW =PW./(Repmat(SUM (PW), Cluster_num,1));% return    %----------M---------------------    % update parameter set     for I=1: Cluster_num PC (I) = Mean (PW (I,:)); MuI) = PW (I,:) *img (:)/sum (PW (I,:)); SigmaI) = PW (I,:) * (img (:)-mu (I)).^2)/sum (PW (I,:));End    %------------Show-result---------------    [~,label]= max (PW);% change SizeLabel =Reshape(Label,size(IMG)); Imshow (Label,[]) Title ([' iter = ', NUM2STR (ITER)]); Pause0.1);    M (iter,:) = mu;    S (iter,:) = Sigma; ITER = iter +1;End% will show the iterative process of mean and varianceFigure for I=1: Cluster_num Plot (M (:,I)); OnEndFigure for I=1: Cluster_num Plot (S (:,I)); OnEnd

Mean value with algebraic change graph

Variance with algebraic variation diagram

You can see that this is the setting of the 4 class change diagram, the program inside the parameter initialization settings, especially for the mean MU setting is very important, set the bad words of the effect is very common, that is, the parameter estimation is also very poor, and the number of classes required to divide the larger (that is, the more parameters need to estimate), the higher the

Copyright NOTICE: This article for Bo Master original article, without Bo Master permission not reproduced.

Real-combat EM algorithm and image segmentation