Introduction to the principle of hierarchical clustering Gaussian mixture model clustering algorithm

Gaussian distribution model GSM

When a multidimensional variable x obeys a Gaussian distribution, its probability density function PDF is:

x is the column vector of dimension D, U is the model expectation, Σ is the model variance. In practice, U is usually substituted with sample mean, Σ is usually substituted by sample variance. It is easy to tell if a sample of x belongs to category C. Because each class has its own u and σ, the X into the (1), when the probability is greater than a certain threshold, we think that x belongs to Class C.

Geometrically speaking, the Gaussian distribution model in two-dimensional space should approximate to the ellipse, in three-dimensional space approximation to the ellipsoid. Unfortunately, in many classification problems, sample points belonging to the same category do not meet the characteristics of the "ellipse" distribution. This introduces the Gaussian mixture model.

Gaussian mixture model GMM

Each GMM is composed of a Gaussian distribution, each Gaussian is called a "Component", and these Component linearly add together to form the probability density function of GMM:

Each GMM is composed of a Gaussian distribution, each Gaussian is called a "Component", and these Component linearly add together to form the probability density function of GMM:

Because the sum of the equations and also the values that need our estimate, we use the iterative method, which we assume and are known when calculating, and we will take the last iteration value (or the initial value). Estimate parameters for each Component: Now let's assume that what we got in the previous step is the correct "probability of the data being generated by Component", which can also be used as a contribution to the Component in generating this data, or we can think of this as a value which has this part is generated by the Component. Concentrating on all data points is now actually considered Component generating these points. Since each Component is a standard Gaussian distribution, it can easily be distributed to find the corresponding parameter values for maximum likelihood:

Which, and also logically, can be estimated as. Repeat the first two steps of the iteration until the value of the likelihood function converges. When the parameters converge, a K-model is created and the K-models are used to classify them.

GMM is a clustering algorithm, and each component is a clustering center. The model parameters (Π,u and σ) are computed in the case of only the sample points, without knowing the sample classification (which contains the implied variables)----which can obviously be solved by the EM algorithm. Then use the well-trained model to classify the classification of different samples by: Step1 randomly selects one of the K component (the selected probability is

ΠK); Step2 the sample into the newly selected component, judging whether it belongs to this category, if it does not belong to the Step1.