Machine learning dimension reduction algorithm 3: lle (Locally Linear embedding) Local linear embedding

Please note that this article is from: http://www.cnblogs.com/xbinworld Lle

Locally Linear embedding (LLE) [1] is a non-linear dimension reduction.AlgorithmIt can make the data after Dimensionality Reduction better maintain the originalManifold Structure. Lle can be said to be one of the most classic tasks of manifold learning methods. Many subsequent Manifold Learning and dimensionality reduction methods are closely related to lle.

As shown in figure 1, after Three-dimensional data (B) is mapped to two-dimensional data (c) Using lle, the mapped data can still maintain the original data manifold (the red points are close to each other, blue is also close to each other), which indicates that lle effectively maintains the original popular data structure.

However, lle is not applicable in some cases. If the data is distributed on the entire closed sphere, lle cannot map it to two-dimensional space, and it cannot maintain the original data manifold. In processing data, we first assume that the data is not distributed on a closed or elliptical sphere.

 

Figure 1 lle Dimension Reduction Algorithm example

The lle algorithm considers that each data point can be constructed by a linear weighted combination of its neighboring points. The main steps of the algorithm are divided into three steps: (1) Finding K neighboring points of each sample point; (2) Calculating the local reconstruction weight matrix of this sample point from the neighboring points of each sample point; (3) the output values of the sample point are calculated from the local reconstruction weight matrix of the sample point and its neighboring points. The specific algorithm flow 2 is shown below:

Figure 2 lle algorithm steps

 

 

 

 

 

 

Reference

[1] roweis, S. T., Saul, L. K. Nonlinear Dimensionality allocation ction by Locally Linear embedding. Science. 2000,290 (5500): 2323.