The understanding matrix and the spectral clustering small note

Recently read some of the matrix and spectral clustering knowledge, special here to record briefly. You can read the references in detail first.

The first thing to see is Meng Yan wrote the three < understanding matrix;.

One: Understanding Matrix (i)

1: The definition of a traditional book space: There is a set that defines a certain concept on this set, and then satisfies some nature "that can be called space." Meng's space contains four points: (1). Consists of a number of (in fact, infinitely multiple) position points; (2). There is a relative relationship between these points; (3). Length, angle, 4 can be defined in space. This space can accommodate motion, where what we call movement is moving from one point to another (transformation ), not a "continuous" movement in the sense of calculus. The 4th is the most important, accommodating the nature of the space during movement .

2: " Space" is a collection of objects that accommodate motion, while transformations specify the motion of the corresponding space.

3: Any object in a linear space can be expressed as a vector by selecting the base and coordinate methods.

4: In linear space, when you select a group of bases, you can not only use a vector to describe any object in the space, but you can use a matrix to describe any motion (transformation) in the space. The way to make an object correspond to motion is to use the matrix representing that movement multiplied by the vector representing that object. the essence of a matrix is the description of motion .

Second: Understanding the Matrix (ii)

1: The so-called transformation is actually a leap in space from one point (element / object) to another (element / object). A matrix is a description of the transformations in a linear space

2: for a linear transformation, as long as you select a set of bases, then you can find a matrix to describe the linear transformation. To change a group of bases, you get a different matrix. All of these matrices are a description of the same linear transformation, but they are not linear transformations themselves.

3: The so-called similarity matrix is a different description matrix of the same linear transformation. like multiple references to the same object.

Three: Understanding the Matrix (iii)

1: The transformation of the object is equivalent to the transformation of the coordinate system. Alternatively, the transformation of the next object in a fixed coordinate system is equivalent to the coordinate system transformation where the fixed object is located.

Three series of matrices let us understand the matrix from the intuition, the image is profound. The first one is to talk about space and Matrix, the second is equivalent to the matrix, and the third is the matrix equivalent to the coordinate system.

Four: Spectral clustering

When we talk about spectral clustering, we need to review some mathematical knowledge first.

1: Orthogonal matrix. Ata=e, then A is an orthogonal matrix. That is, a row or column vector is a 22 vertical unit vector.

2: Positive definite matrix. If there is ftmf> 0 for any vector F, then M is the positive definite matrix. If it is greater than or equal to 0, the semi-positive matrix.

3: Laplace matrix: The Laplace matrix is the degree matrix of the graph-the neighboring matrix of the graph. It is also the basis of spectral theory.

which

3: spectral clustering is initially used to solve the problem of the partition of the graph, the purpose of the graph segmentation is to minimize the similarity between classes, the similarity is the most. It is a NP - Hard problem, it can be converted to minimize the Laplace Matrix eigenvalue problem, this can be proved by the formula . If divided into two categories, at this time the smallest eigenvalues of the corresponding eigenvectors of more than 0 belong to a class, less than 0 belong to another class, you can split the graph into two parts.

if the k classification, you need to get k a characteristic vector corresponding to the smallest eigenvalues (Nxk) , each column is a feature vector, each row represents a sample point, and it is K-means clustering, you can add N a dot together k class .

4: Spectral clustering is used for the actual sample space, just consider each sample as a vertex of the graph, and then the distance of the sample points through the function map to similarity, each point and other points similar to the degree can be spectral clustering.

5: The Advantages: (1) Spectral clustering can be clustered on arbitrary shape sample space, and converge to the most advantages of the global. And like K-means algorithm and EM algorithm is based on convex spherical sample space, when the sample space is not convex, the algorithm will fall into the "local" optimal.

(2) spectral clustering requires only a similarity matrix between the data , rather than requiring the data to be vectors in N dimensional Euclidean's space as K-means.

(3) The Ratiocut method considers only the smallest similarity between classes, and Normalizedcut not only considers the similarity between classes.

V: Spectral Hash

Cond.....

Reference documents:

1: Understanding Matrix (i) http://blog.csdn.net/myan/article/details/647511

2: Understanding Matrix (ii) http://blog.csdn.net/myan/article/details/649018

3: Understanding Matrix (iii) http://blog.csdn.net/myan/article/details/1865397

4:july's http://blog.csdn.net/v_july_v/article/details/40738211 from Laplace matrix to spectral clustering

5: Study of spectral clustering method in machine learning: http://lamda.nju.edu.cn/conf/MLA07/files/YuJ.pdf;

The understanding matrix and the spectral clustering small note