Geometric meanings of eigenvalues and eigenvectors

Geometric meanings of eigenvalues and eigenvectors

What is the multiplication of matrices, don't just tell me that just "the row of the previous matrix is multiplied by the column of a Matrix", it would also be possible to say "the number of columns of the previous matrix equals the number of rows of the latter matrix", but here you will say-that is the appearance.

Matrix multiplication The real meaning is the transformation, we learn "linear algebra" began to study row Transformation Column transformation, that is the core of the line generation-not a bit of a trick to forget this-yes, matrix multiplication is a linear transformation, if one of the vector A is the center, then the role of B is mainly to make a change of the following:

2. Telescopic

   clf;A = [0, 1, 1, 0, 0;...    1, 1, 0, 0, 1];  % 原空间B = [3 0; 0 2];      % 线性变换矩阵plot(A(1,:),A(2,:), ‘-*‘);hold ongrid on;axis([0 3 0 3]); gtext(‘变换前‘);Y = B * A;plot(Y(1,:),Y(2,:), ‘-r*‘);grid on;axis([0 3 0 3]); gtext(‘变换后‘);

   1

   From the known, the Y-direction of the twice-fold stretch, X-direction 3 times times the tensile, which is b=[3 0; 0 2] of the credit, 3 and 2 is the scaling ratio. Note that at this point B, except that the diagonal element is a multiple of each dimension, the non-positive diagonal element is 0, as it will be shown below that the diagonal element is not 0 and the shear and rotation effect occurs.

4. Shear

     CLF;     A = [0, 1, 1, 0, 0; ....  1, 1, 0, 0, 1];       % Original Space B1 = [1 0; 1 1];       % linear transformation Matrix B2 = [1 0;-1 1];       % linear transformation Matrix B3 = [1 1; 0 1];       % linear transformation Matrix B4 = [1-1; 0 1]; % linear transformation Matrix Y1 = B1 * A; Y2 = B2 * A; Y3 = B3 * A; Y4 = B4 * A;subplot (2,2,1);p lot (A (1,:), a (2,:), '-* '), Hold On;plot (Y1 (1,:), Y1 (2,:), '-r* '); grid On;axis ([-1 3-1 3]); SUBPL OT (2,2,2);p lot (A (1,:), a (2,:), '-* '), Hold On;plot (Y2 (1,:), Y2 (2,:), '-r* '), Grid On;axis ([-1 3-1 3]); subplot (2,2,3); Plot (A (1,:), a (2,:), '-* '), Hold On;plot (Y3 (1,:), Y3 (2,:), '-r* '), Grid On;axis ([-1 3-1 3]), subplot (2,2,4);p lot (A (1,:), a (2,:), '-* '); Hold On;plot (Y4 (1,:), Y4 (2,:), '-r* '); grid On;axis ([-1 3-1 3]);  

   2

6. Rotating

   All of the transformations can be achieved through the above scaling and shear transformation, if the transformation matrix B is reasonable to take the value, you can get the effect of the graphics rotation, as follows,

   clf;A = [0, 1, 1, 0, 0;...     1, 1, 0, 0, 1];  % 原空间theta = pi/6;B = [cos(theta) sin(theta); -sin(theta) cos(theta)];Y = B * A;figure;plot(A(1,:),A(2,:), ‘-*‘); hold on;plot(Y(1,:),Y(2,:), ‘-r*‘);grid on;axis([-1 3 -1 3]);

   3

OK, here's the matrix multiply. So, we go on to the topic, for a particular vector, after a matrix transformation, after the transformation, the direction of the vector is not changed (or just reverse), but only for the scaling change (the scaling value can be negative, equivalent to the direction of the vector reverse)? This time we might as well compare the definition of eigenvectors to the book:

Mathematical textbook definition: Set A is n-order matrix, if there is λ and n-dimensional nonzero vector x, so, then λ is called a eigenvalue of square A, X is a matrix a corresponds to or belongs to a characteristic vector of the eigenvalue λ.

is the specific vector above the eigenvector? λ is not the multiplier of the scale? Therefore, the algebraic meaning of the eigenvector is that the matrix multiplication is transformed into a multiply operation; the geometric meaning of the eigenvector is that the eigenvector is only scaled by the square a transform, while the direction of the eigenvector remains unchanged. The eigenvalues represent how important this feature is, similar to weight, and the eigenvector is a point in geometry, representing the direction of the vector from the origin point to the direction of the point.

Eigenvector has an important property: the linear combination of any number of eigenvectors of the same eigenvalue is still a feature vector of the same eigenvalue. With regard to eigenvalues, there is an explanation on the internet about "eigenvalues is a spectrum of vibrations":

In the time of the Song Dynasty, our country brushed off the chance of discovering the matrix eigenvalue theory. Words did not promise Kinshao in the pond to throw a small stone, just get a "cast stone burst underwater days" of the girls, the monkey anxious to go to the bridal chamber, there is no thought of the characteristics of the matrix and the characteristic vector of the scientific sermon. Presumably, any spot of water near the surface vibrates (actually doing approximately circular motion) and does not move with the waves toward the outer ring, while the amplitude of the droplets of the upper and lower vibrations is gradually becoming smaller, until it becomes calmer. In a matrix determined by a certain mass and shape of a stone being put into a particular pool of area and depth at a certain angle and speed, the characteristic value plays a decisive role in the gradient of the ripples in the ripple, which determines the declining rate of the frequency and amplitude of the water droplets vibration.

In the process of understanding the eigenvalues and eigenvectors of vibration, it is necessary to add the concept of complex vectors and complex matrices, because real vectors and real matrices are not much to be done in practical applications. The mechanical vibration and the electric vibration have the spectrum, the vibration one frequency has a certain amplitude, then the matrix also has the matrix the spectrum, the matrix spectrum is the matrix characteristic value concept, is the matrix intrinsic characteristic, all characteristic value forms the matrix one spectrum, each characteristic value is one "resonant frequency point" of the Matrix.

American mathematician Stergen (G. Strang, in its classic textbook "Linear Algebra and its Applications", describes the physical significance of eigenvalues as frequencies, he says:

Probably the simplest example (I never believed in its authenticity, although it is said that a bridge was destroyed in 1831) is an example of a pair of soldiers passing bridges. Traditionally, they want to stop March and walk through. The reason for this is that they may march at a frequency equal to one of the characteristic values of the bridge, which will resonate. Just like a child's swing, once you notice the frequency of a swing and match that frequency, you make the frequency swing higher. An engineer always tries to keep his bridge or his rocket's natural frequency away from the frequency of wind or the frequency of liquid fuel; In another extreme case, a stockbroker devotes his life to the natural frequency line of the market. Eigenvalue is the most important characteristic of almost any dynamical system.

In fact, the matrix is able to form a "frequency spectrum", because the matrix in the direction of the characteristic vector has a constant transformation of the vector: enhance (or weaken) the role of the eigenvector. Further, if the matrix continues to overlap for vectors, then the eigenvectors will be highlighted.

See wikipedia:http://zh.wikipedia.org/wiki/eigenvectors For more practical examples of eigenvectors and eigenvalues.

Eigen-Value decomposition

Set A has n eigenvalues and eigenvectors, then:

Write the above into a matrix form:

if (x1,x2,..., xn) reversible, then the left and right sides are inverse, then the square a can be directly through the eigenvalues and eigenvectors of the unique expression, so that

q= (x1,x2,..., xn)

Σ?=?diag (λ1,?λ2,?...,? λn)

Then, the expression is called the eigenvalue decomposition of the square matrix, so that the matrix A is represented by eigenvalues and eigenvectors only.

All eigenvectors of a transformed phalanx form a set of bases for this transformation matrix. The so-called base, which can be understood as the coordinate system axis. Most of our usual use of the Cartesian coordinate system, in linear algebra can be distorted, stretched, rotated, called the base transformation. We can set the base according to the demand, but the base axis must be linearly independent, that is, to ensure that the different axes of the coordinate system do not point to the same direction or can be combined by other axes, otherwise the original space will not rise. From the point of view of linear space, in a linear space that defines the inner product, the characteristic decomposition of an n-order symmetric square is to produce n standard orthogonal bases of the space, and then to project the matrix onto the N bases. n eigenvectors are n standard orthogonal bases, while the modulus of eigenvalues represents the projection length of a matrix on each base. The larger the eigenvalues, the greater the variance of the matrix on the corresponding eigenvector, the greater the power and the more information. However, eigenvalue decomposition also has a lot of limitations, such as the transformation of the matrix must be a square.

In machine learning feature extraction, the meaning is that the maximum eigenvalue corresponding to the direction of the eigenvector contains the most information, if a certain number of characteristics are small, indicating that the information is very small, can be used to reduce the dimension, that is, the deletion of small eigenvalues corresponding to the direction of the data, only the large eigenvalues of the direction corresponding to the data, After this, the data volume decreases, but the useful information changes little, and the PCA dimensionality reduction is based on this idea.

In Matlab, eigenvalue and eigenvector matrix can be obtained by Eig function.

>> B = [ 3     -2      -.9    2*eps     -2      4       1    -eps     -eps/4  eps/2  -1     0     -.5    -.5      .1    1   ]B =    3.0000   -2.0000   -0.9000    0.0000   -2.0000    4.0000    1.0000   -0.0000   -0.0000    0.0000   -1.0000         0   -0.5000   -0.5000    0.1000    1.0000>> [V D] = eig(B)V =    0.6153   -0.4176   -0.0000   -0.1437   -0.7881   -0.3261   -0.0000    0.1264   -0.0000   -0.0000   -0.0000   -0.9196    0.0189    0.8481    1.0000    0.3432D =    5.5616         0         0         0         0    1.4384         0         0         0         0    1.0000         0         0         0         0   -1.0000

The elements of the D diagonal are the eigenvalues (representing the scale of the scaling), and D is that each column of the q,v in the eigenvalue decomposition formula corresponds to the D column, representing the corresponding eigenvector, the σ in the eigenvalue decomposition.

Singular value decomposition

Eigenvalue decomposition is a good way to extract matrix features, but it only applies to square matrices. In the real world, we see most of the matrix is not a square, for example, there are M students, each student has n subjects, so that the formation of a M * n matrix may not be a square, how can we describe such a general matrix as the description of the characteristics of the important characteristics of it? Singular value decomposition is used to do this, singular value decomposition is a decomposition method that can be applied to arbitrary matrices. It is necessary to first talk about the relationship between eigenvalues and singular values.

For the eigenvalue decomposition formula, ATA is a phalanx, we seek the characteristic value of ata, that is, the eigenvalues obtained at this time corresponding to the square of the singular value, the obtained eigenvector V is called the right singular vector, in addition can be obtained:

The UI is the left singular vector, and ōi is the singular value. There has been a clear analysis of SVD's geometrical mechanism, very useful, do not repeat the wheel, the following is reproduced from http://blog.sciencenet.cn/blog-696950-699432.html.

Geometric meanings of eigenvalues and eigenvectors