Reprint: Singular value decomposition (SVD)---linear transformation geometric meaning (i)

This article is reproduced from others:

PS: Since the SVD decomposition indefinitely, this text is translated, the original text with meticulous analysis + a large number of visual graphics to demonstrate the meaning of SVD. It is not easy to explain the problem so clearly in a limited space. The original text gave a simple image processing problem, simple image, sincerely hope that the passing of the road friends from different angles to explain their own understanding of the actual meaning of SVD, such as personalized recommendations in the application of SVD, text and web mining often use the SVD.

Original: We recommend a singular value decomposition

Brief introduction

SVD is actually a mathematical specialty, but it has now infiltrated into different fields. The SVD process is not very well understood, because it is not intuitive enough, but its effect on matrix decomposition is very good. For example, Netflix, a company that offers online movie leasing, once offered a $1 million reward, if anyone could improve the accuracy of its film recommendation system scoring forecast by 10%. Surprisingly, this goal is fraught with challenges, with teams from around the world using a variety of different technologies. The ultimate winning team "Bellkor's pragmatic Chaos" uses the core algorithm based on SVD.

SVD provides a very convenient matrix decomposition method, which can discover the interesting potential patterns in the data. In this article, we will provide an understanding of SVD geometry and some simple examples of applications.

the geometrical meaning of linear transformation (Thegeometry of linear transformations)

Let's take a look at some simple linear transformation examples, taking a linear transformation matrix of 2 X 2 For example, first of all, a more specific, diagonal matrix:

Geometrically,M is a transformation matrix that transforms a point (x, y) on a two-dimensional plane from a linear transformation to another point, as shown in

The effect of the transformation, as shown, is that the transformed plane is only stretched 3 times times along the X level, and the vertical direction is not changed.

Now look at the matrix.

This matrix produces a transformation effect as shown in

This kind of transformation effect looks very strange, in the actual environment it is difficult to describe the transformation of the law (here should be not clear to identify the angle of rotation, the ratio of stretching and other information). Or based on the symmetric matrix above, let's say we rotate the left plane 45 degrees, and then the linear transformation of the matrix M , as shown in the result:

Does it look a little familiar? Yes, after M linear transformation, the function of the diagonal matrix is the same as the previous one, which stretches the mesh 3 times times in a direction.

M Here is a special case because it is symmetrical. Non-special is that we often meet some asymmetric, non-matrix matrices in practical applications. As shown, if we have a 2 X 2 symmetric matrix m , we first rotate the mesh plane to a certain angle, and the transformation effect ofm is to stretch the transformation in two dimensions.

In a more mathematically expressed way, given a symmetric matrix M , we can find some orthogonal vi , satisfying Mvi is along the V I direction of the stretching transformation, the formula is as follows:

M vi =λiVI

The λi here is the stretch scale (scalar). Geometrically,M stretches the vector Vi and maps the transformation. V I is called the eigenvector of the Matrix M (eigenvector), λi is called as the matrix m eigenvalue (eigenvalue). There is a very important theorem here that the eigenvectors of the symmetric matrix M are orthogonal to each other.

If we use these eigenvectors to linearly transform the mesh plane, then the linear transformation of the grid plane through the m matrix is the same as the linear transform of the eigenvector of the m matrix.

For more common matrices, what can we do to make a grid plane (orthogonal grid) that is perpendicular to each other, linearly transforming into another grid plane perpendicular? PS: Here the vertical, is the two staggered lines are vertical.

After the above matrix transformation effect

As you can see, it does not achieve the effect we want. We rotate the grid plane 30 degrees, and then the same linear transformation after the effect, as shown in

Let's look at the effect when the grid plane rotates at a 60-degree angle.

Well, this looks pretty good. If the precise point, the grid plane should be rotated 58.28 degrees to achieve the desired effect.

Reprint: Singular value decomposition (SVD)---linear transformation geometric meaning (i)