The popular interpretation of principal component (PCA) and singular value decomposition (SVD)

principal Component Analysis

1. Description of the problem

In many fields of research and application, it is often necessary to make a large amount of observations on multiple variables that reflect things, and collect large amounts of data for analysis to find the law. Large-scale multivariate samples will undoubtedly provide rich information for research and application, but also to some extent increase the workload of data collection, more importantly, in most cases, there may be correlations between many variables, thus increasing the complexity of the problem analysis, while the analysis is inconvenient. If each indicator is analyzed separately, the analysis is often isolated rather than integrated. Blindly reducing the indicator will lose a lot of information, easy to produce wrong conclusions.

2. Process

Principal component Analysis (PCA) is a technique of data conversion, and when we measure an object, we represent it in vectors (a1,a2,a3,... an), each of which has its corresponding variance (which indicates the degree of dispersion near its mean); The sum of variance of all its dimensions, We call it the total variance; When we measure objects, we tend to have correlated between them, for example, when we measure pilots, there are two indicators, one is flight technology (x1) and the other is the degree of preference for flying (x2), which is related, That's correlated. We do PCA (principal component analysis), we do not change the dimension, but we do the following transformation, set a new feature (X1,X2,X3...,XN);

which

1) X1 Variance accounted for the total variance proportion of the largest;

2) Remove x1,x2 variance accounted for the remaining variance proportion of the largest;

....

by analogy;

Finally, we convert the resulting (x1,x2,... xn) between the incorrelated, when we do PCA, we only take (x1,x2,.... xk), to represent the object we measure, where k is less than N. The contribution rate of the principal component is the ratio of the variance of a principal component to the total variance. The larger the value, the stronger the ability to synthesize X1,X2,...,XP information into the main component. If the contribution rate of the former K principal component reaches 85%, it indicates that the first K principal component contains the information of all the measuring indexes, which reduces the number of variables and facilitates the analysis and research of the practical problems.

Note that we do not need to do PCA when there is a incorrelated between (A1,a2,a3,... an)

Data points are projected in the direction shown in the figure above, the data still has a great variance, but in the direction shown in the figure below, the variance of the projected data is very small.

All we need to do is find this series of vectors, so that the projection of the data on it has a large variance.

3. Mathematical description

To be able to find this series of vectors, we preprocess the data

1) Alcohol
2) Malic acid
3) Ash
4) Alcalinity of ash
5) Magnesium
6) Total phenols
7) Flavanoids
8) Nonflavanoid phenols
9) Proanthocyanins
) Color intensity
One) Hue
od280/od315 of diluted wines
Proline)

The sample number is 130, under MATLAB in accordance with the above steps, the PCA, the resulting eigenvalues are as follows:

The first k eigenvalues are selected to make the contribution rate of the former K principal components reach 85%, the calculated result is k=1, and the corresponding eigenvector is u=

So that X=x*u can get a new x, where x is the original dimension of 130x13, after PCA, the dimension is 130x1.

Excerpt from: http://blog.csdn.net/lopohui1/article/details/8125542