Common machine learning algorithms Principles + Practice Series 3 (PCA)

PCA Principal Component Analysis

PCA (Principal Component analysis) is mainly to do data reduction, the data from the original coordinate system to the login new coordinate system, the choice of the new coordinate system is determined by the data itself, the first new axis selection is the original data of the most poor direction, The selection of the second new axis is orthogonal to the first axis and has the largest variance, and the process repeats and repeats the data of the characteristics in the original data. Usually, most of the variance is contained in the previous several new axes, so generally we choose the first several contribution value (above 90%) is the main component.

Through this dimensionality reduction technique, it can transform a large number of related variables into a set of very few related variables, which are called principal components, such as the original characteristic is x1,x2,......xk, after the projection of some new axes, the first principal component may become pc1=a1*x1+a2*x2+......+ak* XK, which is a weighted combination of K features, is the most analytic of the variance of the initial data set.

There are several steps to PCA:

1, the original matrix A removes the average value (the original average of each feature is calculated, and the original matrix is subtracted from the average to generate a new matrix B, which is the feature-centric)

2, calculate the covariance matrix C (covariance between 22 features, form a matrix)

3, the eigenvalues and eigenvectors of the covariance matrix C are computed (in order to find the maximum variance direction, which is the eigenvector)

4, the eigenvalues in the order of large to small, preserving the front topn feature vector V (such as the cumulative variance contribution value more than 90%)

5, transform the original data into a new space constructed by the n eigenvectors above, b*v

If you want to revert back to the original matrix A, use the following formula (precision has some loss is normal):

The following demonstrates a case in Python that uses NumPy to implement a PAC:

Here's what the redeigvects output looks like:

So obviously the main components of topn=3 are:

PCA1 = 0.36*x1-0.08*x2+0.85*x3+0.35*x4

PCA2 = -0.65*x1-0.72*x2+0.17*x3+0.07*x3+0.07*x4

Pca3=-0.58*x1+0.59*x2+0.07*x3+0.54*x4

Common machine learning algorithms Principles + Practice Series 3 (PCA)