Principal component Analysis (PCA) _ Machine learning

Defined

The idea of PCA is to map n-dimensional features to K-Dimensions (K- background

In the machine learning process, the first step is the data processing. In most machine learning classes, in order to simplify understanding, the first few lessons are to select only 1~2 features. This leads to problems, if the characteristics of more than what to do. In the analysis of regression problems, the gradient descent method is introduced, which is established for n characteristics. It is also found that the value of theta can be solved by means of a matrix. However, it is emphasized that the product of the transformation matrix (XT x^t) of the feature matrix X and the feature matrix X is reversible. (Of course, the use of octave, even if not reversible, can also find solutions, but theoretically not set up) mathematical knowledge

A mathematical problem is introduced here, when a matrix A is not reversible, in short, the rank of the matrix is less than the number of rows of the matrix. That is to say, there are at least two variables that are linearly correlated. For example, if the length is calculated in terms of "M", and the length is calculated in terms of "centimeter", there is a linear relationship between the two features, and a characteristic must be removed.
The stronger the two feature relationships are, the more independent they are, and the stronger their interaction is, the greater the covariance. Because there are more than one feature, we get a covariance matrix , and a "eigenvector". Intuitive Understanding

As shown in this is a 2-dimensional distribution map, we found that in the direction of V1 v_1 data distribution is more dispersed, in the direction of V2 v_2 distribution of the more concentrated, when the most extreme, V2 v_2 direction only one point, at this time 2-dimensional distribution into 1-dimensional distribution. Descending the dimensions of success. V1 v_1 Direction is the eigenvector of covariance matrix. by summarizing the process , the covariance matrix is obtained and the eigenvalues and eigenvectors are determined, and the former K eigenvalues are taken. Projecting to a feature vector