coursera-Wunda-Machine learning-(programming exercise 7) K mean and PCA (corresponds to the 8th week course)

This series is a personal learning note for Andrew Ng Machine Learning course for Coursera website (for reference only)
Course URL: https://www.coursera.org/learn/machine-learning Exercise 7--k-means and PCA

Download coursera-Wunda-Machine learning-all programming practice answers

In this exercise, you will implement the K-means clustering algorithm and apply it to compressed images. In the second section, you will use principal component analysis to find the low dimensional representation of facial images. 1 K-means

Start with a two-dimensional point and use K-means to classify.

K-means steps as above, in each loop, first update the categories for all points, and then update the central coordinates for each category.

An example is provided in EX7.M, where K has been manually initialized.

We're going to divide the points into three categories and iterate 10 times. The central point of the three classes is initialized (3,3), (6,2), (8,5).
Get the following image. (The middle of the image omitted, only to show the beginning and completion of the image)
This is the initial image:

Images after 10 iterations: image
You can see that the three heap points are well divided into three categories. The image also shows the moving trajectory of the center point.

Use K-means to compress pictures.
With a picture of 128\times 128 as an example, using RGB, a total of 128\times 128 \times 24 = 393,216 bit is required.
Here we compress him, divide all the colors into 16 classes, replace the color of the whole class with its centroid color, and compress the space to 16\times + 128\times 128 \times 4 = 65,920 bit.
Using the examples provided in the topic, the effect is probably as follows:
2 PCA

In this exercise, you will use principal component Analysis (PCA) to perform dimensionality reduction. You will first try using the sample 2D dataset to visualize how the PCA works and then use it for a larger dataset in the 5000-face image dataset.

The provided script ex7 PCA.M will help you step through the first half of the exercise.

First, the two-dimensional vector implementation in the example is reduced to one dimension.


As can be seen from the diagram above, the restored graph retains only the information on one of the eigenvectors, while the vertical information is lost.

Face Image DataSet

Dimension reduction the face picture. Ex7faces.mat has a large number of human faces in the gray scale (\times 32), so the dimension of each vector is \times 32 = 1024.
The following are the first 100 face images:

The principal component is obtained by PCA, and then transformed into a matrix of 32\times 32, which is visualized as follows: (Show only the first 36)

The first 100 eigenvectors are projected,

As you can see, the approximate frame of the face is retained after the dimension is lowered, but some details are lost. This gives us the idea that when we use neural networks to train face recognition, we can sometimes use this method to improve speed.