Exercise: PCA in 2D

Step 0: load data

The starter Code contains code to load 45 2D data points. When plotted usingScatterFunction, the results shocould look like the following:

 

Step 1: Implement PCA

In this step, you will implement PCA to obtainXROT, The matrix in which the data is "rotated" to the basis comprising made up of the principal components

 

Step 1a: finding the PCA basis

Find and, and draw two lines in your figure to show the resulting basis on top of the given data points.

Step 1b: Check xrot

ComputeXrot, And useScatterFunction to check thatXrotLooks as it shoshould, which shoshould be something like the following:

 

Step 2: dimension reduce and replot

In the next step, SetK, The number of components to retain, to be 1

Step 3: PCA Whitening

Step 4: zca Whitening

 

Code

Close all % ============================================ ==========================================%% STEP 0: load data % We have provided the code to load data from pcadata.txt into X. % x is a 2*45 matrix, where the kth column X (:, k) corresponds to % the kth data point. here we provide the code to load natural image data into X. % You do not need to change the code below. X = load('pcadata.txt ','-ASCII '); % load data figure (1); SC Atter (x (1, :), X (2, :)); % draws the data distribution title ('raw data') using a circle '); % ===================================================== ==========================================%% Step 1a: implement PCA to obtain U % implement PCA to obtain the rotation matrix U, which is the eigenbasis % Sigma. % -------------------- your code here ------------------ u = zeros (SIZE (x, 1); % You Need to compute this [n m] = size (X ); % x = x-repmat (mean (x, 2), 1, m); % preprocessing, mean 0 -- Two dimensions, with the mean Sigma = (1.0/m) x on each dimension subtracted from the dimension; % covariance matrix [u s v] = SVD (SIGMA ); % ------------------------------------------------------ hold onplot ([0 U ()], [0 U ()]); % draw the first line plot ([0 U ()], [0 U (2, 2)]); % draw the second line scatter (x (1, :), X (2 ,:)); hold off % ============================================ ========================================%% Step 1b: compute xrot, the projection on to the eigenbasis % Now, compute xrot by projecting t He data on to the basis defined % by U. visualize the points by grouping A scatter plot. % -------------------- your code here ------------------ xrot = zeros (SIZE (x); % You Need to compute thisxrot = u '* X; % percent % visualise the covariance matrix. you shoshould see a line between ss the % diagonal against a blue background. figure (2); scatter (xrot (1 ,:), Xrot (2, :)); Title ('xrot '); % ===================================================== ==========================================%% Step 2: reduce the number of dimensions from 2 to 1.% compute xrot again (this time projecting to 1 dimension ). % then, compute xhat by projecting the xrot back onto the original axes % to see the effect of dimension functions % -------------------- your code here -------------------- k = 1; % use k = 1 Nd project the data onto the first eigenbasisxhat = zeros (SIZE (x); % You Need to compute thisxhat = u * ([U (:, 1), zeros (n, 1)] '* X); % dimensionality reduction % place the feature point in the direction indicated by the feature vector rather than in the original coordinate system % ------------------------------------------ figure (3); scatter (xhat (1, :), xhat (2, :)); Title ('xhat '); % ===================================================== ==========================================%% Step 3: PCA whitening % complute xpcawhite and plot the re Sults. epsilon = 1e-5; % ---------------------- your code here ------------------ xpcawhite = zeros (SIZE (x); % You Need to compute thisxpcawhite = diag (1. /SQRT (DIAG (s) + epsilon) * U' * X; % divide each feature by the corresponding feature vector, to make each feature have a consistent variance % random figure (4); scatter (xpcawhite (1, :), xpcawhite (2, :)); Title ('xpcawhite '); % ===================================================== ============================== ========%% % Step 3: zca whitening % complute xzcawhite and plot the results. % -------------------- your code here ------------------ xzcawhite = zeros (SIZE (x); % You Need to compute thisxzcawhite = u * diag (1. /SQRT (DIAG (s) + epsilon) * U' * X; % random figure (5); scatter (xzcawhite (1, :), xzcawhite (2 ,:)); title ('xzcawhite'); % congratulations! When you have reached this point, you are done! % You can now move onto the next PCA exercise .:)

Exercise: PCA in 2D