"Deeplearning" EXERCISE:PCA and Whitening

EXERCISE:PCA and Whitening

Exercise Links:EXERCISE:PCA and Whitening

Pca_gen.m

%%================================================================%%Step 0a:load Data%Here we provide the code to load natural image data into X.% x would be a144*10000Matrixwherethe kth column x (:, k) corresponds to% The RAW image data fromThe kth 12x12 image patch sampled.% You DoNot need to change the code below.x=Sampleimagesraw (); figure ('name','Raw Images'); Randsel= Randi (Size (x,2), $,1); % A random selection of samples forvisualizationdisplay_network (x (:, Randsel));%%================================================================%% Step 0b:zero-mean the data (by row)% you can make use of the mean and repmat/bsxfun functions.%--------------------YOUR CODE here--------------------x= X-repmat (mean (x,1), size (x,1),1);%%================================================================%%Step 1a:implement PCA to obtain Xrot% Implement PCA to obtain Xrot, the matrixinchwhich the data isexpressed% with respect to the eigenbasis of Sigma, which isThe matrix U.%--------------------YOUR CODE here--------------------%xrot = zeros (size (x)); % need to compute ThisSigma= X*x'./size (x,2);[U,s,v] =SVD (sigma); Xrot= u'* x;%%================================================================%%Step 1b:check Your implementation of PCA% The covariance matrix forThe data expressed with respect to the basis U% should is a diagonal matrix with non-zero entries only along the main% diagonal. We'll verify ThisHere .%Write code to compute the covariance matrix, Covar.% when visualised asAn image, you should see a straight line across the% Diagonal (non-zero entries) against a blue background (zero entries).%--------------------YOUR CODE here--------------------%covar = zeros (Size (x,1)); % need to compute ThisCovar= Xrot*xrot'./size (x,2);%visualise the covariance matrix. You should see a line across the%diagonal against a blue background.figure ('name','visualisation of covariance matrix'); Imagesc (Covar);%%================================================================%% Step2: Find K, the number of components to retain% Write code to determine K, the number of the retaininchOrder% to retain at least About%of the variance.%--------------------YOUR CODE here--------------------%k =0; %Set k Accordinglyeigenvalue=diag (covar);=sum (eigenvalue); Tmpsum=0; fork=1: Size (x,1) Tmpsum= tmpsum+eigenvalue (k); if(Tmpsum/total >=0.9)         Break; EndEnd%%================================================================%% Step3: Implement PCA with dimension reduction%Now so you had found K, you can reduce the dimension of the data by% discarding the remaining dimensions. Inch ThisYou can represent the% datainchK Dimensions instead of the original144, which'll save you%computational time running learning algorithms on the reduced%representation.% %following the dimension reduction, invert the PCA transformation to produce% The Matrix xhat, the dimension-reduced data with respect to the original basis.%visualise the data and compare it to the raw data. you'll observe that% there islittle loss due to throwing away the principal%correspond to dimensions with low variation.%--------------------YOUR CODE here--------------------%xhat = zeros (size (x)); % need to compute ThisXrot (k+1: Size (x,1), :) =0; Xhat= U *Xrot;%visualise the data, and compare it to the raw data%You should observe that the raw and processed data is of comparable quality.%For comparison, wish to generate a PCA reduced image which% retains only -%of the variance.figure ('name',['PCA processed Images', sprintf ('(%d/%d dimensions)', k, size (x,1)),"']);d isplay_network (Xhat (:, Randsel));'name','Raw Images');d isplay_network (x (:, Randsel));%%================================================================%%Step 4a:implement PCA with whitening and regularisation%Implement PCA with whitening and regularisation to produce the matrix%Xpcawhite.%epsilon =0; Epsilon=0.1;%xpcawhite =Zeros (size (x));%--------------------YOUR CODE here--------------------Xpcawhite= Diag (1./sqrt (diag (s) +epsilon)) * U'* x;%%================================================================%%Step 4b:check Your implementation of PCA whitening%Check Your implementation of PCA whitening with and without regularisation.%PCA Whitening without regularisation results a covariance matrix% that isequal to the identity matrix. PCA Whitening with regularisation% resultsincha covariance matrix with diagonal entries starting close to%1and gradually becoming smaller. We'll verify these properties here.%Write code to compute the covariance matrix, Covar.Without regularisation (SetEpsilon to0or close to0), % when visualised asAn image, you should see a red line across the%Diagonal (one entries) against a blue background (zero entries).%with regularisation, you should see a red line that slowly turns%blue across the diagonal, corresponding to the one entries slowly%becoming smaller.%--------------------YOUR CODE here--------------------Covar= Xpcawhite * Xpcawhite'./size (x,2);%visualise the covariance matrix. You should see a red line across the%diagonal against a blue background.figure ('name','visualisation of covariance matrix'); Imagesc (Covar);%%================================================================%% Step5: Implement ZCA Whitening%Now implement ZCA whitening to produce the Matrix Xzcawhite.%visualise the data and compare it to the raw data. You should observe% That whitening resultsinch, among other things, enhanced edges.%xzcawhite =Zeros (size (x)); Xzcawhite= U *Xpcawhite;%--------------------YOUR CODE here--------------------%visualise the data, and compare it to the raw data.%You should observe that the whitened images has enhanced Edges.figure ('name','ZCA whitened Images');d isplay_network (Xzcawhite (:, Randsel));'name','Raw Images');d isplay_network (x (:, Randsel));

"Deeplearning" EXERCISE:PCA and Whitening