Ufldl learning notes and programming assignments: Feature Extraction Using Convolution and pooling (convolution and pooled feature extraction)

Ufldl learning notes and programming assignments: Feature Extraction Using Convolution and pooling (convolution and pooled feature extraction)

Ufldl provides a new tutorial, which is better than the previous one. Starting from the basics, the system is clear and has programming practices.

In the high-quality deep learning group, you can learn DL directly without having to delve into other machine learning algorithms.

So I started to do this recently. The tutorial, coupled with Matlab programming, is perfect.

The address of the new tutorial is: http://ufldl.stanford.edu/tutorial/

Link: http://ufldl.stanford.edu/tutorial/supervised/FeatureExtractionUsingConvolution/
Http://ufldl.stanford.edu/tutorial/supervised/Pooling/
Http://ufldl.stanford.edu/tutorial/supervised/ExerciseConvolutionAndPooling/


Convolution: The conv2 function of MATLAB is used, which is somewhat frustrating here. Conv2 calculates convolution in the mathematical sense, and the function will flip the filter 180 internally. In fact, we do not want to calculate convolution in the mathematical sense, but simply calculate the "inner product ", point-to-Point multiplication and then summation. Therefore, we must first flip the filter and pass it to conv2 to achieve our goal. In fact, I think, in fact, it does not affect the final result because W is adjusted after all.
Pooling: The pooling step here is equal to that of pooldim and will not overlap. Here conv2 is used to calculate the mean value, which can optimize the performance. Remember, you do not need to activate the function here !!!
This exercise is relatively simple. However, a few MATLAB functions should be summarized as follows:
Conv2 perform convolution squeeze to remove the one with only one dimension
Rotate 90 degrees
Reshape dimension transformation


Running result:
The exercise here is mainly to check whether the two written functions are correct.

The following is the main code:
Cnnconvolve. m

Function convolvedfeatures = cnnconvolve (filterdim, numfilters, images, W, B) % cnnconvolve returns the convolution of the features given by W and B with % the given images % parameters: % filterdim-filter (feature) dimension % numfilters-number of feature maps % images-large images to convolve with, matrix in the form % images (R, C, image number) % ------------- note the dimension LOCATION % W, B-W, B for features fro M The sparse autoencoder % W is of shape (filterdim, filterdim, numfilters) % B is of shape (numfilters, 1) % returns: % convolvedfeatures-matrix of convolved features in the form % convolvedfeatures (imagerow, imagecol, featurenum, imagenum) % ---------- note the dimension position numimages = size (images, 3 ); imagedim = size (images, 1); % number of rows, that is, the height. The width is not calculated here. It seems that the height and width are equal by default. Convdim = imagedim-filterdim + 1; % convolvedfeatures = zeros (convdim, convdim, numfilters, numimages); % instructions: % convolve every filter with every image here to produce the % (imagedim-filterdim + 1) x (imagedim-filterdim + 1) x numfeatures x numimages % matrix convolvedfeatures, such that % convolvedfeatures (imagerow, imagecol, featurenum, imagenum) is the % value of the convolv Ed featurenum feature for the imagenum image over % the region (imagerow, imagecol) to (imagerow + filterdim-1, imagecol + filterdim-1) % expected running: % convolving with 100 images shocould take less than 30 seconds % convolving with 5000 images shocould take around 2 minutes % (so to save time when testing, you shoshould convolve with less images, as % described earlier) for imagenum = 1: numim Ages for filternum = 1: numfilters % convolution of image with Feature Matrix convolvedimage = zeros (convdim, convdim); % obtain the feature (filterdim x filterdim) needed during the convolution % Your code here % filter = W (:,:, filternum); % flip the Feature Matrix because of the definition of convolution, as explained later filter = rot90 (squeeze (filter), 2); % squeeze removes the one dimension with only one dimension. Here we remove the third-dimensional form and change the third-dimensional form to the second-dimensional form. % Obtain the image im = squeeze (images (:,:, imagenum); % convolve "filter" with "im ", adding the result to convolvedimage % Be sure to do a 'valid' convolution % Your code here % convolvedimage = conv2 (IM, filter, "valid "); % with the valid parameter added, the following code does not need. % Conv2dim = size (bytes, 1); % im_start = (conv2dim-convdim + 2)/2; % im_end = im_start + convDim-1; % convolvedimage = convolvedimage (im_start: im_end, im_start: im_end); % get the Middle Part % Add the bias unit % then, apply the sigmoid function to get the hidden activation % Your code here % convolvedimage = convolvedimage. + B (filternum); convolvedimage = sigmoid (convolvedimage); convolvedimage = reshape (convolvedimage, convdim, convdim, 1, 1); % 2 dimension change dimension 4 dimensions (:, :, filternum, imagenum) = convolvedimage; endendend

Cnnpool. m

Function features = cnnpool (volumes) % cnnpool pools the given convolved features % parameters: % pooldim-dimension of pooling region % volumes-convolved features to pool (as given by region) % outputs (imagerow, imagecol, featurenum, imagenum) % returns: % pooledfeatures-matrix of pooled features in the form % pooledfeatures (poolrow, poolcol, featurenum, imagenum) % numimages = size (milliseconds, 4); numfilters = size (convolvedfeatures, 3); convolveddim = size (milliseconds, 1); pooledfeatures = zeros (convolveddim/pooldim ,... convolveddim/pooldim, numfilters, numimages); % instructions: % Now pool the convolved features in regions of pooldim x pooldim, % to obtain the % (convolveddim/pooldim) X (convolveddim/pooldim) x numfeatures x numimages % matrix pooledfeatures, such that % pooledfeatures (poolrow, poolcol, featurenum, imagenum) is the % value of the featurenum feature for the imagenum image pooled over the % corresponding (poolrow, poolcol) Pooling region. % use mean pooling here. % Your code here % filter = ones (pooldim); For imagenum = 1: numimagesfor filternum = 1: numfiltersim = squeeze (convolvedfeatures (:,:, filternum, imagenum); % it seems that squeeze cannot or pooledimage = conv2 (IM, filter, "valid"); pooledimage = pooledimage (1: pooldim: end, 1: pooldim: end); % get the middle part of pooledimage = pooledimage. /(pooldim * pooldim); % pooledimage = sigmoid (dimensions); % do not need sigmoid pooledimage = reshape (pooledimage, dimensions/pooldim, dimensions/pooldim, 1, 1 ); % 2-dimension pooledfeatures (:,:, filternum, imagenum) = pooledimage; endendend

This article linger link: http://blog.csdn.net/lingerlanlan/article/details/38502627