How to visualize the output of the CNN layers in the Caffe

As examples of Caffe, CNN model is not a black box, Caffe provides tools to view all the outputs of the CNN layers

1. View the structure of the activations values for each layer of the CNN (i.e. the output of each layer)

The code is as follows:

# 显示每一层for layer_name, blob in net.blobs.iteritems():    print layer_name + ‘\t‘ + str(blob.data.shape)

The inner part of the second cycle body

• Layer_name extract is the name of the first layer of net
• Blob extraction is the output data of net layer I (4d)

The result is:

Data (50, 3, 227, 227) network input, Batch_number = 50, image 227*227*3 RGB image

CONV1 (50, 96, 55, 55) The output image size of the first conv layer is 55*55,feature the number of maps is 96

Pool1 (50, 96, 27, 27) The image size of the first pool layer is 27*27,feature map number is 96

Norm1 (50, 96, 27, 27) The image size of the first norm layer is 27*27,feature map number is 96

Conv2 (50, 256, 27, 27) The image size of the second conv layer is 27*27,feature map number is 256

pool2 (50, 256, 13, 13) The image size of the second pool layer is 13*13,feature map number is 256

Norm2 (50, 256, 13, 13) The image size of the second norm layer is 13*13,feature map number is 256

Conv3 (50, 384, 13, 13) The image size of the third conv layer is 13*13,feature map number is 384

conv4 (50, 384, 13, 13) The image size of the fourth conv layer is 13*13,feature map number is 384

conv5 (50, 256, 13, 13) The image size of the fifth conv layer is 13*13,feature map number is 256

Pool5 (50, 256, 6, 6) The image size of the fifth pool layer is 13*13,feature map number is 256

Fc6 (50, 4096)
The image size of the sixth FC layer is 4096

Fc7 (50, 4096)
The image size of the seventh FC layer is 4096

Fc8 (50, 1000)
The image size of the eighth FC layer is 1000

prob (50, 1000)
The size of the probablies layer is 1000

This data is drawn below to see more image

2. View the parametric structure of each layer

The code is as follows:

for layer_name, param in net.params.iteritems():    print layer_name + ‘\t‘ + str(param[0].data.shape), str(param[1].data.shape)

The inner part of the second cycle body

• Layer_name extract is the name of the first layer of net
• Param extract is the first layer of net parameter

The results are as follows:

CONV1 (96, 3, 11, 11) (96,) the size of the first conv layer of filters, where 3 is because the input layer of data is RGB, can be regarded as three feature maps

CONV2 (256, 48, 5, 5) (256,) The filters size of the second conv layer

CONV3 (384, 256, 3, 3) (384,) The filters dimension of the third conv layer

CONV4 (384, 192, 3, 3) (384,) The filters size of the fourth conv layer

CONV5 (256, 192, 3, 3) (256,) The filters size of the fifth conv layer

Fc6 (4096, 9216) (4096,) weight size of the first FC layer

FC7 (4096, 4096) (4096,) weight size of the second FC layer

FC8 (1000, 4096) (1000,) weight size of the third FC layer

It should be noted that since there are no parameters to optimize for the pool layer and the norm layer, there is no information on the pool layer and the norm layer in the parameters.

Here is a visual picture of how filters can filter the input data.

3. Functions for visualizing 4D data

def vis_square: # input data is a ndarray, size can be (n, height, width) or (n, height, width, 3) # The former is the data of n grayscale image, the latter is n RGB image Data # in a sqrt (n) by sqrt (n) grid, display each image # to the input image normlization data = (Data-data.min ())/(Data.max ()-Data.min  () # Mandatory to make the number of input images squared, less than the square number, manually add a few n = Int (Np.ceil (np.sqrt (data.shape[0))) # Each small image into a small gap between padding = (((0, n * * 2-data.shape[0]), (0, 1), (0, 1)) # Add some space between filters + ((0 , 0),) * (data.ndim-3)) # don ' t pad the last dimension (if there is one) data = Np.pad (data, padding, mode= ' constant  ', Constant_values=1) # pad with ones # speak all the input data images in a ndarray-data (tile the filters to an image) data = Data.reshape ((n, N) + data.shape[1:]). Transpose ((0, 2, 1, 3) + Tuple (range (4, Data.ndim + 1)) # A small example of data, e.g., (3 , 120,120) # that is, the data here is a 2d or 3d ndarray data = Data.reshape ((n * data.shape[1], n * data.shape[3]) + data.shape[4:    ] # Displays the image corresponding to dataPlt.imshow (data); Plt.axis (' off ')

4. View the image of the first convolution layers filters

The code is as follows:

# the parameters are a list of [weights, biases]filters = net.params[‘conv1‘][0].datavis_square(filters.transpose(0, 2, 3, 1))

• Filters stores the data for the first conv layer of filters
• Example: Shape: (96, 3, 11, 11)

The results are as follows: A total of 96 images (96 filters, each filters is 11*11*3)

5. View an image of the first convolution layers output (activations)

The code is as follows:

feat = net.blobs[‘conv1‘].data[0, :36]vis_square(feat)

• Feat storage is the first 36 images of a volume base
  The size of the feat is (36, 55, 55)

The results are as follows:

6. View the output image of the POOL5

The code is as follows:

feat = net.blobs[‘pool5‘].data[0]vis_square(feat)

• Feat storage is the output image of POOL5
  The size of the feat is (256, 6, 6)

The results are as follows:

7. Output of the first fully connected layer

The code is as follows:

feat = net.blobs[‘fc6‘].data[0]plt.subplot(2, 1, 1)plt.plot(feat.flat)plt.subplot(2, 1, 2)_ = plt.hist(feat.flat[feat.flat > 0], bins=100)

The results are as follows:

8. Output of the probability layer

The code is as follows:

feat = net.blobs[‘prob‘].data[0]plt.figure(figsize=(15, 3))plt.plot(feat.flat)

The results are as follows:

9. Summary

After NET has been acquired:
How to get data for each layer of the network: net.blobs[' layer name '].data[0]
How to get parameter data for each layer of the network: net.params[' layer name '][0].data

How to visualize the output of the CNN layers in the Caffe