Parameters of the convolution layer and the pooling layer

1, convolution layer:

Is the convolution layer, which is the core layer of convolutional neural Network (CNN).

Layer Type: Convolution

Lr_mult: The coefficient of the learning rate, the final learning rate is the number multiplied by the BASE_LR in the Solver.prototxt configuration file. If there are two Lr_mult, then the first one represents the weight of the learning rate, the second one indicates the bias of the learning rate. The learning rate of the general bias is twice times that of the weight learning rate.

In the later Convolution_param, we can set the specific parameters of the convolution layer.

Parameters that must be set:

num_output: Number of convolution cores (filter)

Kernel_size: The size of the convolution core. If the convolution cores are of length and width, they need to be set separately with Kernel_h and Kernel_w

Other parameters:

Stride: The step size of the convolution core, which defaults to 1. You can also use Stride_h and Stride_w to set.

Pad: Expand Edge, Default is 0, do not expand. When the expansion is left and right, symmetrical, such as the size of the convolution kernel 5*5, then pad set to 2, four edges are expanded 2 pixels, that is, the width and height of the expansion of 4 pixels, so that the convolution operation after the feature map will not be smaller. can also be set separately by Pad_h and Pad_w.Weight_filler: Initialization of weight value. The default is "constant", the value is all 0, many times we use the "Xavier" algorithm to initialize, can also be set to "Gaussian" Bias_filler: The initialization of the offset. The general setting is "constant" and the value is all 0. Bias_term: Whether to turn on bias, default to True, turn onGroup: Group, default to 1 groups. If it is greater than 1, we limit the connection operation of the convolution within a subset.   If we are grouped according to the channel of the image, then the I output group can only be connected with the input group I.  Input: n*c 0*w 0*h 0 Output: n*c 1*w 1*h 1 wherein, C 1 is the num_output of the parameter, the number of characteristic graphs generated w 1= (w 0+2*pad-kernel_size)/stride+1; H 1= (h 0+2*pad-kernel_size)/stride+1; If you set stride to 1, there is overlap between the front and back two convolution parts. If you set pad= (kernel_size-1)/2, the width and height will not change after the operation. Example: Layer {
Name: "Conv1"
Type: "Convolution"
Bottom: "Data"
Top: "Conv1"
param {
Lr_mult:1
}
param {
Lr_mult:2
}
Convolution_param {
Num_output:20
Kernel_size:5
Stride:1
Weight_filler {
Type: "Xavier"
}
Bias_filler {
Type: "Constant"
}
}
}

2, pooling layer is also called the pool layer, in order to reduce the computation and data dimensions of a layer set. Layer Type: Pooling The parameter that must be set: kernel_size: The kernel size of the pool. can also be set with Kernel_h and Kernel_w respectively. Other parameters: Pool: Pooling method, default is Max. Currently available methods are Max, AVE, or stochastic pad: As with the convolution layer of the pad, the edge is expanded. The default is 0 stride: The pooled step size, which defaults to 1. Generally we set to 2, that is, do not overlap.  You can also use Stride_h and Stride_w to set. Example: Layer {
Name: "Pool1"
Type: "Pooling"
Bottom: "Conv1"
Top: "Pool1"
Pooling_param {
Pool:max
Kernel_size:3
Stride:2
}
}

The pooling layer is basically the same as the convolution layer.  Input: n*c*w 0*h 0 Output: n*c*w 1*h 1 and convolutional layer The difference is that the C remains unchanged W 1= (w 0+2*pad-kernel_size)/stride+1; H 1= (h 0+2*pad-kernel_size)/stride+1; If you set stride to 2, the convolution portions of the front and back two do not overlap. The 100*100 is converted into 50*50 after pooling the feature map. 3. Local Response Normalization (LRN) layer This layer is normalized to an input local area to achieve the effect of "side inhibition". can go to search alexnet or googlenet, it uses this function 3 layer type: LRN parameter: All optional, no must local_size: default is 5. 　　If it is a cross-channel LRN, the number of channels to sum, or if LRN is within the channel, represents the length of the square area of the sum. 　　Alpha: Defaults to 1, normalized parameters in the formula. 　　Beta: defaults to 5, normalized parameters in the formula. Norm_region: Default is Across_channels. There are two choices, and Across_channels represents the summation of the adjacent channels. The Within_channel represents a summation normalization within a specific area within a channel.   Corresponds to the previous local_size parameter. Normalization formula: For each input, to divide, to get the normalized output example:

Layers {
  Name: "Norm1"
  type:lrn
  Bottom: "pool1"
  Top: "Norm1"
  lrn_param {
    local_size:5
    alpha:0.0001
    beta:0.75
  }
}

4, Im2col layer

If you are familiar with MATLAB, you should know what Im2col means. It first divides a large matrix into multiple sub-matrices, serializes each sub-matrix into vectors, and finally gets another matrix.

Take a look at the picture to find out:

In Caffe, the convolution operation is the im2col operation of the data before the inner product operation (inner product). Doing so is faster than the original convolution operation.

Look at the similarities and differences between the two convolution operations: