C + + convolutional Neural Network example: TINY_CNN Code specific explanation (6)--average_pooling_layer Layer Structure class analysis

In the previous blog post, we focused on the code structure of the Convolutional_layer class. In this blog post, we analyze the corresponding bottom-layer Average_pooling_layer class:

First, the role of the next sample layer

The function of the lower sample layer is theoretically two, mainly descending dimension, and secondly, improving the robustness of a little feature. In the LENET-5 model. Each convolution layer is followed by a bottom-like layer:

The reason is when the image is after the convolution layer. Because each convolution layer has multiple convolution templates, the feature matrix that directly results in the convolution result is increased several times in relation to the input data matrix. Plus there are a number of convolutional layers (some of Google's models are even more than 100 layers), without dimensionality reduction operations. The last feature dimension can be imagined, so the sample layer here acts as a feature-reducing role. For example, here is the 2*2 template mean under the sample, it is clear that four adjacent pixels by the weighted calculation into a, the dimension becomes the original four points in one, the reduction of dimensional performance is obvious.

As for the robustness problem, theoretically only reasonable dimensionality reduction means. There will be some improvement in robustness (this is my personal point of view, of course).

Second, Average_pooling_layer class structure

The Average_pooling_layer class inherits from the base class Partial_connected_layer, and the Convolutional_layer class described earlier is the same inheritance hierarchy, so with Convolutional_ The layer class is similar in structure, but the bottom layer is relatively simple because of its function, so it is much more concise than the Convolutional_layer class in structure.

2.1 Member variables

The member variables of the Average_pooling_layer class have two, each of which holds the attributes of the input data matrix and the output feature matrix after the following sample:

As for the layer_size_t, Index3d and other types of knowledge in the previous blog has been specifically introduced, here no longer repeat.

2.2 Constructors

The Average_pooling_layer class's constructor is very concise, basically called the constructor of the base class Partial_connected_layer, and then completes the initialization of two member variables in its own class:

Here's a little bit about the Pooling_size_mismatch () function. This is a function defined in the base class layer, which is the information that throws the dimension exception, and the test criteria for size matching in each layer are different. Therefore, when calling this function to give the exception information, the sub-class needs to be inferred according to the standards of the corresponding layer, for example, in the next sample layer, we think that the size of the input data matrix is not the next sample form of the integer times, it is not normal to block the dimension, that is, size mismatch. Throw Dimension Exception Information:

2.3 Other functions

In addition to the constructor, the following sample layer also provides some other functions, including the following sample matrix weight initialization function and the biased initialization function (the same need to be biased in the process of the next sample) Init_connection (), the weight matrix setting function Connect_kernel (), As well as the output feature matrix image conversion function output_to_image (). These functions are identical to the corresponding function functions in the Convolutional_layer class in the previous blog post (after all, inherit from the same base class). Don't repeat it here.

Third, the matters needing attention

1, the activation function after the sample layer

In the actual convolutional neural network model, after the convolution feature output is taken down, it should be fed into the activation function, in other words, the bottom layer and the activation function should be closely related. Just here the author does not place the activation function directly in the Average_pooling_layer class, but instead encapsulates each activation function into the corresponding class and puts it under the activation namespace:

2, LeNet-5 model of the classical

The LENET-5 model has been used in the introduction of the convolutional extension neural network model, mainly because of its simple structure and the classic of CNN, but the number of layers in the actual application of convolutional neural networks can be more than that. For example, the 2014 VGG model has extended the network to a depth of 16 layers, which is unclear as to how many layers of deep-learning institutes have deepened the network. In short, a two-block Titan-level n card is not enough.

C + + convolutional Neural Network example: TINY_CNN Code specific explanation (6)--average_pooling_layer Layer Structure class analysis