C + + convolutional Neural Network example: TINY_CNN code detailed (11)--Layer structure container layers class source analysis

In this blog post we briefly analyze the class--layers of the last network structure in the TINY_CNN convolutional neural network model.

First of all, layers can be called a layer structure of the vector, that is, the layer structure of the container. Because convolutional neural network is a multi-layered network model, it is necessary to manage all the layers in the network, which leads to the layers class which is introduced in this blog post. The layers class is a vector-type variable in which the element being pressed is the model of each layer in the network, where a simple structure diagram is given, at a glance:

From which we can see clearly the vector structure of the layers, is plainly a layer structure of the container, between each other through the prev_ (refers to the back layer) and next_ (refers to the forward layer) two pointers to contact. The next step is to focus on the internal code structure of layers. First, the structure of the layers is given:

We also interpret the structure and code of this class in terms of member variables and member functions.

First, member variables

Layers has two member variables, one is the vector container variable, which is used to hold each stack of the layer structure class; one is the Input_layer type variable that holds the structure information for the input layer in the network model:

Here are two questions to emphasize:

(1) The type of the container element. It can be seen that the basic element type in the Layers_ container is the layer_base* type, the reason is very clear, layer_base is the convolution layer, the lower sampling layer, the whole layer structure of the common base class, declared as a pointer form is to facilitate the use of forward pointers and reverse pointers to connect.

(2) The uniqueness of the input layer. The input layer is taken out separately as a member variable, mainly considering that the input layer is essential for any convolutional neural network model (whether it is a single-layer model or a multilayer model), and that it is in the forefront of the network model. For a layer member that has been identified for this existence and location, it is chosen to place it in layers for default additions, allowing the user to specify a layer structure model without having to add the input layer in a constant setting.

Second, the structure function

The constructors for two functions are available in the layers class:

The first constructor is used to construct a single-layer network model, where there is only one input layer (First_ is a input_layer type variable), and the second constructor is used to construct a multilayer network model according to the specified layer type. The implementation mechanism of the constructor is mainly through the Add () and construct () two functions, the specific functions of the two functions are described later.

Three, model construction method

Layers is mainly dependent on the two construction methods of Add () and construct () in the process of constructing the network model, with a little analysis here.

3.1 Add () function

As the name implies, the function of the add () function is to add the specified layer type class to the current layers struct, similar to the stack operation in the vector (indeed, the push_back is actually borrowed):

The Add () function is mainly divided into two parts in the process of execution, that is to establish the connection relation and stack of layer structure. You first need to call the Connect () function to establish the relationship between the layer to be joined (New_tail) and the existing model. The Connect () function is a member function defined in the base class layer_base to establish a pointer contact between the front and back two layers in the network model:

In the Connect function, we first judge the connection relationship between the front and back layers, and in the case of connection matching, the next layer of the current layer is specified as tail (the newly added layer) through the "Next_ = tail" statement, and then through "TAIL->PREV_ = this" Specifies the previous layer of the newly added layer as the current layer. In this sense, connect () is constructed in much the same way as the pointer queue.

Continue parsing the Add () function. After a new join layer is established through the Connect function and the pointers between the last layers of the existing model are contacted, call Push_back to stack the layer structure class to be added and complete the construction and storage of the new layer.

3.2 Construct () function

The construct () function is designed to complete the construction of multi-layered models, including the establishment of connections and compression stacks, and the basic principle is to loop the call to the Add () function:

This can also be divided into two processes. First, add the input layer first_ to the layers container, because the input layer is essential for all models, so instead of letting the user write Input_layer in the first place each time the model is built, it's better to put this step inside the constructor to automate it. The next step is to call the Add () function, which will be added to the layer one by one to build the connection, stack, build connection, press stack.

Four, the weight operation

Since layers belongs to the re-encapsulation of layer structure classes, it is necessary to provide interfaces that correspond to the common functions of each network layer, such as weight manipulation. The weight operations here mainly include initialization of weights, resetting and updating of weights. These operation functions are generally provided only one interface, the implementation process is mainly called the corresponding functions within each class, such as the initialization and reset of weights:

Again, for example, the weight update:

Five, property return

Since layers is the topmost layer package that users can reach, it is necessary to provide some interface functions returned by the user to view the output of the network layer and the specifics of the weight kernel, and so on, of course, the interface function here is also as a bridge-type existence, on the one hand to connect users, On the one hand, the corresponding function function of each element class is called. In addition, we need to add some property return functions related to the overall layer structure to tell the user how many layers the layer_base vector actually has, what the first layer is, what the last layer is, and how to access the specified layer.

First of all, how to access the first and last layer, here are two functions head () and tail (), the implementation mechanism is very simple, I believe you can understand:

As for how to access the specified layer, TINY_CNN provides two means, one is to define the at function and type conversion through dynamic_cast:

Another method is to overload the "[]" operation, and to access the array as a class

The above two methods of access are indexed (index) to complete, more convenient.

OK, about the layer structure container layers class source first introduced here, there are some patches of small functions, also on the one or two lines of code, the function is clear, the implementation is simple, here will not repeat, there are three months on the study of three, to think about the work of the matter, hey.

C + + convolutional Neural Network example: TINY_CNN code detailed (11)--Layer structure container layers class source analysis