A study record of CNN convolutional Neural Network

1. Overview

Convolution neural network features: On the one hand, the connection between the neurons is non-fully connected, on the other hand, the weights of the connections between some neurons in the same layer are shared (i.e. the same).

Left: The image has 1000*1000 pixels, there are 10^6 of hidden layer neurons, to be fully connected, there are 1000*1000*100000=10^12 weight parameters

Right: There are also 1000*1000 pixels, feel the wild for 10*10, each node and the upper node of the same position Attachment 10x10 window is connected, there is a 10*10*10^6=10^8 weight parameter

We can easily calculate the output of a network node. For example, for a net input that is labeled as a red node, it is equal to the sum of the product of the weight of the previous neuron node value and the red line that are connected to the red wire. This process of calculation, many books are called convolution.

convolutional neural network Another feature is weight sharing. For example, on the right-hand side of the graph, the weights are shared, which means that all red line labels have the same connection weights. That is, each node is equal to the value of the connection weights of the pixels within the sensing field.

2. The structure of CNN

convolutional Neural Network is a multilayer neural network, each layer is composed of several two-dimensional planes, and each plane consists of several independent neurons.

: Convolution neural Network concept demonstration: the input image through and three can be trained filter and can be offset to the convolution, after convolution in the C1 layer generated three feature map, then the feature map in each group of four pixels to sum, weighted value, offset, A feature map of three S2 layers is obtained through a sigmoid function. These maps are then filtered to get the C3 layer. This hierarchy produces S4 as well as S2. Eventually, these pixel values are rasterized and connected into a vector input to the traditional neural network, resulting in output.

Generally, the C layer is a feature extraction layer, each neuron's input is connected to the local sensation field in the previous layer, and the local characteristics are extracted, and once the local feature is extracted, the position relationship between it and other features is determined; s layer is the feature map layer, and each computing layer of the network is composed of multiple feature mappings. Each feature is mapped to a plane, and the weights of all neurons on the plane are equal. The feature mapping structure uses the sigmoid function which affects the function core as the activation function of convolutional network, which makes the feature map have displacement invariance.

In addition, due to the sharing weights of neurons on a mapped surface, the number of free parameters is reduced and the complexity of network parameter selection is reduced. Each feature extraction layer (c-layer) in convolutional neural network is followed by a computing layer (S-layer) for local averaging and two extraction, and this unique two-time feature extraction structure makes the network more tolerant to the input sample when it is recognized.

The core idea of convolutional networks is to combine the three structural ideas of local sensation field, weighted value sharing (or weight reproduction) and time or spatial sub-sampling to obtain some degree of displacement, scale and deformation invariance.

3. CNN Training

The training algorithm is similar to the traditional BP algorithm. It consists of 4 steps, and these 4 steps are divided into two stages:

The first stage, the forward propagation phase:

A) Take a sample (X,YP) from the sample set and input X into the network;

b) Calculate the corresponding actual output op.

At this stage, the information is transferred from the input layer to the output layer through a gradual transformation. This process is also the process that the network executes when it is running properly after the training is completed. In this process, the network performs a calculation (in effect, the input is multiplied by the weight matrix of each layer, resulting in the final output):

OP=FN (... (F2 (F1 (XpW (1)) W (2)) ... ) W (n))

Second stage, backward propagation phase

A) calculates the difference between the actual output op and the corresponding ideal output YP;

b) The inverse propagation of the adjustment weight matrix by minimizing the error.

Reference: http://www.cnblogs.com/nsnow/p/4562363.html

A study record of CNN convolutional Neural Network