Convolution neural network for picture classification-Next

Next: convolutional neural network for image classification-medium

9 ReLU (rectified Linear Units) Layers

After each convolutional layer, an excitation layer is immediately entered, and an excitation function is called to add the nonlinear factor, and the problem of linear irreducible is rejected. Here we choose the method of excitation function called Relu, his equation is so f (x) = max (0, X), that is, the value of less than 0 is classified as 0, the benefit is that the network training faster, reduce the problem of gradient disappear. How to understand, for example:

A can be used to distinguish 2 different types of data points in a straight line, but B is not possible. However, the invocation of the excitation function after transformation can be a good distinction between them.

10 Training

So far there have been questions about how convolutional layers know what feature images to extract, and how the weight values in the filter are compared if they are determined.

Let's take a look at how the network is trained. In fact, the network in the course of training the computer through the reverse propagation algorithm to adjust the filter weight value. Let's look at an analogy before explaining the inverse propagation algorithm.

When we were born, our minds were brand new, and we didn't know what a bird or a dog was. Similar to CNN before the start of training, the weight value is also randomly initialized, the filter does not know which eigenvalues to extract. When we grow up, parents and teachers show us different pictures and tell us what these pictures correspond to. The idea or method is to label the picture, which is the course of CNN training. Before CNN training, actually we had some training samples, there are thousands of cats, dogs, birds, boat pictures, and each picture has a corresponding tag to explain what animal it is, for example, a training picture is labeled as a dog, its label is [0 0 0 1 0 0 0 0 0 0], The corresponding position of the classification that belongs to it is marked as 1, and the other classification locations that do not belong to it are marked with 0 values.

The inverse propagation algorithm can be divided into 4 different parts, forward transmission, loss function, reverse transfer, update weights. In the process of forward propagation, you get an array of training pictures 28x28x3, passing it through the network. At first, all the weights are randomly initialized, and they may be random values [0.3, 0.1, 0.4, 0.2, 0.3 ...]. The network through the initialization of the weight value, is unable to extract the exact feature image, so can not give any reasonable conclusion, the picture belongs to what kind of category. At this point we need the loss function in the reverse propagation to help the network update the weight value to find the desired feature image. There are many ways to define loss functions, but one common way is the MSE (mean squared erro) mean square error.

Here target is the real classification value of the picture, output is the classification value that the picture trains through the network, then calls the mean square error to obtain the loss value. This loss value may be very high at the beginning of the network training, which is well understood because the weight values are randomly initialized. Our ultimate goal is to get the same predicted value as the real value. In order to achieve this, we need to minimize the loss value, the smaller the loss value, the closer the prediction results. In this process, we need to constantly adjust the weight value, to find out which weight value can reduce the loss of the network. How to find these weights, you need to use the gradient descent algorithm. An example of this is shown below:

At first, there is a point on the surface, this point is our initialization weight, and then constantly update the weight value, so that the point along the edge of the surface slowly down, and finally to the lowest level, to get a small loss value. During training, you can specify a descent rate or a training rate, which is the pace at which the point moves each time. Giving a large training rate means that the weights are bigger at the time of the update, and the training is faster, but when you give the value too large it is possible to jump over it when it reaches the lowest point, so that you don't get the best weight value.

The process of doing a forward pass, a loss function, a reverse pass, and a parameter update is often referred to as one epoch. Each training will specify the number of times to complete the epoch, and then the program will repeat the process, and when the last round of training is finished, our network is trained to get some weights to train.

11 Testing

Finally, in order to detect the accuracy of the trained CNN model, we need some different pictures and tags to pass them on to CNN, and we can use the predicted results to compare with the actual results to see if the trained model is accurate.

Finish

Reference:

Http://neuralnetworksanddeeplearning.com/chap6.html#convolutional_neural_networks_in_practice

Https://adeshpande3.github.io/adeshpande3.github.io/A-Beginner ' s-guide-to-understanding-convolutional-neural-networks/

Convolution neural network for picture classification-Next