Summarization of convolution algorithm (shallow knowledge)

Common algorithms for inductive convolution neural networks
Before the convolution algorithm, there are many image classification and recognition of machine learning algorithms, such as SVM vector machine principle is particularly complex, convolution algorithm is relatively understandable, on the one hand to avoid the full connection of the huge parameters, mainly by extracting eigenvalues, algorithm accuracy rate is also the highest, almost can be compared with artificial recognition.

Classic algorithms:

1,lenet algorithm:

LeNet the flow of the algorithm is:

Input–>conv2->relu->pool->conv2->relu->pool->cf->softmax->output

Input: Enter; The matrix is (W,H,C) W: wide pixels; h: high pixels; C: Channel

Conv: Convolution calculation

Convolution is to put a layer (W,H)

Conv2: Represents the convolution of a two-dimensional graph

A layer may be 64*64, with a W filter layer (6,6) usually have more than one filter layer , the output layer is a matrix dimension of 64-6+1=59, ie (59,59)

Take the following diagram as a simple example:

Pool: Is the pooling operation, may translate the reason to say differently, the pool layer is a single layer , the pool layer is divided into the maximum pooling and the average value pooling; in fact, that is, the most obvious value of extracting eigenvalues, the maximum pooling effect is relatively better, so it is recommended to use.

FC: That is all connected, namely Wx+b=y

Softmax: Normalization of treatment, classification

In the process of conv2 to pool, there is a layer of no_linear_normal operations, but also difficult to understand, the role does not seem so big, it is ignored.

2,alexnet algorithm:
is actually a lenet deformation, first look at a picture:

The general process can be divided into:

Input->conv2->pool->conv2->pool->pool->pool->pool->pool->fc->fc->fc-> Softmax

Of course there are some relu. The intensification function is skipped, and local normalization is done through LRN before activating the function, reducing the use of complex intensification functions.

3,VGG-16 algorithm:

VGG-16 and VGG-19 algorithm is mainly in the level of how much difference, the algorithm effect is similar, so generally recommended to use VGG-16

The biggest difference between the VGG algorithm and alexnet is that the algorithm focuses on conv2 convolution multiple times to achieve the purpose of centralized feature collection.

==========================================================

Efficient algorithm:
Residual error algorithm:resnet algorithm

Convolution algorithm, flow is through a plurality of filters to extract the eigenvalues, the middle process is prone to more layers, the loss of the original feature is more serious, loss function in reverse optimization gradient drop too fast, so need to add some feature optimization to reduce this speed. For some neural depths, the algorithm is more advantageous.
First comes the general neural depth process:
Input–>linear->rule->linear->relu->a
And ResNet is a step.
Input–>linear->rule->linear->? ->relu->a
Operation Process:

Next is:?

No, the larger the network, the higher the probability of the error, so the fidelity:

Take the a[l] and add it in to get the activation function so that it can be guaranteed. Look at one more picture:

The overall process

This is not a real jump, but rather the original a value for the activation function.

Inception Network Algorithm
In fact, Google is a convolution algorithm googlenet, the algorithm is really more responsible, of course, his depth can be very deep, but also by increasing the conv2 level of convolution, to identify a picture of many things; first, let's start with a picture:

Previous layer is after the previous activation function
Then the CONV2 or pool is carried out, then the convolution of each layer is composed into a large convolution.
An auxiliary convolution can be used to identify more objects.

FaceNet algorithm is character recognition

Character recognition can often be used to monitor, or to identify a level or work face, by convolution and then calculating the error based on the characteristics of the CF after the convolution, often requiring more than 10 images of a character as training, and then looking at other images as a comparison.

My own picture of the calculation error is less than equal to other people's picture error

Here is just as a summary of learning, concrete implementation to lenet-5,googlenet,inception Network, facenet and other keywords to githup search the corresponding open source code to study; These are open source, at least you can find his paper, But almost all of them are in English. That's the original.

Reference: Wunda Video Tutorial

Summarization of convolution algorithm (shallow knowledge)