Wunda Deep Learning course4 convolutional neural network

1.computer Vision

CV is an important direction of deep learning, CV generally includes: image recognition, target detection, neural style conversion

Traditional neural network problems exist: the image of the input dimension is larger, as shown, this causes the weight of the W dimension is larger, then he occupies a larger amount of memory, calculate W calculation will be very large

So we're going to introduce convolutional neural networks

2.Edge Detection Example

Neural network from shallow to deep can detect image edge features, local features, the overall characteristics

The Edge feature is an example of vertical edge detection. As shown, a 6x6 gray image convolution a filter, a new picture is a vertical edge detection, wherein,∗ represents the convolution operation. In Python, convolution is denoted by Conv_forward (), TensorFlow, convolution is represented by tf.nn.conv2d (), and convolution is denoted by conv2d () in Keras.

3.More Edge Detection

Image edge Detection There are two ways of gradient, a dark from the Ming side, a dark variable, as shown below

In practice, the absolute value of the output image, the gradient mode will not affect what

Horizontal Edge Detection Examples:

In addition to the filter, the other common filter is the Sobel filter and the ScHARR filter, as follows:

In practical applications, we will detect more features of the edge, not just vertical and horizontal features, filter parameters are trained, similar to the standard neural network in the weight of W as the gradient descent algorithm iterative iteration, to determine the edge features, CNN can be a detection of all the edge features

4.Padding

As previously mentioned, if a picture of NXN, filter is FXF, then the volume of the resulting image size is n-f+1, after convolution will appear two questions:

The picture becomes smaller and the edge information of the picture itself is lost.

To solve these two problems, we can make a fill on the image, as follows: the partial pixels of the fill are 0

Assuming that the P-width is filled, the resulting convolution after padding is (n+2p-f) +1 X (n+2p-f) +1

Valid convolutions: Do not fill the picture

Same convolutions: That is, make (n+2p-f) +1=n ==> p= (f-1)/2, so that the resulting convolution after the picture is the same size as the original picture, F is usually an odd number (odd), the reason may be:

Make P an integer, filled with a symmetric fill

For odd times the filter has a central position

5.Strided convolutions

The step length is only the time of each convolution, as follows:

When the step is s, the resulting picture size after convolution is (n+2p-f)/s +1 X (n+2p-f)/s +1, if the result is not an integer, then rounding down, the convolution will be out of the part of the picture, such as the picture in the X part

The convolution we're talking about actually calculates a correlation coefficient, which is not the same as the convolution in the mathematical sense, and the convolution in the mathematical sense spins a filter, and our process does not, in deep learning, is not important for the filter rotation.

6.Convolutions over volumes

For a picture with r,g,b three color channels, the last parameter in the filter should be consistent with the number of color channels when a convolution is performed on it.

The convolution operation of the 3-channel picture is basically consistent with the convolution operation of the single-channel picture. The process is to sum each single channel (R,G,B) with the corresponding filter, and then add 3 channels and add a pixel value to the output image.

The filter operators of different channels can vary. For example, the R channel filter implements vertical edge detection, the G and B channels do not edge detect, all zero, or the R,g,b three channel filter is all set to horizontal edge detection.

Refers to the case of only one filter, in order to perform multiple convolution operations, to achieve more edge detection, you can add more filter groups. For example, setting the first filter group for vertical edge detection and the second filter bank for horizontal edge detection. In this way, different filter group convolution gets different output, the number is determined by the filter group. , the result should be as follows

Assuming that the color channel for the picture is Nc,n x N x NC, and F x F x NC, the resulting picture is (n-f + 1) x (n-f + 1) x n ' C n ' c is the number of filters

7.One layer of a convolutional network

The single-layer structure of convolutional neural networks is as follows:

The parameters in the total (3x3x3+1) x2, the number of parameters independent of the size of the picture, only with the filter

Summarize all the marker symbols for the CNN single-layer structure and set the number of layers to l .

• F [ l ] >f < Span id= "mathjax-span-180" class= "Mrow" >[ l]    = filter size

• p [ l ] "> p  [l ]   = padding

• s [ l ] "> s  [l ]    = stride

• n [l] C = Number of filters

8.A Simple Convolution Network example

A simple example of CNN is as follows:

The first few layers are convolution operations, n[h] and n[w] gradually reduced, and n[c] gradually increase, the last layer, the picture into a vector (7x7x40,1) vector, and the last layer of the output layer connected

CNN has three types of layers:

9.Pooling Layers

The pooling layer is used to reduce the size, increase the speed of operation, reduce noise and make the model more robust.

The practice of Pooling layers is much simpler than convolution layers, there is no convolution operation, only the maximum value in the sliding area of the filter operator, that is, Max Pooling, which is the most common practice. Note that the hyper-parameter p is rarely used in pooling layers.

Pooling layers only the two parameters F and S of the filter, and they are fixed and do not need to be trained, so the computational amount is very small.

Max pooling: The maximum value for the sliding range of the filter, the advantage: ignoring other features, preserving only the characteristics of that maximum, and reducing the noise of the model to make it more robust

Average pooling: an averaging operation in the sliding area of the filter

Max pooling used more than average pooling.

10.Convolutional Nerual Network Example

CNN Example:

Take a number recognition as an example the first layer consists of a conv and a pool, the second layer is also, the third and fourth layers are an all-connected, the last output layer of the activation function is a Softmax activation

The parameters of each layer are as follows:

11.why convolutions

Why use CNN

Parameter sharing: A feature detector (for example, a vertical feature detector) is applied to a part of a picture and may also be adapted to another part of the picture

Sparse connection: The output of each layer is only related to the input part area

On top of that, CNN has fewer parameters, which requires fewer samples, which may not be easy to fit in some way.

Moreover, CNN excels at capturing the position of the region, and is less susceptible to the position of the object, which enhances the accuracy of the network and the robustness of the system.