Alenet Model Notes

Who created the alexnet?

Alexnet is a deep convolutional neural network proposed by Hinton, a disciple of the Great God, Alex Krizhevsky. It can be considered a deeper and wider version of Lenet.

Alexnet technology used in the main

• The successful use of Relu as the activation function of CNN, and verified its effect in the deeper neural network than the Sigmiod, successfully solved the sigmoid gradient dispersion problem in the deep network.
• At the time of training, we use dropout to randomly inactivate a subset of neurons with a certain probability, and a model is fitted.
• Use overlapping maximum pooling method: The pool core size is larger than the step length, and the output of the convolution layer has overlapping parts, which improves the richness of the feature.
• In this paper, LRN (local normalization) is proposed to create a competitive mechanism for local neurons, which makes the output of neuron with large response larger and suppresses the neuron with less feedback. Increased generalization capability to a certain extent.
• With Cuda acceleration, two GTX 580 3GB graphics cards are accelerated. This causes the network structure diagram in the paper to be divided into two-way training.
• Using data enhancement: Randomly intercept 224*224 size area (horizontal flip and mirror) from 226*226 's original image, data enhancement effectively suppresses overfitting and improves generalization ability.

Network structure

The entire alexnet has 8 layers that need to be trained (not including LRN and pooling layers), the first 5 layers are convolutional layers, and the last three layers are fully connected layers, where the final fully connected layer output is a 1000-channel Softmax mapping normalization result, representing the input in the 1000 category of response, or the probability distribution on the attribution class, and, in detail, the Softmax output of each channel indicates the possibility that the input belongs to that class. Due to the limitation of memory capacity at that time, the author used 2 GTX580 3GB RAM GPU parallel training, so the network is divided into two ways.

Now our video card is enough to be able to do it all the way.

Above the network:

• The convolution cores of 5 convolutional layers are: 11*11*[email protected],5*5*[email protected],3*3*[email protected],3*3*[email protected],3*3*[email Protected], step order is 4,1,1,1,1, mode is Valid,same,same,same,same
• The pooling layer is halved after ①, ② and ⑤, each time the pool is pooled.
• LRN between the ① and ② of the pooling layer and the post-Relu convolution layer
• Then there are three fully connected layers, and the last fully connected layer is the result of the Softmax output.

The realization of alexnet

adjourned

Reference

52134584

"TensorFlow Combat"

Alenet Model Notes