Going deeper with convolutions
Reprinted Please note:
Http://blog.csdn.net/stdcoutzyx/article/details/40759903
This article is aimed at the imagenet2014 competition. The method in this paper is the first in the competition, including Task 1 Classification task and Task 2 detection task. This article focuses on the efficient deep Neural Network Structure for computer vision. By improving the structure of the neural network, we can increase the depth of the network without increasing computing resources.
1. Main Contribution
Improve utilization of the computing resources inside the network, which is achieved by carefully crafted design and allows for increasing the depth and width of the network while keeping the computational budget constant.
Architecture decisions are based on the Hebbian principle and the intuition of multi-scale processing.
A 22 layers deep network is assessed in the competition.
2. ralated work
- The network structure proposed in this paper is inception, which is named after network in network ).
- Recent Trend of CNN is to increase the number of layers and layer size, while using dropout to address the problem of overfitting.
- Reference Document 15 uses Gabor filters of different scales to deal with multi-scale problems, similar to the inception model in this article.
- This article draws on reference 12 and uses many 1 × 1 convolution kernels. In this paper, convolution kernels are used to reduce dimensionality to remove computing bottlenecks.
- Detection task's leading approach is regions with convolutional neural networks (R-CNN) (Reference 6 ). This method is divided into two steps:
- First utilize low-level cues such as color and superpixel consistency for potential object proposals in a category-agnostic fashion.
- Then use CNN classifiers to identity object categories at those locations.
3. Motivation and high level considerations3.1. drawback of increasing CNN size directly:
- More prone to overfitting.
- Dramatically increase use of computational resources. (for example, if most weights end up to be close to zero, then lots of computations is wasted .)
3.2. How to solve it?
- The fundamental way wocould be by ultimately moving from fully connected to sparsely connected ubuntures.
- The Reference 2 in this paper shows that considering the statistical correlation, a sparse network structure can be re-constructed to obtain the optimal structure. And generate Hebbian principle -- neurons that fire together, wire together.
- At the underlying layer, the computing of hardware on inconsistent sparse data structures is very inefficient, especially when databases that have been optimized for intensive matrices are used on these data. Since reference 9 in this paper, random and sparse network structures have been used to break the symmetry and improve the learning rate. However, in reference to article 11, the fully connected structure is re-used to achieve the efficiency of intensive computing.
- Therefore, the question is whether there is a way to maintain the sparsity of the network structure and the high computing performance of intensive matrices. This paper proposes an inception module to achieve this effect.
4. Repeated tural detail
The main idean of the inception architecture is based on finding out how an optimal local sparse structure in a convolutional vision network can be approximated and covered B readily available dense components.
How can we find the optimal structure? We can consider that a lower level corresponds to a region of the image, and a 1 × 1 convolution kernel still corresponds to this region, using a 3 × 3 convolution kernel, you can obtain a larger region. So Design 1.
Figure 1 inception module, Na? Ve version
To reduce the dimension, use a 1 × 1 core for dimensionality reduction and design 2. Dimensionality Reduction can take effect mainly because of the development of embedding technology, even if the lower dimension can still contain a lot of information.
Figure 2 inception module with dimension Functions
In the filter concatenation layer, connect the convolution results of 1x1/3x3/5x5.
The benefit of this design is to prevent the explosive demand for computing resources due to the increase in the number of layers. The Network width and depth can be expanded. The structure of the inception layer can be accelerated by 2-3 ×.
5. googlenet
3. For more detailed structure, see the original paper.
Figure 3 googlet incarnation of inception Architecture
6. Training Methodology
- Using distbelief distributed machine learning system with modest amount model and data parallelism.
- Training with asynchronous stochastic gradient descent with 0.9 momentum, fixed learning rate schedule (decreasing the learning rate by 4% every 8 epochs) and polyak averaging (Reference 13) is used.
- Sampling of various sized patches of the image whose size is distributed evenly between 8% and 100% of the image area and whose aspect ratio is chosen randomly between 3/4 and 4/3.
- Photometric distortions
- Using Random interpolation methods for resizing relatively late and in conjunction with other hyperparameter changes.
7. Experiments setup and results
- Trained 7 versions of same googlet model, specified med ensembel prediction with them. These models are trained with the same initialization, but differ in sampling methodologies and the random order in which they see input images.
- Testing: resize the image to 4 scales where the shorter Dim is 256,288,320,352. take the left, right, center square of these resized images, then take the 4 corners and the center 224 × 224 crop and the square resize 224 × 224, and their mirror version. namely, 4 × 3 × 6 × 2 = 144 crops per image.
- Softmax probabilities are averaged over multiple crops and over all individual classifiers to obtain the final prediction. Simple averaged is the best. The result is as follows:
Figure 4 Performance of the competition
Figure 5 performance of fusions of Models
8. Reference
[1]. szegedy C, Liu W, Jia y, et al. Going deeper with convolutions [J]. arXiv preprint arXiv: 1409.4842, 2014.
Going deeper with convolutions
