The role of 1*1 convolution nucleus in googlelenet

1. Enable cross-channel interaction and information integration

1x1 convolution Layer (possibly) caused people's attention is in the structure of NIN, the thesis Andrew Brother's idea is to use MLP instead of the traditional linear convolution kernel, thereby improving the network's expression ability. At the same time, using the angle interpretation of cross-channel pooling, the proposed MLP is equivalent to the CCCP layer behind the traditional convolution kernel, thus realizing the linear combination of multiple feature maps and realizing the information integration across the channel. The CCCP layer is equivalent to 1x1 convolution, so a closer look at the NIN Caffe realization, is in each of the traditional convolution layer behind the two CCCP layer (in fact, is connected to two 1x1 convolution layer).

2. Reduce and increase the number of convolutional cores, reducing network parameters

The descending and ascending dimension is of great importance (probably) in the googlenet. For each inception module (e.g.), the original module is left, and a 1x1 convolution is added to the image on the right to reduce the dimension. Although the convolution cores on the left are relatively small, when the number of inputs and outputs is large, multiplying will make the convolution kernel parameters very large, and the right figure added 1x1 convolution can reduce the number of input channels, convolution kernel parameters, the complexity of the operation is then lowered. Take the Googlenet 3a module as an example, the input feature map is the 28X28X192,3A module in the 1x1 convolution channel for the 64,3x3 convolution channel for the 128,5x5 convolution channel is 32, if the structure is left, then the convolution kernel parameter is 1x1x192x64+3x 3x192x128+5x5x192x32, and a 1x1 convolution layer with a channel number of 96 and 16 is added to the right to the 3x3 and 5x5 convolution layers, so that the convolution kernel parameter becomes 1x1x192x64+ (1x1x192x96+3x3x96x128) + (1x1x 192x16+5x5x16x32), the parameter is reduced approximately to the original One-third. At the same time after adding 1x1 convolution layer behind the parallel pooling layer can also reduce the output of feature map number, left Figure pooling feature map is unchanged, and then add convolution layer feature map, will make the output feature map expanded to 416 , if each module is this way, the output of the network will be more and more large. On the right, a 1x1 convolution with a channel of 32 is appended to the pooling, which reduces the feature map number of the output to 256. Googlenet uses 1x1 convolution to reduce the dimension, get a more compact network structure, although there are a total of 22 layers, but the number of parameters is only 8 layers of alexnet one-twelveth (of course, there is a large part of the reason is to remove the full connection layer).

Recently the hot MSRA ResNet also used the 1x1 convolution, and was used before and after the 3x3 convolution layer, not only to reduce dimensions, but also to raise dimensions, so that the convolution layer input and output channel number are reduced, the number of parameters further reduced, such as the structure. (Or I can't imagine how the 152-tier network is going to run. Tat)

The role of 1*1 convolution nucleus in googlelenet