Understanding and function of 1*1 convolution nucleus

There are basically two ways to share weights:

• Shared weights are used in the same feature map and in different channel features, so that the convolution parameters are minimal, for example, the previous layer is 30*30*40, and the convolution parameters are: 3*3*120 when using 3*3*120 convolution cores for convolution. (Convolution has a difference with MLP. A neuron is a planar arrangement, one is a linear arrangement)
• The second type only uses the shared weights on the same feature graph, according to the example above, the convolution parameter is: 3*3*40*120.

The convolution of 1x1 has about two functions:
1. Enable cross-channel interaction and information integration
2. Reducing the number of convolution cores and ascending dimension

Take the Googlenet 3a module as an example, the input feature map is the 28X28X192,3A module in the 1x1 convolution channel for the 64,3x3 convolution channel for the 128,5x5 convolution channel is 32, if the structure is left, then the convolution kernel parameter is 1x1x192x64+3x 3x192x128+5x5x192x32, and a 1x1 convolution layer with a channel number of 96 and 16 is added to the right to the 3x3 and 5x5 convolution layers, so that the convolution kernel parameter becomes 1x1x192x64+ (1x1x192x96+3x3x96x128) + (1x1x 192x16+5x5x16x32), the parameter is reduced approximately to the original One-third. At the same time after adding 1x1 convolution layer behind the parallel pooling layer can also reduce the output of feature map number, left Figure pooling feature map is unchanged, and then add convolution layer feature map, will make the output feature map expanded to 416 , if each module is this way, the output of the network will be more and more large. On the right, a 1x1 convolution with a channel of 32 is appended to the pooling, which reduces the feature map number of the output to 256. Googlenet uses 1x1 convolution to reduce the dimension, get a more compact network structure, although there are a total of 22 layers, but the number of parameters is only 8 layers of alexnet one-twelveth.

For example, the input is 28x28x192 (Wxdxk,k represents the number of channels), and then in the 3x3 convolution kernel, the convolution channel number is 128, then the convolution parameters are 3x3x192x128, of which the first two corresponding to each convolution inside the parameters, the last two corresponding convolution total number (generally understood as, The weight sharing of convolutional cores is only valid on each individual channel, as the corresponding convolution cores between channels and channels are not shared independently, so this is 192x128.

Understanding and function of 1*1 convolution nucleus