Deep Residual Network _ depth Learning

Deep Residual network in the 2015 ILSVRC competition to achieve the first achievement, ICLR2016 is also one of the key issues.

Its main idea is simply to add a hop to bypass some layers of connectivity on a standard feedforward convolution network. Each bypass layer produces a residual block (residual blocks), and the convolution layer predicts the residuals of the input tensor. As shown in the following illustration:

Common depth feedforward networks are difficult to optimize. In addition to depth, the added layer causes training and validation error rates to increase, that is, the use of the batch normalization is also the case. Due to the existence of shorcut connections in residual neural network, the data flow between networks is more smooth. ResNet authors believe that the deep residual network is unlikely to be due to a gradient disappearance because it is difficult to appear in batch normalized network. The solution to the residual network structure is to add a shortcut connection to the summation of the output of the convolution layer.

On Facebook, ResNet is implemented with torch, which can be found on GitHub. GitHub Address Transfer Gate

Reference documents:
"Deep residual Learning for Image recognition", authored by kaiming He, Zhang Xiangyu, shaoqing Ren and Jian Sun.