Reverse convolution (transpose convolution)

The transpose convolution is actually equivalent to the reverse propagation of the normal convolution.

Consider an input of x=4x4, convolution kernel is w=3x3, step is stride=1,zero-padding=0

Expand the convolution kernel to a sparse matrix C:

To do convolution can be 2x2 output y,c*x=y. As shown in the following illustration:

So how do you get the transpose convolution? As we said above, the transpose convolution actually corresponds to the reverse propagation of the normal convolution.

And then to the upper formula to enter the gradient of x:

We can find that in the process of reverse propagation, the gradient from the high level to the bottom, just from the high level of the gradient left-by Ct. So we're going to get the 4x4 output from the 2x2 input y as long as: Ct*y.

However, it is important to note that the transpose convolution simply restores its shape and does not recover its value, which is why it is called a deconvolution.

In fact, in the Tensorflo of the implementation of the convolution is also the use of this property.

Another way to realize the transpose convolution operation is to zero-padding the input feature map, which can be realized by the normal convolution operation:

But this kind of operation has a disadvantage to the above one, the padding of a map is often required to do many rows and columns, so the efficiency is relatively low.

Finally, it is necessary to note that there is a difference between the convolution and the deconvolution in signal processing, and it is not the reverse operation of convolution.

If you do not understand transpose convolution, you can set parameters like this, for example, now you need to nxn from a map to a MXM map, and you don't know how to set up its convolutional kernel size, padding and strides, This time you can in turn consider how the normal convolution from MXM to the NxN map, the two parameters are the same. You define a convolution kernel, decide whether to convolution or transpose the convolution, just to see if you are left by C or Ct.

References: A Guide to convolution arithmetic for deep learning

https://www.zhihu.com/question/43609045?rf=53910187