Caffe Reverse convolution

Ben wanted to simply record the process, I did not expect everyone so like this blog, extremely honored. Maybe I have not recorded enough detail, there are some problems in the actual operation. The project code is given here (because it is often not enough to add points to the resource). If there is no points to download can dms my mailbox). introduce

The paper "Visualizing and Understanding convolutional Networks" by reverse operation of convolution network, the activation value of the specified convolution layer is projected back to the input pixel space. This reveals the contribution of different parts of the input image to the activation value of the layer.
The paper concludes: For each layer basically has the following four operations composition, Conv relu maxpooling [optionally] Norm [optionally]

The corresponding reverse operation is, Unpool rectify (Relu) filter (DECONV)

As the pooling operation is irreversible, as an approximate inverse operation Unpool need to know the location of the maximum value in pooling region when the pooling operation is used, and store it with switches.

Figure 1. Top:a deconvnet Layer (left) attached to A con-vnet layer (right). The deconvnet would reconstruct a approximate version of the convnet features from the layer. Bottom:an illustration of the unpooling operation in the deconvnet, using switches which record the location of the Max in each pooling region (colored zones) during pooling in the convnet. add a new layer to the Caffe

April 20, 2017 update:
How to add new layer to the Caffe
The new version of the Caffe added layer, more relaxed, in general, in four steps:

Add the paramter message corresponding to the layer in the./src/caffe/proto/caffe.proto;

Add a declaration of the layer class in the./include/caffe/layers Directory

Create a new. cpp and. cu file under the./src/caffe/layers/directory for class implementation.

Add layer test code to the./src/caffe/gtest/, test the written layer forward and back, and test the speed.

The last step many people save, or not aware of, but to ensure that the code is correct, the proposal or strict testing, the knife is not a mistake to cut wood work.
Excerpt from: How to add layer in Caffe and Caffe triplet loss layer implementation

Github-piergiaj/caffe-deconvnet:a deconvolutional Network in Caffe gives an open source implementation of this article. You can see that the source code only gives the new three layers: poolingswitches: Replace the original pooling layer, output pool after the results and the maximum index position switches. Slicehalf: The poolingswitches layer of the results of segmentation, is a special slice layer. Invpooling:pooling the reverse operation.

However, due to the Caffe version of the reason can not directly add new layer to the Caffe project, need to make some changes. Change 1: Three layers of header files written in common_layers.hpp and Vision_ LAYERS.HPP, this is the old version of the Caffe, the new edition of the need to independently establish the corresponding header file, from the original file copy the corresponding content, CPP file also need to refer to the new version of the wording of a slightly modified. Change the factory function that adds a new layer to the 2:LAYER_FACTORY.HPP (cannot paste replace.) ）。 Change 3: Modify the V1layerparameter in the Caffe.proto file.

Finally recompile the Caffe. Test

Direct use of GitHub to download test cases. Convnet uses the modified bvlc_reference_caffenet to replace the pooling layer with the poolingswitches layer. Deconvnet is the reverse operation of the former. Share the parameters.
Convnet:

Deconvnet:

The test code is as follows:

Import NumPy as NP import matplotlib.pyplot as PLT import OS import sys sys.path.append ('./python ') Import Caffe CAFFE.S ET_MODE_CPU () net = caffe.net (' python-demo/deploy.prototxt ', ' Models/bvlc_reference_caffenet/bvlc_referenc E_caffenet.caffemodel ', Caffe. TEST) invnet = caffe.net (' Python-demo/invdeploy.prototxt ', Caffe. TEST) # Input preprocessing: ' Data ' is the name of the input blob = = Net.inputs[0] Transformer = Caffe.io.Transformer ({' d ATA ': net.blobs[' data '].data.shape} ' transformer.set_transpose (' data ', (2,0,1)) Transformer.set_mean (' Data ', Np.load (' python/caffe/imagenet/ilsvrc_2012_mean.npy '). Mean (1). Mean (1)) # mean pixel Transformer.set_raw_scale (' Data ', 255] # The Reference model operates on images in [0,255] range instead of [0,1] transformer.set_channel_swap (' data  ', (2,1,0)] # The Reference model has channels in BGR order instead of RGB def norm (x, s=1.0): X-= X.min () x/= X.max () return x*s #对特征图进行可视化 def vis_square (data, padSize=1, padval=0): Data-= data.min () data/= Data.max () # force the number of filters to be square n = i NT (Np.ceil (NP.SQRT (data.shape[0))) padding = ((0, N * * 2-data.shape[0]), (0, Padsize), (0, Padsize)) + ((0, 0),) * (data.ndim-3) data = np.pad (data, padding, mode= ' constant ', constant_values= (Padval, Padval)) # Tile the filter
    s into an image data = Data.reshape ((n, N) + data.shape[1:]). Transpose ((0, 2, 1, 3) + Tuple (range (4, Data.ndim + 1))


data = Data.reshape ((n * data.shape[1], n * data.shape[3]) + data.shape[4:] Plt.axis (' Off ') plt.imshow (data) net.blobs[' data '].data[...] = transformer.preprocess (' Data ', caffe.io.load_image (' test/butterfly.jpg ') out = Net.forward () #参数共享 for B in invnet.params:invnet.params[b][0].data[...] = Net.params[b][0].data.reshape (invnet.param


S[b][0].data.shape) feat = net.blobs[' Pool5 '].data feat[0][feat[0] < 0] = 0 vis_square (feat[0), padval=1) Plt.show () #相关赋值操作 invnet.blobs[' pooled '].data[...] = feat invnet.blobs[' switches5 '].data[...] = net.blobs[' switches5 '].data invnet.blobs[' switches2 ' ...


] = net.blobs[' switches2 '].data invnet.blobs[' switches1 '].data[...] = net.blobs[' switches1 '].data invnet.forward ()
PLT.CLF () feat = norm (invnet.blobs[' conv1 '].data[0],255.0) gci=plt.imshow (transformer.deprocess (' Data ', feat))
Plt.colorbar (GCI); Plt.show ()

Input Image:

starting from the activation value of the CONV5 layer, deconvolution results: