Image segmentation "Refinenet-multi-path refinement Networks for high-resolution semantic segmentation"

The main problem with CNN for semantic segmentation is the reduction in resolution caused by repeated sampling operations. Refinenet proposes a multipath improved network to extract all the information from the sampling process and to obtain high-resolution predictions using long distance residual connections. With the characteristics of fine layers
, the high-level semantic information can be improved. In addition, the paper uses chain-type residuals to obtain rich background knowledge.

High-level semantic features are helpful to get the category identification of image region, while low-level features are helpful to gain edge and mutation, and how to obtain the characteristics of middle layer is an open problem. This paper designs a method to obtain the middle-layer characteristics.

Related Work
Fcnn:region-proposal based methods
Deeplab-crf
Deeplab-rnn
FCN: Getting middle-tier features
Hypercolumn: Merging the features of the middle tier
Segnet,u-net:skip-connections

background
ResNet effect is good, and there is a pre-training model, ResNet easy to change for dense segmentation classification task. Using the compact prediction layer instead of the single marker prediction layer, the classification confidence of each pixel is output, as shown in the following figure:
The step size is 2, so the resolution of each layer is reduced, and the lower sample layer has two effects: increasing the volume layer sensation field, making the filter obtain the global high quality information for the classification, the balance filter number and the resolution. Usually the last level of the feature graph is 1/32 of the input image. Low-resolution feature graphs lose a lot of information, especially the detail information obtained by low-level features. Deeplab uses the convolution of the hole to solve this problem, but the computational capacity is large:

refinenet
The structure of the refinenet can be identified in the following figure:
Multi-Path improvement
The ResNet is divided into 4 blocks, using a 4 cascade structure containing 4 refinenet units, each structure is directly linked to the output of the ResNet block and the previous refinenet block, and the refinenet block can accept input from different resnet blocks. Can have a lot of different variants.
In the figure above, RefineNet-4 has only one input, RefineNet-4 output and ResNet The goal of block-3 input to refinenet-3.refinenet-3 is to improve the low-resolution features of RefineNet-4 output using high-resolution features. Refinenet Structure and composition
The structure of the refinenet block is shown in the following illustration:

Consists of three components:
Residual convolution unit
Multi-scale Fusion
Chain-type residuals

Experimental Results
1. Use of person-parts compared with other methods

2. Results on the Cityspace