Semantic segmentation--end-to-end Instance segmentation with recurrent Attention

End-to-end Instance segmentation with recurrent Attention
CVPR2017
Https://github.com/renmengye/rec-attend-public

This article uses recurrent neural Network (RNN) architecture for Instance segmentation to split each object sequentially, using an attention mechanism to mode L A human-like counting process

2 Recurrent attention Model
Our system consists of four parts: 1) An external memory is used to record the state of each segmented object, 2) box proposal network for locating objects of interest, 3) segmentation network pixel-level segmentation of objects within a rectangle , 4) The scoring network is used to determine whether an object is found and whether it ends
The entire system Segmentation example diagram:

Network structure diagram:

Input pre-processing image preprocessing
Here we train a FCN as a preprocessing system, this FCN consists of two parts: 1) A deconvnet with skip connections for foreground detection, 2) The second module is a angle map that outputs each object in reference to "40"

2.1. Part A:external Memory
This module is mainly used to divide the area of an image on the basis of what part of the object is already divided.
To decide where to look next based on the already segmented objects, we incorporate an external memory, which provides OB Ject boundary details from all previous steps.

2.2. Part B:box Network
Locate the next object to be segmented, using the LSTM
Localizing the next object of interest

2.3. Part C:segmentation Network
This part is based on the semantic segmentation of a single object, using a variant of the deconvnet [] with skip connections

2.4. Part D:scoring Network
Used to count divided objects and terminate the split process
To estimate the number of objects in the image, and to terminate our sequential process

2.5. Loss functions
Here the main definition of three loss function: The segmentation matching IoU loss l-y; The box IoU loss l-b; And the score cross-entropy loss L-s

4 experiments

cityscapes Instance-level Segmentation Results

Split effect Diagram