CS231N Spring LECTURE11 Lecture Notes

1. Semantic segmentation

Classify each pixel to a certain semantics.

In order to reduce the amount of computation, the sample is dropped first and then raised. Reduced sampling is generally used in the pool layer, up sampling has a variety of "unpooling", "Transpose convolution" (the literature is also called "upconvolution" and other names).

The problem of training data acquisition is very expensive because it takes a pixel to label a pixel.

2. Classification + Localizatoin

Generally use the same network, on the one hand to obtain the classification, on the one hand to bounding box position and size.

3. Object Detection

Pre-set what objects to find, once the picture is found, frame it out. Classification + Localizatoin are generally for a single object, and this is for multiple objects.

Sliding window: The calculation is too large to discard.

Region proposals: First find the picture area that may have the object, and then deal with it, on the CPU for about a few seconds. This method comes out before deep learning.

R-CNN: First find the region proposal, then the region proposal to adjust the size of the neural network needs, and then to the neural network calculation, and finally through the SVM classification.

Training is slow (84h) and consumes memory. Predictions are also slow (47 seconds VGG16)

Fast r-cnn: Much faster than R-CNN, training (8.75h), predictions (calculate region proposal spend 2 seconds, neural network predicts 0.32 seconds).

During training, linear + softmax and linear are added together to get multi-task loss.

Faster r-cnn: Use convolutional layer to predict region proposal. Faster than fast R-CNN, the forecast takes 0.2 seconds.

YOLO (Redmon et al, CVPR)/ssd (Liu et al, "Single-shot multibox DETECOTR", ECCV 2016): These two methods do not use region proposal, faster, but relatively Well, that's accurate. Faster r-cnn more slowly, but more accurate.

Object Detection + captioning (Densecap, CVPR 2016)

4. Instance segmentation

Semantic segmentation and Object detection combine to find multiple objects and determine which category each pixel belongs to.

Mask r-cnn (He et al., 2017), the network has two branches, the first one executes the object Detection, and the second executes semantic segmentation. This network combines the previous, is the synthesizer, performance very very good. In the object detection branch joins the recognition to the human body joint, but also can recognize other people's pose. Based on faster R-CNN, close to real-time.

CS231N Spring LECTURE11 Lecture Notes