SLIC Hyper-pixel Segmentation (iii): Application _slic

After reading the above introduction, we should think about: what is the use of segmented pixels? How to use. Where to use.

First of all, hyper-pixel can be used for tracking, you can refer to Lu topic Group published in the IEEE Tip on the "robust superpixeltracking"; second, you can do tag classification, referring to the 09 ICCV "Class Segmentation Andobject localization with Superpixel neighborhoods, this article still has a great guiding significance to the later articles And do hyper-pixel-level word bags (superpixel-basedbag-of-words), referring to the 13 CVPR "improving an objectdetector and extracting using Superpixels ", the author puts the pixel-level feature of the sample through K-means clustering for the hyper-pixel-level word bag, and finally combines SVM to classify difficult classification samples, and the algorithm block diagram is as follows;

Super pixel-level word bag

There's also a lot of video foreground segmentation, because compared to pixels, hyper-pixel processing speed will be dozens of times times faster, hundreds of times times even higher, this is more important for real-time video segmentation, as well as the recently proposed Supervoxel concept (can be considered three-dimensional superpixel), the article can refer to the Video Object Segmentationwith shape cue based on spatiotemporal Superpixel neighbourhood and the Supervoxel-consistentforeground propagation in the video ".

Supervoxel (3D superpixel) sample

In addition to the above mentioned above, the hyper-pixel can also be used for skeleton extraction, human posture estimation ("Guiding Modelsearch Using Segmentation"), medical image segmentation and so on.

Skeleton Extraction Sample

Medical Image Segmentation

In fact, the traditional pixel-level processing can be considered to create a hyper-pixel processing, so if you want to use, hyper-pixel can be applied to computer vision in all aspects of the big area. Give a concrete example to analyze: Each red closed-loop contour in the following image is a hyper-pixel, the pixel-level optical-flow transformation (for example, the mean value) is a hyper-pixel level of light vectoring with a green arrow. Such a 3 million-pixel picture can be represented by a pixel of 300 (a given split number k=300).

Below an article "improving video foreground segmentation" with a object-like pool "Download the paper as an example of the specific analysis of the use of hyper-pixel. The objective of this article is to segment the moving object in the sequence image without supervision. The algorithm flow is as follows:

Algorithm flowchart

1. The authors first compute the common features of pixel level: binary mask (binary mask, obtained by frame difference), and optical flow. We know that a graph cut based approach requires multiple iterations, and if it is time-consuming to directly perform pixel binary, the authors convert pixel-level mask and optical flow features to hyper pixel-level features.

2. Binarymask characteristics through the 2012-year CVPRW's paper "Improving foreground segmentationswith probabilistic superpixel, Markov random Fields" The method in

The transformation to the hyper-pixel feature, in fact, is the binarymask of pixels per pixel statistics, with the number of target points accounted for the total number of pixels (that is, the size of the pixel) to give the hyper-pixel to a binarymask feature under the foreground/background probability.

Optical flow pixel-level feature transformation slightly more complex, because each pixel is a two-dimensional vector, the author first vectoring the average amount of light as the vectoring of the pixel, and then all the pixel level of the light vectoring mean value as a reference vector, The cosine similarity of each hyper-pixel vector and reference vector is calculated as the probability that the hyper-pixel belongs to the foreground/background under the optical flow feature.

3. The segmentation problem is in fact the problem of each hyper-pixel tag, the author uses conditional random field (conditional random field) to model the distribution of hyper-pixel, both of these hyper-pixel features need to be normalized to [0,1] and weighted fusion, and the result as unary Potential (a unary potential function that describes itself as a foreground/background probability). The expression of pairwise potential (two-yuan potential function, describing neighborhood relation) refers to the pixel-level approach: The neighborhood domain of the seed point as the central hyper-pixel neighborhood. The advantages of the Slic method are shown here, the result of the segmentation is relatively compact and the size difference is not big, so the neighborhood relation is still better than other hyper pixel segmentation methods.

4. The Object-likepool and Background-like pool is the concept introduced by the author, which is the result of the previous segmentation as a priori condition to guide the current segmentation, which retains the hyper pixel level of information (color, optical flow, position). Then, a new normalized characteristic foreground likelihood is calculated by using the formula (11), which is incorporated into the conditional random field model.

Let's discuss how to set the number of pixels to be split. When using hyper-pixel to segment an image, set the number of pixels k is more important: if the k is relatively small, each super pixel size will be larger, so that the pixel to the boundary will be maintained, if the k is larger, each pixel size will be relatively small, then there will be similar "over fit" phenomenon, The hyper-pixel shape becomes very irregular, the neighborhood is difficult to maintain, and the number is more numerous. The following figure:

Comparison of the results of different sizes of hyper-pixel segmentation. (a) original image, Super pixel size: (b) 100x100, (c) 30x30, (d) 8X8

Actually the specific partition number k is related to the specific application, for example, if the above figure to do the main character (the left of the Little Witch) segmentation, 100x100 size of the super pixel is enough, but if the two of the characters are also split, you need to use the 30x30 size, but at this time the smaller character segmentation accuracy is not high, If you have higher requirements, you need to use 8x8 or even smaller sizes. In addition, because the image above is animation, so the details and textures are not many, the real scene of the picture texture will be more complex, size also need to choose according to different occasions.