Slic Hyper-pixel algorithm

Source: https://blog.csdn.net/Fighting_Dreamer/article/details/77170859 comparison between Slic and current optimal hyper-pixel algorithm

Radhakrishna Achanta, Appu Shaji, Kevin Smith, Aurelien Lucchi, Pascal Fua, and Sabine s¨usstrunk abstract

In recent years, computer vision applications have become increasingly dependent on hyper-pixels, but are not always clear about what is a good hyper-pixel algorithm. In order to understand the advantages and disadvantages of the existing methods, we compare the best five kinds of super-pixel algorithms, compared with the image boundary adhesion, algorithm speed, storage efficiency, and their influence on the segmentation performance. Then we introduce a new hyper-pixel algorithm, a simple linear iterative clustering (SLIC), which uses the K-means clustering method to generate super pixels efficiently. Although it is simple, slic is better at acquiring boundaries than previous algorithms, and it has faster speeds, higher memory efficiency, improved segmentation performance, and can be scaled directly to the generation of hyper-body elements.
Keywords-hyper-pixel, split, cluster, K-mean.

Fig. 1: Use Slic to split the dimensions (approximately) to 64,256 and 1024 over pixels. I. Introduction

The

Hyper-pixel algorithm combines pixels into a sense-aware atomic region (atomic regions), which can be used to replace the rigid structure of a pixel mesh. They capture image redundancy, provide convenient primitives (primitive) for computing image features, and greatly reduce the complexity of subsequent image processing tasks. They have become key building blocks for many computer vision algorithms, such as the multi-class object segmentation in the Pascal VOC Challenge [9],[29],[11], depth estimation [30], segmentation [16], Body model estimation [22], and object positioning [9].  
There are many ways to produce hyper-pixels, each with its own advantages and disadvantages, which can be better suited to a particular application. For example, if adherence to image boundaries is critical, [8] The graph-based approach may be an ideal choice. However, [23] may be a better choice if the hyper-pixel is used to build a graphic with a more regular lattice (lattice). Although it is difficult to define an indicator to judge the merits and demerits of an algorithm, we believe that the following properties are usually desirable:  
1) The pixels should be glued to the image bounds well.  
2) when used as a preprocessing step for reduced computational complexity, hyper-pixels should be calculated quickly, with high memory efficiency and ease of use.  
3) when used for segmentation purposes, hyper-pixels should increase speed and improve the quality of the results.  
As a result, we compared the five most advanced hyper-pixel methods [8],[23],[26],[25],[15], assessing their speed, the ability to connect image boundaries, and segmentation performance. We also provide qualitative analysis of these and other hyper-pixel methods. We concluded that the existing approach could not be satisfactory in all respects.  
To solve this problem, we propose a new hyper-pixel algorithm: Simple linear iterative Clustering (SLIC), which uses Kmeans clustering to generate hyper-pixels in a manner similar to [30]. Although very simple, slic on the Berkeley Benchmark [20] produces the best effect on the image boundary and is better than the existing method when splitting on the pascal[7] and msrc[24] datasets. In addition, it is faster and higher storage efficiency than existing methods. In addition to these quantifiable benefits, the slic is also easy to use, more compact, has a more flexible generation of super-pixels, and can be scaled directly to higher dimensions, and is free to use (meaning that the code is open source). ii. existing hyper-pixel generation algorithm

The

algorithm used to generate the hyper-pixel can be broadly categorized as a method based on graph or gradient ascent. Below, we review the common hyper-pixel methods for each category, including some algorithms that were not originally designed to generate a hyper-pixel. Table I provides a qualitative and quantitative summary of the methods studied, including their relative properties.  

Table I: Summary of existing hyper-pixel algorithms. The performance of the hyper-pixel adhesion to the boundary can be evaluated in the Berkeley DataSet [20], which can be ranked according to two standard metrics: Under-segmentation error and boundary recall (for 〜500 pixels). We also showed the average time required to use segmented images of Intel dual-core 2.26 GHz processors with 2GB of RAM, and the class average segmentation accuracy obtained on the MSRC dataset using the method described in [11]. Bold entries represent the best performance for each category. It also provides the ability to specify the amount of hyper-pixels, to control its compactness, and to generate a super-voxel. A. Graph-based algorithm

The

Graph-based hyper-pixel generation method treats each pixel as a node in the diagram. The edge weights between two nodes are proportional to the similarity of neighboring pixels. The hyper-pixel is created by a cost function defined in the minimized diagram.  
nc05-the normalized cut algorithm [23] recursively uses contour and texture cues to segment all the pixels in the image, thereby globally minimizing the cost function defined at the edge of the split boundary. It produces very regular, visually pleasing hyper-pixels. However, NC05 's boundary adhesion is relatively poor, and it is the slowest of the method (especially for large images), although the algorithm that attempts to accelerate exists [5]. NC05 has the complexity of [15], where n is the number of pixels.  
Gs04-felzenszwalb and Huttenlocher[8] propose an alternative graphics-based approach that has been applied to generate hyper-pixels. It takes pixels as nodes of the graph so that each pixel is the smallest spanning tree that makes up the pixels. GS04 is well glued to the image boundaries in practice, but produces hyper-pixels with very irregular dimensions and shapes. Its complexity is, in practice, fast. However, it does not provide explicit control over the amount of hyper-pixels or its compactness.  
Sl08-mooreetal presents a method of generating a mesh-compliant hyper-pixel by determining the best path or seam to divide the image into smaller vertical or horizontal areas [21]. Find the best path using a graph-cutting method similar to seamcarving[1]. Although the author gives the complexity, this does not take into account the pre-computed boundary graph, which strongly affects the quality and speed of the output.  
GCa10 and Gcb10-[26],veksleretal. Use a global optimization method similar to [14] for the texture synthesis work. By stitching overlapping image blocks together to get the super-pixel, each pixel belongs to only one of the overlapping regions. This method has two variants, one for generating compact hyper-pixels (GCA10) and one for constant-intensity hyper-pixels (GCB10). B. Gradient-based approach

The

starts with a coarse pixel initial cluster, and the gradient increases iteratively to modify the cluster until some convergence criteria are met to form a super-pixel.   in
Ms02-in[4], the average offset, the iterative pattern used to locate the local maximum value of the density function, is applied to the first pattern in the color or intensity feature space of the image. Pixels that converge into the same pattern define the mega-pixels. MS02 is an older method that produces a non-uniform size of irregular shapes over pixels. It is complex, makes it relatively slow, and does not provide direct control over the amount, size, or tightness of the super-pixel.  
qs08-Quick Shift [25] also uses patterns to find the tessellation scheme. It initializes the split using the medoid shift process. The search points in the feature space are then moved to the nearest neighbor, thereby increasing the parzen density estimate. Although it has a relatively good boundary adhesion, the QS08 is very slow and complex (d is a small constant [25]). Also, QS08 does not allow explicit control over the size or quantity of the hyper-pixel. Previous works used QS08 object positioning [9] and motion segmentation [2].   The
ws91-watershed method [28] starts with a local minimum to perform a gradient rise to produce a watershed, and to separate the lines of the catchment basin. The resulting hyper-pixels are usually highly irregular in size and shape, and do not exhibit good boundary adhesion. [28] The method is relatively fast (with the complexity), but does not provide control over the amount of super-pixel or its compactness.   The
Tp09-turbopixel method gradually expands a set of seed positions using a geometric flow based on the horizontal set [15]. The geometric flow relies on the local image gradient to distribute the hyper-pixels in a regular manner on the image plane. Unlike WS91, TP09 hyper-pixels are constrained to have uniform dimensions, compactness, and boundary adhesion. TP09 relies on algorithms of varying complexity, but in practice, as the author claims, having approximately the complexity [15] is one of the slowest algorithms examined and shows relatively poor boundary adhesion. III. Slic Super Pixel

We propose a new method of generating hyper-pixel, which is faster than the existing method, has higher memory efficiency, shows the optimal boundary compliance, and improves the performance of the segmentation algorithm. The simple linear iterative Clustering (SLIC) uses the K-means algorithm to generate the super-pixel, which has two important differences compared with other algorithms:  
1) significantly reduces the number of distance calculations in optimization by limiting the search space to areas proportional to the size of the pixel. This reduces the linear complexity of the number of pixels N and is independent of the amount of super-pixel K.  
2) weighted distance measurement combines color and spatial proximity, while providing control over the size and compactness of the mega-pixel.  
Slic is similar to the method described in [30] for a preprocessing step for depth estimation, which is not studied in the hyper-pixel direction. A. Algorithm

The

Slic is straightforward to use. By default, the only parameter to the algorithm is k, which means the number of pixels that are roughly equal in size. For color images in the Cielab color space, the clustering process begins with the initialization step, where K initial cluster centers are sampled on a regular grid spaced s pixels. To produce a roughly equal-sized super-pixel, the grid interval is. Moves the center to the seed position corresponding to the lowest gradient position in the 3x3 neighborhood. This is done to avoid positioning the hyper-pixel on the edge, and to reduce the chance of using noise pixels to vaccinate over pixels.  
Next, in the allocation step, each pixel I is associated with the nearest cluster center where the search area overlaps its location, as shown in Figure 2. This is the key to accelerating our algorithm, because limiting the size of the search area significantly reduces the number of distance calculations and results in significant speed benefits relative to the regular Kmeans clustering, where each pixel must be compared to all cluster centers. This can only be achieved by introducing a distance measurement d, which determines the nearest cluster center for each pixel, as discussed in section Iii-b. Because the expected spatial extent of the hyper-pixel is an area of approximate size sxs, a pixel-like search is made in the area 2sx2s around the center of the hyper-pixel. &NBSP
 
figure. 2: Reduces the hyper-pixel search area. The complexity of slic is linear in the number of pixels in the image O (N), whereas the regular K-means algorithm is O (KNI), where I is the number of iterations. This provides the search space for each cluster center in the allocation step. (a) in the conventional K-mean algorithm, distances are calculated from each cluster center to each pixel in the image. (b) Slic only calculates the distance from each cluster center to pixels within the 2sx2s region. Note that the desired super pixel size is only SxS and is represented by a smaller square. This method not only reduces the distance calculation, but also makes the complexity of slic independent of the number of pixels.  
Once each pixel has been associated with the nearest cluster center, the update step adjusts the cluster center to the average vector of all pixels belonging to the cluster. The L2 norm is used to calculate the residual error between the new cluster center position and the previous cluster center position E. The allocation and update steps can iterate over and over until the error converges, but we find that 10 iterations are sufficient for most images and report all the results of using this standard in this article. Finally, the post-processing step implements connectivity by reallocating disjoint pixels to nearby hyper-pixels. The whole algorithm is summarized in algorithm 1.  
 
 
b. Distance measurement

Figure 3: Slic for the video sequence. (top) A short shortwave video sequence of the resulting frames. (bottom left) the volume that contains the video. The last frame appears at the top of the volume. (bottom right) video's hyper-pixel segmentation. For ease of display, the voxel with an orange cluster center is removed C. Post-processing

Like some other hyper-pixel algorithm [8],slic does not explicitly force a connection. At the end of the clustering process, some "orphaned" pixels that do not belong to the same connection component as their cluster center may be preserved. To correct this, the connected component algorithm is used to assign the tags of the nearest cluster center to these pixels. D. Complexity

Iv. comparison with prior art

v. Biomedical Applications

Many popular graphics-based segmentation methods, such as graph cutting [3], are becoming more expensive because more nodes are added to the diagram, which in practice limits the size of the image. For some applications, such as mitochondrial segmentation from Electron micrograph (EM), the size of the image is large, but the resolution cannot be reduced at this time. In this case, the tessellation on the graph defined on the pixel grid will be tricky. In [18],slic hyper-pixel significantly reduces the complexity of the diagram, making the segmentation easy to handle. The segmented mitochondria from [18] are shown in Figures 3 (a) and (b). In [19], this method extends to the 3D image stack, which can contain billions of voxel. Only the most frugal algorithms can operate on such a large amount of data without the need to reduce the size of the graph in some way. The slic hyper-BODY element reduces memory requirements and complexity by more than three orders of magnitude and significantly increases performance compared to regular cubes, as shown in Figure 3 (c)-(e).

Figure 3:slic is applied to mitochondria from 2D and 3D em images from nerve tissue. (a) slic over pixels from em slices. (b) The result of the division of the method from [18]. (c) 1024x1024x600 volume of the slic BODY element. (d) Use the method described in [19] to extract the mitochondria. (e) Comparing the slic of the volume of a cube with a similar size in (c). Vi. Conclusion

Hyper-pixels have become an important tool in the visual community, and in this article we provide readers with in-depth analysis of the performance of modern hyper-pixel technology. We compare the best five mega-pixel algorithms with boundary adhesion, segmentation speed and performance as the preprocessing step in the segmentation frame. In addition, we propose a new method based on Kmeans clustering to generate hyper-pixels, Slic has been proven to be superior to existing hyper-pixel methods in almost every aspect.
Although our experiment was thorough, there was a warning. Some hyper-pixel methods, that is, GC10 and TP09, do not consider color information, while other methods are considered. This may adversely affect its performance. Reference Documents

[1] Shai Avidan and Ariel Shamir. Seam carving for content-aware image resizing. ACM Transactions on Graphics (SIGGRAPH), 26 (3), 2007.
[2] A. Ayvaci and S. Soatto. Motion segmentation with Occlusions on the Superpixel graph. In Workshop on dynamical Vision, Kyoto, Japan, October 2009.
[3] Y. Boykov and M. Jolly. Interactive Graph Cuts for Optimal Boundary & Region Segmentation of Objects in n-d Images. In international Conference on Computer Vision (ICCV), 2001.
[4] D. Comaniciu and P. Meer. Mean shift:a Robust approach toward feature space analysis. IEEE Transactions on Pattern analysis and Machine Intelligence, 24 (5): 603–619, May 2002.
[5] T. Cour, F. Benezit, and J. Shi. Spectral segmentation with Multiscale graph decomposition. In IEEE computer Vision and Pattern recognition (CVPR) 2005, 2005.
[6] Charles Elkan. Using The triangle inequality to accelerate k-means. International Conference on machine Learning, 2003.
[7] M. Everingham, L. Van Gool, C. K. I. Williams, J. Winn, and A. Zisserman. The PASCAL Visual Object Classes Challenge. International Journal of Computer Vision (IJCV), 88 (2): 303–338, June 2010.
[8] Pedro felzenszwalb and Daniel huttenlocher. Efficient graph-based image segmentation. International Journal of Computer Vision (IJCV), 59 (2): 167–181, September 2004.
[9] B. Fulkerson, A. Vedaldi, and S. Soatto. Class segmentation and Object localization with Superpixel neighborhoods. In international Conference on Computer Vision (ICCV), 2009.
J.M Gonfaus, X. Boix, J. Weijer, A. Bagdanov, J. Serrat, and J. Gonzalez. Harmony potentials for Joint classification and segmentation. In computer Vision and Pattern recognition (CVPR), 2010.
Stephen Gould, Jim Rodgers, David Cohen, Gal Elidan, and Daphne Koller. Multi-Class segmentation with relative location prior. International Journal of Computer Vision (IJCV), 80 (3): 300–316, 2008.
[Tapas] Kanungo, David M. Mount, Nathan S. Netanyahu, Christine d. Piatko, Ruth Silverman, and Angela Y. Wu. A Local Search approximation algorithm for K-means clustering. Eighteenth annual Symposium on Computational Geometry, Pages 10–18, 2002.
[Amit] Kumar, Yogish Sabharwal, and Sandeep Sen. A Simple Linear time (1+e)-approximation algorithm for K-means cluster ing in any dimensions. Annual IEEE Symposium on Foundations of Computer Science, 0:454–462, 2004.
[Vivek] Kwatra, Arno Schodl, Irfan Essa, Greg Turk, and Aaron Bobick. Graphcut textures:image and video synthesis using graph cuts. ACM transactions on Graphics, SIGGRAPH 2003, 22 (3): 277–286, July 2003.
[K] A. Levinshtein, A. Stere, K. Kutulakos, D. Fleet, S. Dickinson, and K. Siddiqi. Turbopixels:fast superpixels using Geometricflows. IEEE Transactions on Pattern Analysis and Machine Intelligence (Pami), 2009.
[+] Yin Li, Jian Sun, Chi-keung Tang, and Heung-yeung Shum. Lazy snapping. ACM Transactions on Graphics (SIGGRAPH), 23 (3): 303–308, 2004
[Stuart] P. Lloyd. Least squares quantization in PCM. IEEE Transactions on Information Theory, IT-28 (2): 129–137, 1982.
A. Lucchi, K. Smith, R. Achanta, V. Lepetit, and P. Fua. A fully automated approach to segmentation of irregularly shaped cellular structures in EM images. International Conference on Medical Image Computing and Computer assisted intervention, 2010.
[Aur´elien] Lucchi, Kevin Smith, Radhakrishna Achanta, Graham Knott, and Pascal Fua. supervoxel-based segmentation of mitochondria in EM Image Stacks with learned Shape Features. IEEE transactions on Medical Imaging, 30 (11), 2011.
D. Martin, C. Fowlkes, D. Tal, and J. Malik. A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring Ecolo Gical statistics. In IEEE international Conference on Computer Vision (ICCV), July 2001.
[Alastair] Moore, Simon Prince, Jonathan Warrell, Umar Mohammed, and Graham Jones. Superpixel lattices. IEEE computer Vision and Pattern recognition (CVPR), 2008.
[] Greg Mori. Guiding model search using segmentation. In IEEE international Conference on Computer Vision (ICCV), 2005.
[Jianbo] Shi and Jitendra Malik. Normalized cuts and image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence (Pami), 22 (8): 888–905, 2000.
J. Shotton, J. Winn, C. Rother, and A. Criminisi. Textonboost for Image understanding:multi-class Object recognition and segmentation by jointly Modeling Texture, Layout, and Context. International Journal of Computer Vision (IJCV), Bayi (1), January 2009.
A. Vedaldi and S. Soatto. Quick Shift and Kernel methods for mode seeking. In European Conference on Computer Vision (ECCV), 2008.
[+] O. veksler, Y. Boykov, and P. Mehrani. Superpixels and Supervoxels in a energy optimization framework. In European Conference on Computer Vision (ECCV), 2010.
[Verevka] O. and J.W. Buchanan. Local K-means algorithm for color image quantization. Graphics Interface, Pages 128–135, 1995.
[Vincent] Luc and Pierre Soille. Watersheds in digital Spaces:an efficient algorithm based on immersion simulations. IEEE transactions on Pattern analalysis and Machine Intelligence, 13 (6): 583–598, 1991.
Y. Yang, S. Hallman, D. Ramanan, and C. Fawlkes. Layered Object detectionformulti-classsegmentation. Incomputer Visionand Pattern Recognition (CVPR), 2010.
[C]. L. Zitnick and S. B. Kang. Stereo for image-based rendering using image over-segmentation. International Journal of Computer Vision (IJCV), 75:49–65, October 2007.

Article Download: https://infoscience.epfl.ch/record/177415/files/Superpixel_PAMI2011-2.pdf
Source Address: Http://ivrl.epfl.ch/research/superpixels