Graph-based image segmentation (graph-based image segmentation)

First, Introduction

Image segmentation based on graph (graph-based image segmentation), paper "Efficient graph-based Image segmentation", p. felzenszwalb, D. Huttenlocher , international Journal of Computer Vision, vol. 2, September 2004

The paper downloads and the paper provides the C + + code here.

Graph-based segmentation is a classic image segmentation algorithm, and its author Felzenszwalb is also a Daniel that proposes the DPM (deformable Parts Model) algorithm.

graph-based segmentation algorithm is based on the graph of greedy clustering algorithm, the implementation of simple, faster speed, accuracy is also OK. However, the current direct use of it to do the segmentation should be relatively small, a lot of algorithms used it as a stepping stone, such as object propose of the mountain "segmentation as selective Search for Object recognition" It is used to produce over-segmentation.

Second, the basic concept of the graph

Because the algorithm is to abstract the image with a weighted graph, so some basic concepts of supplementary graphs.

1. Figure

is composed of the vertex set V (vertices) and the edge set E (edges), expressed as g= (V, E), vertex v∈v, the paper is a single pixel point, the edge connecting a pair of vertices (vi, VJ) has the weight W (vi, VJ), the meaning in this article is the non-similarity between vertices (dissimilarity), which is used in the direction of the graph.

2. Tree

Special graphs, any of the two vertices in the graph, have paths connected, but no loops. A diagram with a thick edge attached to the image below. If you are looking at a group of beads, just keep the beads and wires in the tree, then pick a bead, and all the beads in the tree can be raised.

If the edges of vertex I and H are also preserved, the vertex h,i,c,f,g form a loop.

3. Minimum spanning tree (minimum spanning trees)

A special tree, given the vertices that need to be connected, select the smallest tree with the right edge.

In this paper, each pixel in the initialization is a vertex, and then gradually merge to get an area, or rather, an MST that connects the pixels in the area. As shown below, the brown circle is a vertex, the segment is an edge, and the MST generated by the Brown Vertex is merged, corresponding to a segmented area. The result of the split is actually the forest.

Third, similarity

Since it is a clustering algorithm, it should be based on what rules to determine when to merge, when should continue to draw the boundaries. For isolated two pixels, the difference is the gray value, naturally using the distance of gray scale to measure the similarity of two points, this article is the use of RGB distance, that is

Of course can also use perceptually uniform luv or Lab color space, for grayscale image can only use brightness value, in addition, you can use texture feature filtering, and then calculate the distance, such as first do census transform and then calculate Hamming Distance distance.

Iv. Global threshold >> Adaptive thresholds, intra-class differences in regions, inter-class differences

It is mentioned that the difference between the two pixel points should be measured by the difference of luminance values. For the similarity of two regions (sub-plots) or one region and one pixel, the simplest approach is to consider only the non-similarity of the edges that connect the two. As shown in the figure below, two areas of brown and green have been formed, and the two regions are now judged by their purple edges to merge. Then we can set a threshold that is merged when the difference between two pixels is less than that value. Iterative merging, eventually merging into a region, the effect is similar to regional growth: a single spark, can be a prairie fire.

To illustrate:

For the upper right, it is clear that the upper-left image should be clustered in 3 categories: High frequency zone H, slope area S, flat zone p.

If we set a global threshold, then if the H-zone is to be merged into a block, then the threshold will be very large, but that will also include the P and S regions, and the segmentation result is too coarse. If P is the reference, then the threshold value should be selected for a particularly small value, so that the P-zone will be merged into a piece, but the H-zone will be merged into a particularly special number of small pieces, like a fragmented mirror, the segmentation results are too thin. Obviously, the global threshold is not appropriate, so it is natural to use adaptive thresholds. For the P-zone, the threshold is particularly small, the S-area is slightly larger, and the H-zone is huge.

First, two definitions, the original text based on these two additional information to get the adaptive threshold value.

Intra-class variance Int (C)within a region:

Can be approximated as the maximum luminance difference within a region, defined as the largest edge of the MST in the same degree of similarity.

Inter-class difference diff for two regions (C1, C2):

That is, the non-similarity of the edges with the least similarity between all the edges of the two regions, that is, the similarity of the most similar places in the two regions.

Intuitive judgment, when:

, two regions should be merged.

Five, the algorithm steps

1. Calculate the similarity of each pixel point to its 8 neighborhood or 4 neighborhood.

As shown above, the solid line is only calculated 4 fields, plus the dashed lines are calculated 8 neighborhood, because it is a graph, from left to right, from top to bottom in order to calculate, only need to calculate the right of the Gray line in the figure.

2, the Edge according to the non-decreasing arrangement (from small to large) sort to get E1, E2, ..., en.

3. Select ei

4, the current selection of the Edge EJ(VI and VJ does not belong to a region) for the combined judgment. Set the vertex to which it is connected (vi, VJ),

If the similarity degree is less than the internal similarity:

5. Update thresholds and class labels

Else

6, if I < n, then in the order of the row, select the next edge to step 4, or end.

Vi. The code provided in the paper

Open the connection at the beginning of this blog post, go to the paper website, download C + + code. After downloading, make compiles the program. Command line run Format:

/********************************************
Sigma  Gaussian filtering of the original image
K control the      number of areas after merging
min:   After processing parameters, there will be a lot of small areas, when the number of pixels in the region is less than min, select the region with the smallest difference to merge
input  image (ppm format)
output image (ppm format)

sigma:used To smooth the input image before segmenting it.
K:     Value for the threshold function.
Min:   Minimum component Size enforced by post-processing.
Input:input image.
Output:output image.

Typical parameters is Sigma = 0.5, K = $, min =.
Larger values for k result in Larger, the result.
*/
./segment Sigma K min Input Output

Seven, OpenCV3.3 cv::ximgproc::segmentation::graphsegmentation class

/opencv_contrib/modules/ximgproc/include/opencv2/ximgproc/segmentation.hpp