Image segmentation-graph-based Image Segmentation)

Zookeeper

Image segmentation-graph-based Image Segmentation)

Reference:

Efficient graph-based image segmentation, ijcv 2004, MIT code

Image segmentation-image-based image segmentation (opencv source code)

 

Graph-based segmentation is a classic image segmentation algorithm. The author felzenszwalb is also the master of DPM algorithm. This algorithm is graph-based.Greedy ClusteringThe algorithm is easy to implement, fast, and accurate. However, at present, it should be less widely used for segmentation. After all, it is a century old man in 99 years, but many algorithms use it as a stepping stone, for example, segmentation as selective search for object recognition, which is an example of object propose, is used to generate oversegmentation ). The senmatic segmentation algorithm is used to generate superpixels ......

Basic Concepts

Because this algorithm abstracts pictures and represents them with weighted graphs, it complements some basic concepts of graphs.

FigureYesbyVertexVertices andEdgeSet (edges), represented as, vertex, in this article is a single pixel, connected to a pair of vertex edge hasWeightIn this article, the meaning is between verticesNoSimilarity, which isUndirected graph.

Tree:In a special graph, any two vertices in the graph are connected by paths, but noLoop.Ru ZhongBoldGraph formed by the edge. If we see a mess of beads,Retain onlyThe beads in the tree and the link, then you can pick up all the beads in the tree. If the edge I and H are retained, H, I, C, F, and g constitute a loop.

Minimum Spanning Tree(MST, minimum spanning tree): special tree. For the vertex to be connected, selectTree with the smallest sum of edge weights.Is an MST

In this article, each pixel is a vertex during initialization, and then gradually merged to get a region, specifically a MST connecting the pixel in this region ., The brown circle is the vertex, the line segment is the edge, and the MST generated by the brown vertex is merged, which corresponds to a split area. The split result is actually forest.

 

Similarity

Since it is a clustering algorithm, which rule should be used to determine when to merge and when to continue to draw a line?

ForTwo isolated pixelsThe difference is the color. Naturally, we use the distance of the color to measure the similarity between the two points. In this article, we use the RGB distance, that is

Of course, you can also use the perceptually uniform luv or LAB color space. For grayscale images, you can only use the brightness value. In addition, you can also use texture feature filtering before calculating the distance. For example, perform census transform first and then calculate the Hamming distance.

Global threshold à Adaptive Threshold

The difference between two pixels should be measured by the Brightness Value Difference mentioned above. For the similarity between two regions (subgraphs) or one region and one pixel, the simplest method is to consider only the non-similarity of the edge connecting the two.

The brown and green areas have been formed. Now, the purple edge is used to determine whether the two areas are merged. Then we can setThreshold, When the difference between two pixels (that is, no similarity) is smaller than this value, combine them into one. Iterative merge will eventually be merged into multiple regions. This is the basic idea of regional growth: the fire of stars can be restored.

Obviously, the figure above should be grouped into three types of things on the right, high-frequency Zone H, slope zone S, and flat zone p. If we setGlobal thresholdIf the H area is to be merged into one partition, the threshold value is very large, but the P and S areas will also be included and the split result will beToo rough. If we use P as a reference, we should select a very small value for the threshold value. In this case, the P zone will be merged into one piece, but the H zone will be merged into a very large number of small pieces, split results like a broken mirrorToo fine.

Obviously, the global threshold is not suitable.Adaptive Threshold. For the p-zone, the threshold is very small, the S-zone is slightly larger, and the H-zone is huge.

For two regions (component in the original text, which is essentially an MST, a separate pixel can also be regarded as a region), this article uses a very intuitive method, but the anti-interference is not strong. Two definitions are given first. The original Article obtains the Adaptive Threshold Value Based on the two additional information.

In a regionIntra-class differences:

It can be roughly understood as the maximum brightness difference value within a region. It is defined as an edge with the highest non-similarity in the MST.

TheInter-class differences:

That is, the similarity between the edges of the two regions is the least similarity, that is, the similarity between the two regions.

Then you can intuitively determine whether to merge:

Equivalence Condition

Explanation: the difference is the biggest difference between the region and the tolerable one. When both of them can tolerate the current difference, you may feel the same. If one party does not want it, you cannot force it.

In special cases, when both are isolated pixel values, all pixels are "zero-tolerance" and can be merged only when the pixel values are identical, which naturally leads to over-segmentation. Therefore, at the beginning, we should set a tolerable range for each pixel. When it grows to a certain extent, we should remove the effect of this initial tolerable value. The original Article conditions are as follows:

Add:

The number of pixels contained in the area. As the area expands, the effect of this item becomes smaller and negligible. This is the size of a region that can be controlled. If so, almost every pixel becomes an independent area. If it is clear that the entire image will be merged. Therefore, the larger the image, the larger the split image.

Of course, median can be used to cope with hyperadjustment, but this becomes an NP-hard problem. For proof, see the original article.

Similar shape

As mentioned above, color information is used for clustering and similarity measurement standards can be modified to form the desired shape. For example, if we want to get a lot of long-stripe areas, we can useArea/Perimeter + Brightness Value DifferenceMeasure the similarity between two subgraphs or two pixels. Because the area/perimeter of a long bar is relatively small.

Algorithm steps

Step 1: Calculate the similarity between each pixel and its eight or four neighbors.

As shown on the left, the solid line is only used to calculate four fields, and the dotted line is used to calculate eight adjacent areas. Because it is an undirected graph, it is calculated from left to right, from top to bottom, you only need to calculate the gray line on the right.

Step 2:SetEdgeBased on Non-SimilarityNon-decreasing arrangement (From small to large)Sorted.

Step 3:Select

Step 4:Merge and judge the selected edge. Set the connected vertex. If the merging conditions are met:

(1) does not belong to the same region;

(2) The non-similarity is not greater than the internal non-similarity between the two. Then runStep 4. Otherwise, executeStep 5

Step 5:Update threshold and class label.

Update class labels: unify the class labels into labels.

The update non-similarity threshold for this class is :.

Note:Because edges with low similarity are merged first, they are the largest edges of the merged area.

Step 6:If so, select the next edge to executeStep 4Otherwise, it ends.

Result

Segmentation parameters: Sigma = 0.5, K = 500, min = 50.

Sigma:First, Gaussian filter is used to remove noise from the original image. Sigma is the Gaussian Kernel.

K:Control the size of the merged area. See the previous article.

Min:After the post-processing parameter is split, there will be many small areas. When the number of regional pixels is smaller than min, merge the regions with the smallest difference.

Nature Discussion

Although the results are not very good, they have a good global nature, and the conclusions are very interesting. If you are interested, you can look at them.

The first thing to note is that for any image, alwaysExistA segmentation method, so that the results of the segmentation are neither small nor rough. HoweverNot unique.

Theorem

If Step 4 is not merged, that is, there must be a region that has been split. For example, the range of the region will not be increased, it will become a region in the final split area.

Proof:

Assume that because the edges are sorted by non-decreasing, the similarity between the remaining connected edges is certainly not lower than that of the smallest edge. The remaining edges are naturally placed on the edge.

However, in the original article, only one partition has been divided, but I think there is another situation. In addition, it is only right that the two zones should be divided.

Not too fine

Segmentation is too fine, that is, the areas that should not be separated should be intercepted, but this algorithm can ensure that there are lovers eventually, and it will never be a great deal to break up a pair of things.

Proof:

Proof Method: for example. This should not be split, but the conditions should be met. If the two are separated, there must be an edge which will lead to the two not merged. Then, from the above theorem, there must be a region which will become part of the final split result. Suppose it is part, when we look back to the judgment of this edge, there must be, and thus, because it is in the non-decreasing order, the smallest side of Part A and Part B is, then it is in conflict with the assumptions.

Not too coarse

The split is too rough, that is, the areas that should be separated are not separated. However, this algorithm ensures that the system is disconnected when it is disconnected and will not be disconnected.

Proof Method: for example. This should be split, and the conditions should be met. Assume that it is the smallest edge connecting a and B. If it is merged, and it is in the non-decreasing order, the region has been formed before the edge is determined. If the split is too rough, the smallest side of the edge is satisfied, and the two must be merged. Conflict with conditions.

Influence of equi-edge processing order

If the weights of the two edges are the same, who is in the front of the sorting and who is in the front of the ranking will have an impact on the lagging sides? The conclusion is that Mu you.

Proof:

Case1:, The connected areas are the same, that is, the connected areas are all areas, so it does not matter who is in front of them.

Case2: The connected areas are completely different, such as the connected areas, and the connected areas. Then, the merging or merging will not be affected.

Case3: Connection

Case3-1: first, after, And, makes the merge, exchange the two processing order, first processing, after processing. If merging is not performed, the merging is not affected. If merging is performed, the merged operations are still performed.

Case3-2: first, after, and, do not merge, exchange the processing order of the two, first processing, after processing. If yes. Whether or not to merge will not enable the merge. If so, there will also be no impact.

Supplement: color pictures

For color images, the preceding section uses the R, G, and B as the distance. The entire image is only divided once. The original Article says that each channel is divided once, and the result is intersection, that is to say, to divide the two points in the image into the same region, they must always be in the same region in the Division results of the R, G, and B channels. In the original article, the effect is better ...... However, his program uses one split.

Nearest Neighbor Graphs

In the previous article, we used only space locations to component the graph's connection relationship. The disadvantage was obvious, the space was not adjacent, and the color was completely the same, so a slight disconnection in the middle would be divided into multiple parts. So the otherEqualityThe policy is to map the two to the feature space before creating a graph. In this case, the connection relationship is not necessarily the 4/8 neighbor. Because there is a pair of edges, It is terrible to consider the connection relationship between all edges! The original text is to find 10 points closest to the Euclidean distance for each pixel, that is, 10 nearest neighbor, to build a graph. Of course, another method is not to set the number of neighbors, but to limit the distance range.

The explanation of intra-class distance is intuitive, and the shortest distance within the class will take this edge as the radius to form a hyper-sphere in the feature space, but it will overlap with others.

It is also the shortest distance between two classes.

It is also quite tiring to find 10-nn. The original Article uses the approximate algorithm Ann "approximate nearest neighbor searching" to find 10 nearest neighbors, which is fast.

The rest is the same as above, but I do not understand it. For example, if we use the Green Line for update, the meaning is no longer the shortest radius of all vertices in the set, solving?

The result is as follows: the grass separated by the railing is also connected, and the flowers below belong to the same category.

Zookeeper

Image segmentation-graph-based Image Segmentation)