Image edge detection based on hed network TensorFlow and OpenCV

Traditional edge detection:

OpenCV inside of the two functions, Cv2. Canny () and cv2.findcontours ():

It looks like it's easy to come true, but the real picture is that it doesn't work in a complex background.

The detection effect of the canny algorithm relies on several threshold parameters, and the selection of these threshold parameters is usually a human-set experience value. In the process of improvement, when additional steps are introduced, a number of new threshold parameters are often introduced, as well as experience values that depend on the debug result settings. Overall, the number of these threshold parameters can not be particularly many, because once too many, it is difficult to rely on experience to set up, in addition, although there are these threshold parameters, but the final parameter is only a group or a few sets of fixed combinations, so the robustness of the algorithm will be discounted, it is easy to encounter the edge detection effect is not ideal.

HED (holistically-nested Edge Detection) Network:

Edge detection This demand, in the field of image processing, usually called Edge Detection or Contour Detection, according to this idea, found the holistically-nested Edge Detection network model. HED network model is designed on the basis of VGG16 network structure, so it is necessary to look at VGG16 first.

Above is the schematic diagram of VGG16, in order to facilitate the transition from VGG16 to HED, we first turn VGG16 into the following diagram:

OK, next throw the HED network structure:

HED Network is a multi-scale multi-fusion (Multi-scale and multi-level feature learning) network structure, so-called multi-scale, is as shown above, the VGG16 of each group of the last convolution layer (green part) of the output out, Because the length and width of the image given by each group is not the same, it is also necessary to use the transpose convolution (transposed convolution)/deconvolution (DECONV) for each set of image to do another operation, from the effect, the equivalent of the second to fifth group to get The length and width dimensions of the image are enlarged by 2 to 16 times times, so that the image obtained on each scale (each group of VGG16 is a scale) is the same size.

HED network git address written based on TensorFlow:

Https://github.com/s9xie/hed

after the hed is segmented out of the edge, it is optimized with OPENCV:

Although using neural network technology, has obtained a better edge detection than the canny algorithm, but the neural network is not omnipotent, interference is still there, so, the second step of the mathematical model algorithm is still needed, but because the first step in the edge detection has been greatly improved, So the algorithm in the second step is properly simplified, and the overall adaptability of the algorithm is stronger.

This part of the algorithm is shown in the following figure:

In order of numbering, several key steps do the following:

Using the HED network to detect the edge, you can see that the edge line obtained here is still some interference in the image obtained in the previous step, using the HOUGHLINESP function to detect the line segment (blue Line) to the previous step to extend the line (Green Line) in the second step of the detected line segment, some are very close, Or some of the short-term segments can be connected to a longer line segment, so you can use some strategy to merge them together, this time, you need to use the third step to get the straight line. Defines a strategy to determine whether two lines are equal, and when two straight lines are encountered, the corresponding segments of the two lines are merged or joined together into a line segment. When this step is complete, the next steps require only the Blue line segments and do not need the green lines to calculate the intersection between them according to the fourth step, the intersection of the adjacent points can also be merged, at the same time, each intersection point and the line that produces this intersection is also associated (each blue point, Have a set of red lines and associated with it) for the fifth step to get all the intersection points, each time to take out 4 of them, to determine whether the 4-point quadrilateral is a reasonable rectangle (with perspective transformation effect of the rectangle), in addition to the conventional judgment strategy, such as angle, edge length of the ratio, There is also a judging condition is whether each side can and fifth step to get the corresponding point of the associated segment coincident, if not coincident, then this quadrilateral is not likely to be we expect to detect the rectangle after the sixth step of filtering, if you get a plurality of quads, you can use a simple filtering strategy, such as sorting to find the perimeter or the largest area of the rectangle