Easypr-detailed development (3) Gaussian blur, grayscale, and Sobel Operators

　　In the previous article, we learned all the steps in the platelocate process. In this article, we analyze the first three steps, Gaussian blur, grayscale, and Sobel operators.

1. Gaussian blur

　1. Objectives

Remove image noise and prepare for edge detection algorithms.

　2. Results

In our License Plate positioning, the first step is Gaussian fuzzy processing.

　　

Figure 1 Gaussian blur effect

　3. Theory　

For more information, see this article: Ruan Yifeng describes Gaussian blur.

Gaussian Blur is a very famous image processing technology. As its name implies, it is generally used to blur the image, but Gaussian Blur is also applied to the image pre-processing stage. Before understanding Gaussian blur, Let's first look at the average fuzzy algorithm. The average fuzzy algorithm is very simple. See, the value of each pixel takes the average value of all the surrounding pixels (8 in total.

Figure 2 average blur

In the middle, the pixel value of the left red point is originally 2. After being blurred, it becomes 1 (taking the mean of all the surrounding pixels ). In the average blur, the weights of the surrounding pixels are the same, and they are all 1. If the weights of the surrounding pixels are different and the values of the two-dimensional Gaussian distribution are the same, Gaussian Blur is called.

In the above fuzzy process, each pixel takes the average value of the surrounding circle, also known as the Blur radius of 1. If three circles are taken, the radius is 3. If the radius increases, it will be more blurred.

　4. Practice

In platelocate, Gaussian Blur is called in this way.

// Gaussian blur. The number in the size field affects the License Plate positioning effect. Gaussianblur (SRC, src_blur, size (m_gaussianblursize, m_gaussianblursize), 0, 0, border_default );

The size parameter specifies the radius of Gaussian blur. The value is the m_gaussianblursize variable of the cplatelocate class. Because the Gaussian blur of opencv only receives an odd radius, an exception is thrown when the variable is an even value.

Here we provide the Gaussian Blur API of opencv (English, 2.48 or later ).

Is the process of Gaussian Blur necessary. The author's answer is necessary. If we comment out the code and make some modifications, run it again. You will find that the platelocate process changes when the operation is closed. The final result is as follows.

Figure 3 result without Gaussian blur

We can see that the rectangle where the license plate is located is skewed. The following figure shows the candidate "license plate:

Figure 4 "license plate" block without Gaussian blur

If the edge detection algorithm is used without Gaussian blur, the number of candidate "license plates" is 8! This not only increases the processing time of the license plate judgment, but also increases the probability of an error. As the license plate block is skewed, if our character recognition algorithm requires a horizontal license plate, we will almost certainly not be able to get the correct character recognition effect.

The radius in Gaussian Blur will also bring significant changes to the result. In some images, if the Gaussian Blur radius is too high, the license plate cannot be located. In some images, the Gaussian Blur radius is low, and the license plate cannot be located. Therefore, the radius of Gaussian Blur cannot be too high or too low. The static constant default_gaussianblur_size with a value of 5 in the cplatelocate class indicates the radius of the Gaussian blur. This value is the highest overall positioning rate obtained after testing for nearly images. In the constructor of the cplatelocate class, m_gaussianblursize is assigned the default_gaussianblur_size value. Therefore, the default Gaussian Blur radius is 5. If not, you do not need to modify it.

After several experiments, we must acknowledge that the best practice is to retain the Gaussian Blur process and the radius value of 5. To meet special requirements, the cplatelocate class should also provide a method to modify the Gaussian radius value. The call code (assuming a Gaussian Blur radius of 3 is required) is as follows:

    CPlateLocate plate;    plate.setGaussianBlurSize(3);

Currently, the process of easypr is first Gaussian blur and then grayscale. According to the current experimental results, the color-based Gaussian fuzzy process is easier to detect edge points than the Gaussian fuzzy process after gray scale.

 

Ii. grayscale Processing

　1. Objectives

Prepare a grayscale environment for edge detection algorithms.

　2. Results

The grayscale effect is as follows.

Figure 5 grayscale Effect

3. Theory

In the grayscale processing process, the biggest controversy is the loss of information. Undoubtedly, the original platelocate process is faced with a color image, and after this step, it will face a gray image. As mentioned above, the advantages and disadvantages of this step need to be discussed.

Undoubtedly, for computers, color images are much more difficult to process than gray images. Many image processing algorithms are only applicable to gray images, such as the Sobel operator mentioned later. In this case, you have no choice but to convert an image into a grayscale image before processing it, unless you re-design the algorithm. On the other hand, after being converted to a grayscale image, the most detailed information is lost. You must know that the real world is colored, and that human identification of things is based on the color frame. It can even be said that because our naked eyes can distinguish between colors, we have a strong ability to distinguish, distinguish, and remember things.

The same is true for removing color from the License Plate positioning process. Although converting to a grayscale image is helpful for using a variety of dedicated algorithms, it loses the most important tool in the real world-the distinction of colors. For example, how do people find a license plate in an image? Very simple. At a glance, a rectangle of the right size, blue, or yellow, or other colors, is in another black or white rectangle similar to the shape of the car. This process is very intuitive and obvious, and can eliminate many effects such as blur, color, and unclear. If grayscale images are used, you must use horizontal and vertical derivation methods.

In the future, if the platelocate process can be determined by color, it may be clearer and more accurate than the current positioning. However, this requires research and experiment, which may be implemented in future easypr versions. However, color recognition is undoubtedly a trend because it not only conforms to the pattern of human eye recognition, but also approaches the essence of artificial intelligence, and is more accurate and faster.

　4. Practice　

In the platelocate process, grayscale is called in this way.

cvtColor( src_blur, src_gray, CV_RGB2GRAY );

Here we provide the phased-out API of opencv (English Version 2.48 or later ).

 

Iii. Sobel operator

　1. Objectives

Detect the vertical edges in the image to facilitate the identification of license plates.

2. Results

　　Is the result of the Sobel operator.

Figure 6 Sobel Effect

　3. Theory　　

If you want to determine which step is the core and soul of platelocate, it is undoubtedly the Sobel operator. Without the Sobel operator, there will be no vertical edge detection, and the possible location of the license plate will not be obtained, so there will be no subsequent series of license plate judgment and character recognition processes. Through the Sobel operator, we can easily obtain a relatively accurate position of the license plate and lay a solid foundation for our subsequent processing. During the execution of platelocate, we can see that it is through the Sobel operator that the characters in the license plate are clearly distinguished from the background of the vehicle, laying the foundation for binarization and closed operations. So how does the Sobel operator work?

The soble operator is used to obtain the horizontal and vertical derivative of the First Order of the image, and determine whether it is an edge based on the value of the derivative. For more information, see the blog of csdn Xiao Wei (Be careful when the GX and Gy are reversed in her blog ).

For ease of calculation, the soble operator does not actually perform derivation. Instead, it uses the weighted sum of peripheral values, which is technically called convolution ". The weight is called a convolution template ". For example, the left side is the GX convolution template of Sobel (vertical edge calculation), the center is the original image, and the right side is the new image after the convolution template.

Figure 7 Sobel operator GX

The following example shows that the red pixel of the original image is 5 in a convolution template, after convolution calculation (-1*3-2*3-1*4 + 1*5 + 2*7 + 1*6 = 12), the value of the red pixel is changed to 12.

4. Practice

It takes many steps to call the soble operator in the code.

    /// Generate grad_x and grad_y    Mat grad_x, grad_y;    Mat abs_grad_x, abs_grad_y;    /// Gradient X    //Scharr( src_gray, grad_x, ddepth, 1, 0, scale, delta, BORDER_DEFAULT );    Sobel( src_gray, grad_x, ddepth, 1, 0, 3, scale, delta, BORDER_DEFAULT );    convertScaleAbs( grad_x, abs_grad_x );    /// Gradient Y    //Scharr( src_gray, grad_y, ddepth, 0, 1, scale, delta, BORDER_DEFAULT );    Sobel( src_gray, grad_y, ddepth, 0, 1, 3, scale, delta, BORDER_DEFAULT );    convertScaleAbs( grad_y, abs_grad_y );    /// Total Gradient (approximate)    addWeighted( abs_grad_x, SOBEL_X_WEIGHT, abs_grad_y, SOBEL_Y_WEIGHT, 0, grad );

Here we provide the Sobel API of opencv (English, 2.48 or later)

Two constants sobel_x_weight and sobel_y_weight in the call parameters represent the horizontal and vertical weights. The former is 1 by default, and the latter is 0, indicating that only the horizontal direction is obtained, without vertical direction. The significance of this is that if we perform vertical direction derivation, many horizontal edges will be detected. Many horizontal edges may help to generate more precise outlines. However, because there are too many horizontal edges at the front end of some cars, such as air holes and signs at the front of the front, many horizontal edges may mislead our connection results, as a result, we cannot get a correct license plate location. For example, if we perform the following experiment on the test graph and set sobel_x_weight and sobel_y_weight to 0.5 (which indicates that the weights of the two are equal), the final closed operation result graph is

Because Sobel operators are so important that license plates can be clearly distinguished from other regions, the problem arises. Is there any operator similar to the Sobel function that can achieve the same effect, or is there a better operator than Sobel?

The Sobel operator is used to obtain the first derivative of the image, while the Laplace operator is used to obtain the second derivative of the image. In general, the edge can also be detected. However, the Laplace operator does not detect verticals or verticals. It is a comparison between the Laplace operator and the Sobel operator.

Figure 8 Sobel and Laplace
　　

It can be seen that the image using the Laplace operator contains the horizontal edge and the vertical edge, according to our description just now. The horizontal edge is generally harmful to the detection of license plates. After tests on nearly images, the Sobel operator is superior to the Laplace operator, so it is not suitable to use the Laplace operator instead of the Sobel operator.

In addition to the Sobel operator, there is also a shcarr operator. However, this operator is actually only a variant of the Sobel operator. Because the Sobel operator is not accurate in the 3*3 convolution template, there is a special Sobel operator, its weights are expressed by Scharr operators. Is a comparison between Sobel operators and Scharr operators.

Figure 9 Sobel and Scharr

In general, the Scharr operator is better than the Sobel operator in edge detection. However, this "better" is a double-edged sword. Our goal is not to draw an image edge, but to determine an area of the license plate. The finer the edge, the more it will interfere with the closed operation. Therefore, for testing a large number of images, Sobel operators are generally better than Scharr operators.

For more detailed explanations of Sobel operators and the similarities between Scharr operators and Sobel operators, refer to the introduction on the official website: Sobel and Scharr.

To sum up, the Sobel operator is the best operator that fits the needs of plate number locating in the process of finding the image edge. The Laplace operator is inferior to Scharr operator.

Note: The Sobel operator can only work on grayscale images and cannot use color images as input. Therefore, the previous gray-scale chemical work must be performed before the soble operator is implemented.

Copyright description:

All texts, images, and codes in this article are copyrighted by the author and the blog. You are welcome to reprint it, but be sure to indicate the author and source. Any unauthorized plagiarism and crawler crawling are infringing, and the author and blog garden reserve all rights.

 

References:

1. http://www.ruanyifeng.com/blog/2012/11/gaussian_blur.html

2. http://blog.csdn.net/xiaowei_cqu/article/details/7829481

3. http://docs.opencv.org/doc/tutorials/imgproc/imgtrans/sobel_derivatives/sobel_derivatives.html

Easypr-detailed development (3) Gaussian blur, grayscale, and Sobel Operators