A license plate Locating Algorithm Combining edge detection and scanning lines

Abstract: This paper proposes a method for plate locating based on edge detection, which combines binarization with scanning lines. Experiments show that the method can quickly and accurately locate the license plate, and has a strong anti-interference ability. The positioning accuracy is over 99%, and light and weather have little impact on the positioning result.

Keywords:Pattern recognition, License Plate Recognition, Edge Extraction

The License Plate Recognition (LPR) technology is a very important technology in Intelligent Transportation Systems. It integrates computer vision, image processing technology, and pattern recognition technology. Generally speaking, the processing technology in the early stage of License Plate Recognition is crucial. The preliminary technologies include License Plate positioning, binarization of license plate images, and character segmentation. This article focuses on License Plate positioningAlgorithm.

License Plate positioning is to locate the license plate area in the vehicle image. Because vehicle images are collected in natural environments, the imaging conditions of license plates and backgrounds in natural environments are generally uncontrollable, and the factors of random changes (especially illumination conditions) and complex background information bring huge difficulties to the target search. The color, brightness, and brightness of the license plate vary greatly under different lighting conditions. The background information is generally more complex than the license plate information, and some background areas may not differ much from the license plate area; coupled with the difference in camera distance and angle, it is very difficult to distinguish the target from various kinds of interference. The license plate area accounts for a small proportion of the entire image. To locate the license plate area from the entire image, you must search for a large amount of background information. Due to the particularity of the application, the license plate must be located quickly and accurately. If there is no efficient search method, it takes a lot of computing time and storage space. Therefore, the License Plate positioning technology has always been a difficult issue and is a key technical link of the License Plate Recognition System.

At present, many methods have been proposed to locate license plates. These methods share a common starting point, that is, determining licenses based on the characteristics of the license area. Based on different implementation methods, the existing positioning methods can be divided into two categories: Direct Method and indirect method.

(1) Direct Method: directly analyzes image features. For example, a line template-based corner detection algorithm for binarization images is proposed in the document. This algorithm uses the border corner of the license plate to detect the four corners of the license plate and locate the license plate. This document introduces an image region location algorithm based on linear edge recognition, and uses this algorithm to locate the border line of the license plate to locate the license plate. This document describes how to use texture features such as the size, Character spacing, and character features of a license plate to locate the license plate. Based on the rich features of license plate vertical frequencies, the paper first uses wavelet to extract the vertical high-frequency information of the image, and then uses Mathematical Morphology to perform a series of morphological operations on the detailed image after wavelet decomposition, eliminate useless information and noise to locate the license plate.

(2) Indirect Method: mainly refers to the method of locating license plates using neural network or genetic algorithms. Computing Using flexible methods such as Neural Networks and genetic algorithms is currently one of the hot topics. The document proposes to use the BP neural network to locate the license plate, and proposes a method to combine the discrete-time cellular neural network and fuzzy logic. The document uses multi-layer sensor networks (MLPN) to locate license plates. The document proposes to use the genetic algorithm to locate the license plate. It uses the genetic algorithm to optimize the image search, and combines the fitness function constructed by the regional feature vector to find the optimal positioning parameter for the license plate's versioning area.

According to the analysis and observation, compared with other areas in the vehicle image, the license plate area has the following features:

(1) the vertical edges in the character area of the license plate are relatively dense compared with the horizontal edges, while the horizontal edges of other parts of the vehicle body (such as bumpers) are obvious and the vertical edges are few. In addition, the license plate is generally suspended at a lower position on the body, and there is basically no obvious edge-intensive area below it.

(2) A series of province name abbreviations (Chinese characters) and region codes (English letters) are arranged with clear borders) and a five-character letter/number (common civil vehicle ). The color of the base card and character mainly includes four types: blue-white, yellow-black, black-white, and white-Black (or red. As the shooting angle and the damage degree of the license plate are different, the acquired license plate border is often skewed or broken.

Therefore, the direct Positioning method uses the border feature (Feature 2) use the model matching method to find the four corners of the license plate to locate the license plate or find the border of the license plate to locate the license plate in a straight line, and use the gray-scale change frequency of the license area (using feature 1) to locate the license plate.

Because the border of the license plate is sometimes contaminated, the gray-level frequency variation in the character area of the license plate is the most stable feature. Therefore, this paper proposes a method to locate the license plate based on the gray-level variation frequency in the character area of the license plate, this is a license plate locating algorithm based on the combination of edge detection and scanning lines. The idea of this algorithm is to first enhance the edge of the license plate image, and then use the horizontal scanning line to detect the license plate area.

1 license plate pre-processing

The function of image preprocessing is to highlight the useful information in the image. Different image preprocessing corresponds to different image segmentation to obtain the best license plate features. License Plate positioning preprocessing highlights the features of the license plate area and suppresses other useless features. The license plate positioning preprocessing aims to highlight the features of the license plate area and suppress other useless features. One of the main features of the license plate area is the dense vertical edges. Therefore, this paper proposes a difference operator for vertical edge enhancement.

For the sake of universality, this paper studies gray images. Because the image collected in this article is a color image, we need to convert the color image into a gray image. In order to reduce unnecessary color-grayscale conversion operations, this paper only processes the green components of the input color car image.

Edge Detection involves many operators. For example, Sobel operator, Robert ts operator, Prewitt operator, and Laplacian operator. Sobel, Roberts, Prewitt, and Laplacian operators are not used to detect vertical edges, and the calculation workload is large. Although only their vertical edge detection operators can be used, the calculation workload is relatively large. To this end, this article specifically designs a horizontal template operator, that is, [, 1 ..., 1, 1, 1]. This operator convolution with the image and then performs a difference operation with the original image. When the difference value is greater than a certain threshold value, it is considered as an edge target; otherwise, it is the background.

Horizontal template: [, 1 ..., Convolution with the image is equivalent to performing a low-pass filter in the horizontal direction of the image, and then performing a difference with the original image, the aim is to highlight the high-frequency information in the vertical direction of the image (equivalent to performing high-pass filtering on the image ). This operator can perform incremental operations, that is, when calculating the local average value, the sum of all points in the horizontal direction window is calculated first, subtract the result of the previous operation from the value of the leftmost vertex of the window and add the value of the new vertex on the right. This reduces the number of summation operations, so the calculation workload is less than that of the Sobel operator.

Operator description:

1/N [111... 1111]-[000... 1000] (1)

The left half of formula (1) is a horizontal template, that is, [, 1 ..., ,] Convolution with the image, the right half can be understood as the original image. The difference between them is the result of formula (1.

Because incremental operations can be performed, the length of the operator has little impact on the calculation workload. Here, 9 is used to match the width of the character stroke.

As shown in figure 1, the image is convolution and then differentiated from the original image. Only the gray-scale abrupt changes such as the license plate area, wheel, and headlight are relatively high, while the others are almost zero. After processing the entire image, the binarization Image 2 (a) of the vehicle image is obtained, and the differential image is binarization. Result 2 (B) (where the threshold value is 10 ).

2. Reserved position of the license plate

2.1 Long-range filtering and Particle Filtering

(1) Long-range Filtering

As shown in figure 2 (B), after preprocessing, many edge points are connected to a long-range band curve (especially in wheels and windows ). This is not the case in the license plate area. If no corresponding processing is performed, it will cause interference to the identification of the license plate. Therefore, long-range filtering should be performed before positioning.

The long-range filter algorithm is based on the following idea: if many vertices and edges are combined into a long-range curve L, the long-range filter tests the extent of L's spatial extension in the X and Y directions. If the threshold value is exceeded, all candidate vertices belonging to L are filtered out from the candidate point set.

(2) weeping Filter

After long-range filtering, there are still many candidate points in the image to form small particles separated from each other. This is not the case for candidate points in the license plate area. Therefore, granular filtering is performed to filter out granular noise.

This article defines two modules: Click the target template and hit the template, as shown in 3. As long as the product of the hit template and image exceeds a certain threshold, and the product of the hit template and image is less than a certain threshold, the image in the hit template is set to zero.

Particle Filtering is based on the idea that for a candidate particle set, a small rectangular area must exist around it (the size of the rectangular area is related to the particle size ), the surrounding area of this small rectangle contains very few candidate points (ideally, no candidate points ). Then, scan the entire graph in a small rectangle to check whether the number of candidate points around the rectangle is smaller than the threshold. If it is smaller than, all the candidate vertices in the rectangular area are considered as isolated granular commands. Deleting these vertices achieves the goal of particle filtering. Because of the different particle sizes, different particle sizes can be wiped several times. Figure 2 (B) shows the processing result 4 after long-range filtering and particle filtering.

2.2 license plate reservation

This article uses the continuous characteristics of license plate characters. The license plate area contains seven consecutive characters, and the distance between characters is within a certain range. This article defines a jump from the target to the background or from the background to the target. Compared with other non-license plates, the license plate area jumps more frequently, and the distance is within a certain range and the number of jumps exceeds a certain number. Generally, it is more than 14 characters. Because the license plate contains seven characters today, each character has more than two hops. To prevent the characters from being broken, blurred, and skewed, this article is conservative and uses 10.

Therefore, the scanning line in this article scans and locates the license plate in a binarization image (generally, the license plate is located at the bottom of the vehicle, and most of the text interference similar to the license plate text is located at the top ), scanning is performed sequentially from left to right and from bottom to top. The algorithm is as follows:

From bottom to top, Scan each line of the image from right to left.

(1) The current position is recorded at the hop point. If a row has more than 10 consecutive hop points, in addition, the distance between the previous hop and the next hop is within a certain range, and the starting position of the start point and the end point of the last hop is recorded.

(2) If there are 10 consecutive jump points or less, and the starting point of the adjacent upper and lower rows is adjacent to the ending point. This area is considered as the license plate pre-selection area. Figure 5 shows the Positioning Result of a typical vehicle image.

Based on the positioning results, this positioning algorithm is also suitable for vehicle images that contain multiple license plates, and the positioning speed is not significantly affected by multiple license plates. Although the lights and the characters on the car (including license plates, lights, and other vertical edge areas, and paid license plates) may also be positioned as pre-selected areas, most of them are above the license plate, this article also uses the bottom-up method to filter the pre-selected area, so it has little impact on the positioning speed. In more than 99% of the cases, the first pre-selection area is the license plate area 6 (a), (c), and there are few situations like Figure 6 (B. Based on this situation, this paper designs a positioning strategy: if real-time processing is required, only the first pre-selection area is selected and sent to the segmentation and recognition modules. If real-time is not required, the pre-selection areas can be separately sent to the segmentation and recognition modules. At the same time, this algorithm can accurately locate the 2000 type license plate (figure 6 (c. For the characters listed on the top of the 2000 type license plate, you can only use the feedback from the splitting module to further locate the left and right boundary of the license plate.
for most vehicles, license plate positioning is very accurate, but for some license plates, especially truck license plates, the license plate is likely to be stuck in some areas of the nearby car Texture profile, all of which constitute a candidate license area. Therefore, in order to extract the correct license plate area, we must try to remove the false candidate license area and separate the real license area from the candidate license area and the compound block. In actual scenarios, the areas pre-selected by the above algorithm are stuck with some texture outlines on both sides of the license plate, almost not with the upper and lower part of the license plate (because the algorithm only uses the vertical edge ), therefore, the height of the pre-selected area can be similar to the height of the license plate. Based on the prior knowledge of the license plate, the charwidth of the license plate character width can be estimated based on the license letter aspect ratio. See the algorithm of the document here. This algorithm estimates the license plate width platewidth by using the license plate character level (generally 10 x charwidth is used for general license plates). Meanwhile, according to the license plate image's vertical edge image, the license plate character height is concentrated, in other places, the texture features are relatively scattered and the estimated license plate width is used to automatically search for the location of the license plate character area, that is, the biggest vertical projection of the real license plate area edge. For detailed steps, see the document.

This method is used to determine the left and right boundary of the license plate in figure 7, as shown in result 8. The White Rectangle is the pre-selection area of the license plate. We can see that the license plate and the headlight are stuck. According to the length ratio of the predetermined position is greater than a certain value, we think that the license plate and the headlight are stuck in the area. Run the above algorithm to further determine the left and right boundary of the license plate. Result 8 is displayed.

3. Experiment results

Partial vehicle Image Positioning Result 9 is shown.

Result Analysis:

① Accuracy: The experimental results show that the accuracy of the License Plate positioning is greater than 99%. The license plate positioning area is slightly larger than the license plate when there is external interference and license plate tilt.

② Time features: the running time in the VC environment varies according to the license plate situation between 0.1 and 0.15 seconds (733 M memory piII ).

③ Adaptability: License Plate noise has no effect on positioning. License plate images can be extracted when the light is strong and the light is weak. However, when the license plate has a severe leg color, the positioning failure may be caused by the fact that no stroke or other edges are detected.

④ Other features: This positioning algorithm is suitable for vehicle images with multiple license plates, in addition, the positioning speed is not significantly affected by multiple license plates (at the same time, multiple vertical edges, such as incorrect positioning areas and headlight areas, and paid license plate areas may also occur, this can be discarded by the segmentation and recognition modules in the future ).