Design and realization of vehicle license plate location recognition system

Http://www.eeworld.com.cn/qrs/2015/1023/article_25403.html

I. Project background and feasibility analysis

2.1 Project background and technical difficulties

• Project Name: Intelligent Transportation: Vehicle license plate location recognition;

• Project content: This project is in real-time FPGA front-end image acquisition, preprocessing, license plate location and character segmentation and data transmission work, in the backend to complete the license plate character recognition work. FPGA receives the real-time image of the acquisition, in the internal pipeline method to complete the image preprocessing, license plate location and license plate character segmentation work, and finally through the high-speed USB port to transfer the segmented characters to the backend for character recognition. The image acquisition and data transmission is realized by EDK embedded Microblaze kernel control, which simplifies the FPGA internal control circuit, and the pipeline method used in the FPGA internal signal processing circuit greatly improves the system's ability of real-time processing video frame.

• Technical difficulties: (1) The diversity of the license plate's own characteristics and the uncertainty caused by the external environment are inaccurate; (2) The real-time is not satisfied by the high computational complexity of the image processing algorithm.

2.2 Key technologies and innovation points

• Algorithm: a license plate location algorithm based on edge detection and region search is adopted, which is intuitive and computationally small, which is advantageous to hardware implementation.

• The circuit realizes: The FPGA uses the pipeline method to complete the image processing, the license plate localization and the character segmentation three parts in turn, greatly improves the real-time processing ability to the video stream;

• Embedded control: Using Xilinx EDK embedded Microblaze Soft core control to complete image acquisition and data transmission work, simplifying the control circuit inside the FPGA.

2.3 Performance Index and feasibility analysis

• Set performance indicators: China's TV video standard using PAL system, its frame rate of 25, so set up the system to achieve the processing speed of 25 frames per second, and can be the character segmentation information via the USB port upload to the PC or other devices.

• Feasibility Analysis: This program has been simulated by MATLAB, the results show that the license plate positioning, segmentation and identification can be effective. Under the premise of guaranteeing the effect of license plate recognition, the real-time requirement can be achieved by improving the algorithm structure and combining the FPGA realization of pipelining. In addition, the team members have a wealth of FPGA design skills and a solid knowledge of signal processing theory, the ability to ensure the successful delivery of the project.

Ii. Overview of the Software design scheme

Car license plate Recognition system is generally divided into three parts, that is, license plate area location, license plate character segmentation and license plate character recognition. In order to highlight the useful features of the image, it is usually necessary to pre-process the image before the license plate is positioned to achieve better positioning effect. Figure 3-1 is the design of the software program flowchart, in which the license plate region positioning part of the design uses a edge detection and regional search for the license plate location segmentation algorithm, the simulation results are shown in the appendix.

Figure 3-1 Flowchart of software design scheme

3.1 Image preprocessing

Pre-processing the image of the camera before the license plate location refers to highlighting the useful information in the image, suppressing the invalid information which may adversely affect the next step, in order to reduce the complexity of the operation and improve the recognition effect. Image preprocessing mainly includes three parts: image grayscale, denoising and gray-scale stretching.

3.1.1 Rgb2gray

The images taken by the camera are usually RGB color images, each pixel includes R, G, b three color components, each color component is expressed as 8 bit, that is, the "a" is a pixel. Grayscale images are images that contain only the luminance components, each pixel is represented by 8 bit, and the luminance value is quantified to 256 levels. for license plate recognition, gray image is sufficient to meet the requirements, and compared with RGB image has the advantages of low computational complexity and small storage space. Therefore, the RGB color image can be converted to grayscale image, the calculation formula is Gray = 0.299R + 0.587G + 0.114B.

3.1.2 Image de-noising

The energy of the image is mainly concentrated in the low frequency part, the high frequency part is the edge information, and the noise is usually concentrated in the high frequency part, so the image denoising is needed to make the edge detection in the license plate location better. Median filtering is a nonlinear image processing method, which replaces all the pixels in a window in the center of the window, and can effectively protect the edge of the image while removing the noise, so this scheme chooses the median filtering method to de-noising.

3.1.3 Grayscale Stretching

In order to enhance the contrast of vehicle image and license plate image, it is advantageous for license plate location and recognition, and the gray-scale stretching of image after denoising is necessary. Gray-scale stretching refers to the gray scale of a point in the input image, which is mapped into gray-scale output by the segmented mapping relationship T.

3.2 License plate Positioning

License plate location refers to the segmentation of the license plate region from the correct recognition in the image, whose basic principle is to mark the license plate area in the image according to the prior knowledge of the target and background, and to complete the effective segmentation. In the image, the license plate region has obvious texture features and rich edge information, if the whole image is processed by edge detection, the license plate region contains more detail edge information relative to other non-license plate regions. Therefore, this scheme chooses a license plate location segmentation algorithm based on edge detection and region search, which is intuitive and low in operation complexity, which is advantageous to hardware implementation.

3.2.1 Sobel Edge Detection

The essence of edge detection is to use some algorithm to extract the intersection between object and background in the image, including filtering, image enhancement, detection and positioning four parts. Sobel edge detection refers to the template convolution of the Sobel operator and the given image, then analyzes the direction of each edge point in the convolution result image, and takes the local maximum point of the image as the edge output, so that the edge information of the image can be obtained. Relative to the general simple convolution and threshold processing method, Sobel edge detection can reduce clutter, suppress noise, better highlight the texture features of license plate area, and its low computational complexity, hardware implementation is less difficult, so we use Sobel edge detection to extract image edge information.

3.2.2 License Plate Area Search

There is a relatively continuous gray-scale jump in the license plate area, and the interval of two jumps is within a certain range, and the non-license plate region generally does not have this feature. According to this characteristic, we use the method of row scan and column scan to search the upper and lower boundary and the left and right boundary of the license plate area respectively.

The row scan determines that the upper and lower bounds refer to left to right, from the top down to scan each row of pixels, if you encounter a jump point count plus 1, if a row in the jump variable is greater than the threshold m, it is considered to be a possible photographic area, and set it as the lower boundary; if the number of rows with a jump variable greater than m exceeds the threshold N, the true license area is considered The last line is set to the upper boundary, otherwise the row count is 0 and the bottom boundary of the license plate is re-searched.

The column scan determines that the left and right boundary is the number of license plate areas that have been determined to the upper and lower bounds, searching from left to left, searching from right to left, stopping and recording the first jump point, and then jumping to the next line. This will obtain both the left and right borders of the license plate area.

3.2.3 Candidate License Plate area determination

There may be more than one candidate region searched by the above method, so they need to be judged to determine the true license plate area. The criteria for determining the license plate area include the size of the license plate and the length-width ratio, the distribution of the pixels, whether the binary projection is the crest-trough distribution, etc., because the license plate size is the known condition, so the size and length-width ratio is used as the criterion to determine the real license plate area.

3.3 License plate Character segmentation

As the license plate image is likely to appear to the left or right tilt, and any kind of tilt will affect the division of license plate characters, so the search out of the license plate area needs to be pre-processing, in order to achieve better character segmentation recognition effect.

3.3.1 License plate Area pretreatment

License plate area pretreatment generally includes two value, geometric transformation and other steps, the effect will directly affect the subsequent character segmentation recognition work. The license plate area consists of foreground character and background color, which is equivalent to determining appropriate threshold separating characters and backgrounds, which can greatly reduce the computational complexity of subsequent work; Geometric transformation refers to the image of the license plate by scaling, rotation, translation, etc., to correct the deformation and tilt of the license plate, to increase the number of characters segmentation, recognition rate.

3.3.2-character Segmentation

Character segmentation refers to the segmentation of license plate area images into a single character image, which belongs to the problem of image segmentation. Character segmentation is the precondition of feature extraction and pattern matching, and it is directly related to the character recognition effect, so character segmentation is a key step in license plate recognition. This scheme chooses a new character segmentation method, that is, contour feature and vertical projection joint segmentation method, the basic principle is as follows: the demarcation between the characters is often less than the projection, the projection at the dividing point is close to 0 or zero, so we can use this feature to get the first set of split points, Then, from left to right, from top to bottom, from bottom to top, each column is scanned to get the upper and lower contours of the character, the split point is shown as the trough on the upper contour curve, the crest is shown on the bottom contour curve, and the real segmentation point is determined according to the relative position of the three sets of points. Compared with the normal vertical projection method, this method can solve the problem of character adhesion caused by light pollution of license plate character image.

3.4 License plate character recognition

In this scheme, the template matching method is used for character recognition, whose basic principle is to scale the two-valued image after the character segmentation to the size of the template in the character database, then match all the templates, and finally select the best match as the result.

Three, the hardware design plan outline

4.1 FPGA design overall scheme and module analysis

Figure 4-1 is the hardware schematic diagram of this project. The video image input video decoding chip captured by CCD camera is converted into a digital signal into the block ram of FPGA by video decoding. The FPGA chip internal circuit completes the image preprocessing, the license plate localization and the license plate character segmentation in sequence three functions, the three parts adopt the pipeline way realization, This will greatly improve the efficiency of image processing, and finally, the segmented license plate characters are passed through the high-speed USB port to the backend for subsequent character recognition. Among them, the video decoding chip and the USB data Transmission control is through the FPGA internal Microblaze soft core to realize.

Figure 4-1 System hardware block diagram

4.1.1 Image Acquisition Module

The System Image Acquisition module includes PAL CCD camera and Philips SAA7113 video decoding chip, through the Microblaze soft core control video decoding chip, the captured analog video signal is converted into digital signal. The SAA7113 integrates decoding, sampling and quantization into one, supports interlaced scanning and multiple data output formats, and incorporates A/D conversion circuit, pre-processing circuit and I²C interface, which can be used to control the internal circuit of the chip by configuring the internal register via the I²c interface.

4.1.2 Rgb2gray Module

Figure 4-2 Rgb2gray hardware block diagram

Image Acquisition module output RGB color image conversion to grayscale image of the hardware block Diagram 4-2, the module is composed of three multipliers and an adder, where the weight of decimals in the FPGA using XQN format fixed-point number.

4.1.3 Image denoising Module

In this scheme, the Image denoising module is realized by the median filter of the image, and figure 4-4 is its hardware block diagram. Figure 4-3 is a 3x3 filter template, the different templates can be different filtering effect. 4-4, using shift register to the image data cache output, the image data and template parallel one-dimensional convolution can be obtained three convolution results, the input adder can be filtered results, where buffer size is the number of columns of the image.

[ 1] [2]

Design and realization of vehicle license plate location recognition system