Video-based vehicle identification technology

I. Overview of Intelligent Transportation Technology
The Intelligent Transportation System (Intelligence Transport System, its) is a frontier research topic in the World transportation field, which integrates the theories of electronic information technology, communication technology, automatic control theory, computer technology and traditional transportation engineering theory, And it is applied in the modern transportation management system, so as to realize the transportation service and management intelligence.
Traffic monitoring system is an important link in intelligent transportation system, which is mainly responsible for collecting various parameters about traffic flow, such as vehicle volume, speed, model, queue time and length. At present, there are many methods of road parameter detection, such as ultrasonic detection, infrared detection, sensor loop detection and video-based detection, etc. The ultrasonic detection accuracy is not high, is susceptible to the vehicle occlusion and the pedestrian influence, the detection distance is short, the infrared detection is affected by the vehicle own heat source, and the anti-noise ability is not strong, therefore the detection precision is not high, although the ground sense coil detection precision is relatively high, but the request is set in the road structure, and has the damage Construction and installation of relative inconvenience, the service life is relatively short, easy to damage and other shortcomings.
In recent years, video detection based on the continuous development of computer technology, image processing, artificial intelligence and pattern recognition, in traffic flow detection has been more and more widely used, compared with other traffic flow detection technology, it has the following advantages:
1. Video detection can detect large traffic scene area;
2. Compared with other detection methods, less investment and low cost; Video sensors, such as cameras, are easy to install and debug, and do not cause damage to pavement facilities;
3. Use video detection technology! More traffic flow parameters can be collected.

Two Introduction and description of related technologies
Preprocessing of images--background extraction
The background difference method is used for vehicle detection, in general, the whole video image captured by video card is not processed. Here, purposefully depending on the lane, several areas of interest (also known as virtual coils) are selected in the entire image, and these areas are processed in real time to detect vehicles and obtain relevant traffic flow parameters.
Extract several frames continuous video images, and the virtual coil in the pixel gray value in the sequence of frames stored in the array, the virtual coil of each pixel point to the gray value of the histogram, select the most frequently occurring gray value as the background image of the gray value of the current pixel.
In general, it is reasonable to choose the gray value that most often appears in each pixel as the gray value of the corresponding pixel in the background image, and if the vehicle is more dense, the number of frames can be increased appropriately to obtain better results.
Background extraction requires an initialization process, in which the video image is not detected during the background extraction process.
Background update uses a background extraction and background updates every once in a while: The program set a timer, every time, the program began a new round of background extraction, the background of the extraction process, the program for vehicle detection.
However, the background used at this time is the background of the last extraction, when the current background is extracted, with the current background covering the previous background, the image of the next frame of the vehicle detection using the updated background, that is, the current background, to detect.
This method can effectively suppress the slow change of light and natural condition, and can improve the effect of the background difference algorithm to detect the vehicle. When the vehicle passes through the virtual coil, the sum of the absolute difference between the current gray value of all pixels in the virtual line and the corresponding pixel of the background will have a change, and the sum of the absolute difference between the gray value of the current image of all pixels in the circle and the gray value of the background image is variable.
In the case of the virtual coil without vehicle passing, the current image information in the virtual line remains relatively constant, and the gray value of the pixel points in the line circle changes very little. Then, when a vehicle starts to enter the area of the virtual coil, the value increases gradually due to the huge difference between the background and the gray value of the vehicle's pixels, and the vehicle's exit from the virtual coil area will gradually decrease. When the vehicle is completely away, it becomes very small.

This approach may have drawbacks and weaknesses:
The difference between the frame is commonly used for continuous three-frame for 22 differential, although this method has a strong self-adaptability, but for the continuous frame to make the choice of the timing of high demand, and depends on the movement speed of moving objects, if the movement faster, and the selected time interval is too long, it will cause no coverage between the two frames, So that moving objects cannot be segmented, and if the movement is too slow and the chosen time is too small, it can cause excessive overlap, and the worst case is that the object is almost completely overlapped and the object is not detected at all.

Image preprocessing-foreground extraction:
The basic idea of foreground extraction is to subtract the current frame from the current background frame and get the foreground point of motion after the threshold. Before the operation, the current frame is smoothed to reduce the effect of noise. Here you use the 3*3-sized Gaussian template. Because the smoothing also causes the loss of image edge information, we also need to enhance the edge of the filtered image at this time. Considering the non-sensitivity of the G-Channel to noise, we only use the Sobel operator for edge enhancement of the current and background frames on the G-Channel. In order to adapt to changes in the background, select the door limit:

Among them,. C=r, G, B
Because the histogram of the background image shows a single peak characteristic in the R, G and B channels, the threshold is reasonable. After that, the difference between the current frame and the background frame at any point (x, y) is defined


When the DBT (x, y) ≥tl, the point is considered to be the point of movement of the foreground, otherwise it is considered a background point and will be updated into the background. That is pfore={(x, y) | DBt (x, y) ≥tl}.
After the above operation, including the shadow of the motion of the foreground has been basically extracted, but due to the impact of noise, may make the area is incomplete, and even a region is broken into two or more than two areas, it is also necessary to detect the point set of a number of morphological operations to ensure the integrity of the motion foreground.

Preprocessing of image--Filter processing
The original image obtained by the imaging system can not be used directly because of various conditions and random interference, such as the lens of the visual imaging system, the surface is blurred, and so on, the original image must be preprocessed. For the visual system, the image preprocessing method does not need to take into account the reason of the quality of the image, the characteristics of interest in the image can be selectively highlighted, attenuation of its unwanted features.
The filtering processing of computer image mainly adopts two kinds of methods:
A class of methods are processed in the spatial domain: that is, the image is processed in the image space, and the other method is to change the spatial image, such as Fourier transform, so that it can be processed in the frequency domain, then change back to the spatial domain of the image and form the processed image.
At present, the frequency domain processing method has means transform and inverse transform, various wavelet transform and inverse wavelet transform. These methods use a large amount of computer memory and computational time, and are not suitable for real-time systems such as smart vehicles.
Therefore, using median filtering method in spatial domain, this method is a kind of local average smoothing technique, which is good for suppressing impulse interference and salt and pepper noise. Under certain conditions, the image details blurred by linear filter such as the minimum mean square filter mean filter can be overcome, and the edge of the image is effectively protected. Because the statistical characteristics of the image are not needed in the actual operation, it is very convenient for the preprocessing of the image.

Preprocessing of images--edge enhancement
According to the visual theory, the recognition of an object starts from its edge, and the edges of the different parts of an image are often the most important features of pattern recognition. The Edge is a set of pixels whose surrounding pixels have a step change or a roof change, which is widely present between objects and objects, between objects and backgrounds, between primitives and Primitives,
In the image captured by the machine vision system, the edge information of the lane is buried between the background.
The purpose of edge enhancement is to highlight the edge information of the road to facilitate road boundary recognition. In addition, the Edge enhancement algorithm can help to overcome the influence of road illumination inequality.
The common edge enhancement operators are Robert Operator, Sobel operator, Krisch operator, Prewitt operator, Laplace operator, etc. Here, the edge detection uses the Sobel operator, which is actually a first order difference operator, which can effectively eliminate most of the useless information in the road image.
The discrete-BCDF algorithm is defined as the following type $

The Sobel operator has a strong ability to suppress noise, in fact, the essence of Sobel operator is the gray-scale difference characteristic of pixels that reflect the neighboring or distance. For the road, the normal surface due to physical properties close, and light is generally uniform. In this case, the gray value of the neighboring pixel is not quite the same, after the Sobel operator, the proximity is converted to a certain value, which is usually close to the value. The boundary of the road and the other parts of the road have a certain gray-scale difference, the Sobel operator can highlight the difference of the pixel value at the boundary, and the value is larger relative to other pavement parts. At the same time, the gradient direction information of the edge can be obtained, and the algorithm is simple and easy to realize.
After Sobel operator operation, the boundary is highlighted from the whole image.

Preprocessing of image--two value
The key to binary image processing is the reasonable selection of threshold value, the threshold setting too small prone to noise "threshold set over the General Assembly to reduce the resolution, so that non-noise signal is treated as noise and filter out, considering the general light is more uniform or in the light does not change the conditions of the test, Therefore, the whole optimal threshold method is chosen for Image binary processing.
The overall optimal threshold principle is to statistic the distribution characteristic of each image gray scale, using the category variance as the criterion, and choosing the maximum variance between the classes as the selected threshold value.
The two-valued image using the optimal threshold algorithm further reduces noise and provides cleaner image data for subsequent Hough transformations.

Image preprocessing-camera calibration and Setup
The video camera can be mounted on a overpass, on a high-rise building, or on a pole that is high enough. Figure 1 depicts the configuration of the video camera, with some configuration parameters including tilt angle, rotation angle, amplitude angle and camera height H. We assume that the rotation angle equals 0, while the parameters, h, and the camera's focal length F are known beforehand. The camera's perspective will be determined at the time of calibration.
Figure 1. Configuration of the camera
Mapping relationship between two-dimensional space and three-dimensional spatial image
The three-dimensional image is the mapping of three-dimensional space coordinates on the two-dimensional camera plane, and the actual geometric relation is calculated after the advanced mapping of the acquired image. Usually the location of the camera and the road coordinate system.

Detection and tracking-vehicle detection

How to detect the trajectory of a moving target
The background method can detect moving and stationary vehicles, and the moving regions can be divided into moving and stationary targets with differential backgrounds and difference frames. The prediction information of filtering is used as the information of cluster initialization, which greatly reduces the number of iterations and enhances the real-time nature. In the trajectory correspondence between the frames, according to the lag of the target movement in the unit time, the moving target has little displacement in the unit time, the velocity is approximate and so on, the cluster result of the adjacent frame is matched to obtain the trajectory of the moving object.
How to handle noise
In general, if the detected trace is a moving target point trace, the point trace of the target can be detected in the adjacent frame cycle, and the trajectory is established according to the trajectory initiation criterion, which solves the problem of the initialization of the tracking. If a certain trace is generated by random noise, it is impossible to make a continuous occurrence in a certain place in a few adjacent frame periods, that is, the noise trace can not establish the trajectory, so the problem of false alarm in the tracking process is solved effectively.
Advantages of adaptive backgrounds
The adaptive background model is used instead of the fixed background to enhance the robustness, but the establishment of adaptive background image due to the fusion of spatio-temporal information increases the computational capacity, and has some influence on the real-time, in the event of abrupt changes in the environment, such as the abrupt change of the weather, adaptive background image initialization takes some time.
Limitations
The external environment changes have brought great challenges to the vehicle's motion segmentation and detection, and the establishment of vehicle segmentation and robust detection adapting to any changes will require great effort. Because the road is complex and changeable, many kinds of situations can not be predicted beforehand, so it is difficult to detect the vehicle by simply relying on the difference between the frame and the difference background for the model detection. The information fusion method of the image based on multi-vision system at the same time from different angles will be a method, but it must solve the problem that the information fusion affects the real-time.

Detection and tracking-vehicle tracking process
(1) Use each area of the first frame as a different target and start the target chain for each target area.
(2) According to the ruling criteria, if the region in a target chain finds a matching region in the current frame, the region feature in the target chain is updated with the found matching region feature.
(3) If there is a large difference in the area between the current frame area and the target chain in the position of the centroid prediction value, the merging or splitting phenomenon can be considered. Surrounds a rectangle in the area of the target chain, and finds the rectangle in this frame that covers several areas, and if there is more than one area, it is considered a split phenomenon. Start a new target chain for new areas where the phenomenon of fragmentation occurs. Similarly, for the bounding rectangle of this frame area, finding the rectangle covers the region of several target chains, if more than one, it is considered that a merge phenomenon is occurring, the new target chain is initiated with the merge region, and the target chain of those merged regions is terminated.
(4) for the region in the target chain, if there is no match in this frame, it is considered that the vanishing phenomenon occurs. The target chain does not terminate immediately, and the target chain is terminated only after several frames have not yet been found.
(5) Find out if there is a newly entered area in this frame, and if so, start a new target chain. By using this method, we can quickly track the target in the image sequence and get the average speed of the vehicle in the monitoring range. When counting, only the target appears in consecutive frames to be considered a true target area, only the target does not appear in consecutive frames to be considered to disappear, so can eliminate those caused by the temporary disappearance of the count error.


Detection and tracking-measurement of parameters such as vehicle flow and speed:
One according to the vehicle's driving path:
Multi-person decision making technology (MPDM) false with M independent attribute characteristics to describe the vehicle, accordingly, in the multi-person decision-making process also has m decision maker. We are also building separate from video images and detected in a vehicle. Then each decision maker I, based on an attribute trait, determines the correspondence between the image and the vehicle in.
These corresponding vehicles will be further sorted according to the difference between the respective vehicles, and this n is expressed as the result of the sequencing. Next, according to the following order S, we define a value D for each pair of vehicles to represent their similarity level:
(13)
This indicates the height, n is the number of vehicle pairs.
In practice, we select the corresponding vehicle pair according to order 1, D (n,1), which must be greater than a certain threshold value, so as to control the occurrence of false matches caused by the vehicle moving out of the monitoring range. Even if the vehicles are shielded from each other, they can then be tracked by a multi-person decision-making process. In this case, we want to further calculate the value of D (n,2), and the larger d (n,2) means that the two shielded vehicles are separated from each other.
From the results of the tracking, the number of vehicles and speed can be determined. The number of vehicles can be obtained by means of a number of vehicle routes, while the moving distance of the vehicle on the road plane is divided by the tracking cycle to get the vehicle's speed. Here, the moving distance of the vehicle can be calculated by the relationship between the length of the driving route and the plane of the road and the plane of the image.

Second, according to the number of vehicles crossing a certain virtual line to judge over the line
In addition to the specific tracking of a vehicle's specific operating trajectory, if only to get a certain section of the road unit time flow, and do not need to locate a car, you can go to a line in a certain direction, through the detection of the image of the number of vehicles across the line, you can also get traffic. For the two-way road, uplink and downlink each take a virtual line, the location of the line can not be too far, or the vehicle overlap is very strong, not too close, due to the system processing capacity constraints, the number of frames per second should not be too much, and the near vehicle displacement is more obvious, easy to cause missing.


Shadow suppression and occlusion--suppression of vehicle shadows
The shadow caused by the moving object will move with the motion of the object, so the object separated by the HSV method will be affected by the shadow, so that the object will be larger than the actual object, and even cause two separated objects to join together, affecting the accuracy of the counting.
The key algorithm of shadow detection is the shadow detection algorithm based on HSV space, because HSV space is closer to human visual response, and more accurately reflects some grayscale information and color information, especially for the very bright and very dark objects in the image industry can reflect the corresponding information, after processing, You can divide the image pixel into two parts: the background and the moving object. Here the moving object includes the shadow, which usually compares the pixel value of the shadow with the pixel value of the background, and if it contains the corresponding color and grayscale values within a certain threshold range, then it is considered a shadow.

Shadow suppression and occlusion--occlusion of vehicles

There are several ways to solve this problem due to the location of the camera and the driving of the vehicle, which sometimes results in the occlusion of the vehicle:
(1) According to the vehicle driving path, to determine whether the car at this moment of occlusion is separated from the previous moment;
(2) According to the difference between the blocked vehicles, the method of using the motion area analysis;
We can calculate the difference between the frame-the background and the frame and the frame, and come to the displacement of the moving object in a particular area of interest, due to the vehicle's trajectory and the continuity of the speed, if it is found that the displacement of this area is not always consistent, when it exceeds a certain threshold, we can determine the occlusion occurred.
In addition, I have also envisaged several other methods: according to the width of the vehicle occupied lane to determine whether there are two vehicles to create occlusion, according to the vehicle's own characteristics, such as in the HSV space, we detect the vehicle mainly relies on its V-gray characteristics, in the inability to determine whether there is no occlusion, we can according to its H and s characteristics Judging the body color is consistent, or to determine the color saturation of the difference, if a car in a larger area of the color is basically consistent and so on (this will be affected by the car's own painting, but with the human eye recognition process in line with).
In the detection area, the speed of the two moving vehicles will be different, which determines the difference between the different frames, the displacement of the region, by setting the threshold, we can be blocked objects identified.
In addition, according to the shape, contour and other two different motion area features "Mark" two different vehicles, and then through the morphology of the expansion of technology, expand each car's marking area, creating overlapping parts, this part is called "cutting zone", so that can be divided into two independent vehicles.
Pedestrian interference
According to the length and width of the vehicle, adjust the position of the camera, convert the vehicle data from the two-dimensional image space into three-dimensional real-space values, so that by properly selecting a certain threshold, you can be detected in the basis of the vehicle, the pedestrian will be separated. When counting, it can eliminate the pedestrian influence.
Detect Queue Length
According to the mapping relationship, we can calculate the actual vehicle length according to the queue length of the image on the bus.
Here the algorithm for detecting queue length consists of two parts:
One is the total length of the vehicle queue----in the visible range of all vehicle lengths, including stopping and moving vehicles
The edge image of the vehicle on the road is obtained by using the Sobel edge operator, then the resulting edge is mapped to the middle axis of the road in the image, and the number of the highlights (the number of pixels after the binary value equals 1) is obtained, and then the length of the vehicle in the two-dimensional image is calculated and converted into the actual queue length.
But this method gets the whole queue length of the motorcade, unable to distinguish between the length of the moving vehicle and the stationary queue vehicle.
The second is to get the queuing length of the parked vehicle.
In the intersection monitoring system, some conditions (such as the change of the control signal) need to get the length of the stopped vehicle, using the edge information to get the overall queue length based on:
Actually in the number of statistical highlights, is not a progressive scan, but one scan n rows, n is actually slightly smaller than the length of a car mapped to a two-dimensional image of the resulting pixels.
Assuming that the driver will automatically stop, that is to say, the use of all available lanes (which is in line with the actual situation).
In statistics, if the statistical area (to achieve the length of the set car) is less than half the number of highlights, it is assumed that this area may be stopped the vehicle queue end, and then start the corner detection program, if the corner of the area is less than a certain threshold (can be selected according to the actual needs, usually choose a car average number of corners) It is assumed that the current scan behavior of the vehicle queue endpoint, if the corresponding threshold is greater than the scan continues.



DSP Technology
Digital Signal processing, referred to as DSP, is a new subject that involves many disciplines and is widely used in many fields. Since the 1960s, with the rapid development of computer and information technology, digital signal processing technology came into being and developed rapidly. Over the past more than 20 years, digital signal processing has been widely used in communications and other fields. Digital signal processing is the use of computer or special processing equipment, digital form of signal acquisition, transformation, filtering, valuation, enhancement, compression, identification and other processing, in order to achieve the needs of people to meet the signal form.
DSP chip, also known as digital signal processor, is a kind of microprocessor which is especially suitable for digital signal processing operation, and its main application is to realize various digital signal processing algorithms in real-time and fast. According to the requirements of digital signal processing, the DSP chip generally has the following main features:
(1) One multiplication and one addition can be done within a single instruction cycle;
(2) The program and the data space are separated, can access the instruction and the data simultaneously;
(3) on-chip with fast RAM, usually can be accessed through separate data bus in two blocks simultaneously;
(4) Hardware support with low overhead or no overhead cycle and jump;
(5) Fast interrupt processing and hardware I/O support;
(6) Having multiple hardware address generator in single cycle operation;
(7) Multiple operations can be performed in parallel;