[AI Development] applies deep learning technology to real projects

This paper describes how to apply the deep learning-based target detection algorithm to the specific project development, which embodies the value of deep learning technology in actual production, and is considered as a landing realization of AI algorithm. The algorithms section of this article can be found in the previous blogs:

[AI Development] Python+tensorflow to build its own computer Vision API Service

[AI development] Video Multi-object tracking implementation based on deep learning

[AI development] Video multi-target tracking premium (from autopilot and ' near ' a little)

[AI development]centos7.5 based on keras/tensorflow deep learning environment Building

Background

The field of transportation is an area where deep learning technology can play a powerful role. Road traffic security requirements are relatively high, road monitoring equipment is complete, these devices can produce countless video data every day, and these data is very conducive to deep learning model training. Internet can also be downloaded to a very many traffic data sets, including pedestrians, vehicles, roads on both sides of the infrastructure such as street lights, traffic lights, bus stop and so on.

How to use the video data generated by traffic monitoring equipment to analyze the video automatically? This is obviously an application scenario for deep learning. The following describes how to use deep learning technology to automatically detect vehicle traffic events, such as illegal parking on high-speed roads, retrograde, break-in pedestrians, occupy emergency lanes and other events. Here's the final one:

For a host device, the real-time detection of 8 HD surveillance video, if there are abnormal events, will be issued a real-time warning.

Business process

The overall system composition is very simple and the business process is relatively simple. Here are two main business branches:

Video Stream Branch

As shown, the tracked trajectory is superimposed directly into the video frame and then coded to push it to the Nginx server for playback by the Web player.

Data Flow Branch

As shown, the traffic events for each vehicle are detected and the results are written to the MySQL database, based on the traces obtained. Another service module regularly statistics data for the Web module to display data.

System structure

Server environment

1) CentOS 7.5 (3.10.0 core);

2) Python 3.6 + tensorflow-gpu 1.9 + Keras 2.2.2 + various python libraries;

3) CUDA 9.0 + CUDNN 7.1 + nvidia graphics driver 384.130;

4) Tomcat + Mysql + Java;

5) i7 8 core + Nvidia GTX 1080 graphics card;

The above is the server hardware and software environment.

System composition

is the software system composition:

The top part of this is the location of the core algorithm throughout the system.

Model Training

The most important model training is to collect high-quality, various weather, various video angles, various road conditions of the material. Here is a simple classification of the footage, with a quantity of about 10w+ Zhang.

By scene:

1) Outfield

2) Tunnel

By weather:

1) Sunny Day

2) Cloudy

3) Rainy Days

4) Night (IR)

by camera angle:

1) Straight ahead

2) Right rear

3) Front Side

4) Rear Side

By Target type:

1) Pedestrians

2) Automotive

3) Bus

4) Truck

(High-speed cameras are generally installed too far, other targets are too small, so other targets are not considered)

By road:

1) Smooth

2) congestion

3) Severe congestion

Material labeling is a manual work, find a useful labeling tool can do more with less. I wrote a labeling tool, which is not particularly good, but is suitable for annotations of the material of this project. For related connections See here:-------link--------------

Event Type/function

By tracking the trajectory, we can detect the abnormal events of the road surface separately, including:

1) Pedestrian intrusion events

2) Abnormal parking events

3) retrograde/reversing Events

4) occupy Emergency lane events

5) Truck-occupied super-lane

6) Non-motor vehicle intrusion event

7) Congestion level alarm

8) Model Count

9) Average Speed statistics

This article has no source code, ideas for reference only.