Application of video Big Data technology in Smart city

The amount of information in modern society is growing at a rapid rate, and there is a lot of data accumulating in it. It is expected that by 2025, more than 1/3 of the data generated each year will reside on the cloud platform or be processed with the cloud platform. We need to analyze and process this data to get more valuable information. In the future of "smart cities", there will be more and more structural and unstructured data. So how do we store and manage this data efficiently, and how do we analyze that data? The answer is that we need a strong big data-processing system for support.

As one of the hottest words in the world, big data has more mature applications in all fields. In the field of video surveillance, the era of big data is creeping in. In the urban security, traffic management will deploy a large number of video surveillance equipment, these video surveillance equipment would produce a large number of video and related data, such as traffic card port data reached 1 billion or even greater levels, the amount of face database data reached thousands or even greater. For such a large amount of data, the current system will gradually reveal that the data retrieval speed is more and more slow, data statistics, analysis efficiency is increasingly low, these problems need a mature technology to solve.

With the rapid development of urban safety and intelligent transportation in "smart city", the amount of non-structured data in cities is getting bigger and larger. Video Big Data technology focuses on helping customers improve the efficiency and accuracy of their decision making by rapidly discovering high-value information from an ever-increasing amount of unstructured video data. Therefore, the processing of video big data has become the focus of customer attention, video big Data processing technology has become the embodiment of the ability of manufacturers. Aiming at the demand of fast retrieval and statistic analysis of large amount of video data in the construction of "smart city", the leading enterprises in the video field such as Hai Kang Wei Vision have already had the mature video big Data solution, and have already applied many "smart city" construction.

First, video Big data platform technology

The Big data platform serves as a platform for the application platform to provide high-efficiency processing capability for large data volumes, especially for structured or semi-structured data. With big data platforms, applications can efficiently store, retrieve (second-level), analyze, and count these data, effectively improving efficiency and improving user experience.

The video Big data platform is generally built in a distributed cluster, the cluster can load balance the data processing, and the cluster can be easily extended, which can improve the overall performance of the platform by increasing the cluster nodes. Data storage needs to consider the following aspects: first, which data need to be saved to the video big data platform; second, if the original system is reformed, the existing system of the existing data on how to deal with, third, how to ensure the reliability of the data. The video Big Data platform uses distributed computing, combining memory acceleration, load balancing, and local processing to provide efficient data computing power.

Video large data processing system, in response to large-scale video data processing of storage problems, the use of distributed storage, improve the reading and writing speed, and expand the storage capacity, in response to the large-scale video data processing of computing problems, the use of distributed computing systems, improve the data analysis and mining capabilities. The overall architecture of video data processing system is shown in three levels, including resource layer, platform layer and application layer.

Overall architecture of video Big Data processing system

Resource layer: Includes it basic resources, data resources, video resources and so on. The ability to generate, store, and process massive amounts of data, such as over-vehicle data, face data, case event data, etc.

Platform layer: The video Big data platform, including data storage, data processing, migration, cluster management and other functions, while providing interfaces for upper-level applications. The video Big data platform is similar to the database, but it is much more powerful than the database processing ability, can deal with the massive data.

Application layer: Based on the video big data platform to provide efficient data processing services, industry application platform (public security, transportation, justice, energy, education, etc.) can provide users with high-volume data storage, retrieval, analysis and statistics and other functions.

Second, the application of video big Data technology in intelligent traffic

In the road traffic application of intelligent traffic, the data of the bayonet car is exploding. For a city, each bayonet passes through thousands of cars per day, assuming that a city has 1000 cards, each with an average of 10,000 records per day, and data to be processed within a year, the total number of car records is 3.65 billion. When the data size of the card is 1 billion or even larger, the current system is running more and more slowly, the time of vehicle information retrieval, research and analysis is more and more long, the user experience is getting worse. The use of traditional relational database solutions will greatly reduce the efficiency, or in the case of efficiency, will make the software, hardware input costs greatly increased, such as the adoption of minicomputer. For such scenarios, big data technology can be used, the data volume of the vehicle is particularly large data from the big video platform to deal with, video Big data platform can provide enough memory and local storage, at the same time, the use of distributed computing, each node parallel computing, greatly improve the ability to data processing. The video Big data platform has the characteristics of reliable system, data security, high efficiency of data processing, low input cost and strong expansion capability.

Traffic bayonet Big Data application design as shown:

Traffic Bayonet Big Data application

The difference between the application of big data in traffic bayonet is the design of the rear terminal system. In the traditional scheme, there is only one intelligent transportation platform, and in big Data solution, there is a big video data platform under the Intelligent Transportation platform, and the massive vehicle data is stored in the video Big Data platform, and the Big Data platform provides the high-level data processing service for the upper platform. Take 1 billion data for example, in the search, research and statistical analysis and other applications of the traditional program retrieval time is minute, using big data technology retrieval time can reach the second level. Based on the video big data platform, the Intelligent transportation platform realizes efficient retrieval, analysis, statistics and other functions, which can provide better service to the users. The following are the main features that can be implemented today:

1. Quick Search

Able to quickly find out the vehicle data record according to the precise condition/fuzzy condition. The big data solution can quickly filter out the target vehicle record from billions of data, and improve the efficiency of traffic Management department's duty.

2. Analysis

The analysis includes the application of frequency evaluation, vehicle analysis at a specific time period, the first entry into the city, the driving track, and the vehicle. Big Data solutions can quickly and timely detect event information, improve the traffic management department's rapid response ability.

3. Statistical analysis

Statistics and analysis include traffic flow statistics, vehicle flow comparison, vehicles illegal statistics, the specific period of traffic flow, driving trajectory statistics and other applications. Big Data solutions can achieve a larger range of traffic data statistics, can be more comprehensive and complete response to a city's traffic conditions, for Traffic management department decision-making to provide data support.

Third, the application of video big Data technology in intelligent security

In the intelligent security case detection and other applications, often need to carry out personnel information query, the personnel database is an important part of the human face library, the human Face library includes the basic information of the person, Avatar image and the human face feature vector based on the Avatar model. One of the most important applications in case detection is human face retrieval, that is, the extraction of the human face from the video image of the feature vector with the face of the data in the database, to query out the basic information of this person. When the number of face banks reached 10 million or even larger, the current face retrieval method will appear bottleneck. Because the whole face library needs to be retrieved in the face retrieval process, the whole face library needs to be imported into memory to improve the retrieval efficiency and the server memory requirements are high, while most face retrieval schemes do not support the cluster architecture, but can only increase the memory of single server to meet the demand, so it is prone to bottleneck. In view of this kind of scene, can adopt the face retrieval big Data solution, the human face library is saved in the video Big Data platform, the video Big data platform uses the cluster structure, and can smoothly expand, can provide the large enough memory, simultaneously, uses the distributed computation, each node parallel computation, greatly enhances the data processing ability.

Face search Big Data application design as shown:

Face retrieval Big Data scheme

The difference of face retrieval Big Data application is that the video Big data platform is added to the rear terminal system. Intelligent Security Application platform to provide users with face retrieval function; The video big data platform stores human face database data, carries on the efficient face analysis, provides the face retrieval service for the application platform. Take 10 million data as an example. If the traditional way database requires the administrator to do the manual sub-database table operation, as the data growth maintenance work will be more and more large, but also difficult to ensure the normal operation of the system, the use of video big data technology can easily achieve the horizontal expansion of the system to ensure efficient operation of the system. Based on the video big data platform, intelligent Security application platform can provide users with fast face retrieval function.

Iv. Summary of the outlook

Video Big Data technology can solve the problem of the current system processing massive video and related data ability, help customers to quickly mining high-value information from the massive video data, help customers improve the efficiency and accuracy of their decision-making. Can be foreseen in the future of the "smart city" construction, the video data volume will be explosive growth, the massive video data processing system requirements will be more and more high, the ability of video mining more and more strong demand, video Big data platform will introduce more and more machine learning, graph algorithm and other cutting-edge technology to improve the intelligent level of the whole system.

Application of video Big Data technology in Smart city