On the construction of public security large data

In recent years, with the development of information technology, the existing public Security Police data center is difficult to adapt to the data management and analysis under the massive data scene, which directly affects the pre-judgment and the important decision of the public security situation. Therefore, it is urgent for the Public Security Department to construct a new large data system which can match the public security business scene at the present stage. The newly-built public security large data system will become a large data management and analysis platform for each duties. Through the collection, collation, archiving, analysis and prediction of massive data, the author excavates the intrinsic and inevitable causal relationship behind all kinds of data from the complicated data, and finds the Secret law, which promotes the data from quantitative to qualitative change, To achieve the deep application of massive data, integrated applications and high-end applications. Through the construction of large data, the new system can provide the duties with centralized resources, centralized management, centralized monitoring and implementation of a unified large data application environment, and guarantee the support, service and safeguard function of the actual combat application of the whole police in a long period of the future. So, what kind of large data system does the public security need? 1. PB-level data storage management: Information construction in advance, the rapid growth of data scale, in order to meet large-scale data storage and analysis, large data storage systems should support a single system to scale up to 10PB or more to meet future data explosion storage needs 2. A variety of data types and protocol support: Police data in various forms, including documents, pictures, videos, grids, vectors, etc., so the system needs to be able to support structured, semi-structured, unstructured data types, provide NFS/CIFS/JDBC/ODBC and other interfaces, For business to access and operate on a variety of data; 3. High-quality data integration: Good data quality is the basic condition for effective application of data analysis and mining, in the face of the complex and numerous interactive system of the police industry, it is necessary to extract, transform and load these multi-source heterogeneous data, to realize the integration and weight of data, and to provide high quality data, which is based on the correlation and modeling, Provide available data for combat operations 4. Efficient data analysis capabilities: Bai Records retrieval, thousands of table collisions, hundreds of-hour video analysis, a large number of mobile internet and social media data processing applications, all of the data analysis of large data systems, the ability to put forward higher requirements; 5. Manageable and open: manageable, open, standardized large data technology architecture, not only for the public security to bring more cost-effective, more excellent scalability, more for the police construction in the large data platform to carry out new exploration, new applications to lift the worries; 6. Safe and reliable, autonomous controllable: Many of the data in the public security system is related to national security and people's life and property security, therefore, the system requires a very high reliability, at the same time, in order to further enhance data security, to avoid data leakage, it is best to choose with fully independent intellectual property rights of domestic The essence of large data is the management and exploitation of data, and the currentThe public security work with the core of information resources development has wide commonality, how to promote the development and reform of public security work with the help of large data technology? Technology selection is very important. In the various enterprises and organizations to help push, the large data field related technologies present a flourishing situation, covering data, storage, computing, mining, resource scheduling, the following is based on the core of the calculation layer and storage layer two dimensions of the relevant technical route and development trends. Data processing: In short, no matter what application, when the data volume is very large can not solve the computation problem on a server, at this time the distributed computing advantage is reflected, and the important innovation of Hadoop mapreduce is when processing a large dataset will decompose its task and handle in the running of multiple nodes , the batch processing framework is often used for off-line complex unstructured data processing, such as ETL, data mining and other scenarios; unlike Hadoop, which uses hard disks to store data, Spark is a memory-based iterative computing framework that is suitable for applications that require multiple operations on a particular dataset; Storm is a kind of real-time data type of flow analysis framework, applied in the low latency scene, the implementation of large-scale events real-time analysis, processing and decision-making. In addition, in order to cope with the growing volume of structured data storage and rapid processing and flexible business modeling requirements, the database system will be introduced into the distributed architecture, MPP processing technology. Data storage: This refers to the ability of mapreduce to distribute tasks to handle large data on multiple servers. For distributed computing, each server must have access to the data, this is the role played by HDFs, HDFs has a high fault tolerance, high throughput characteristics, suitable for large data set applications. At the same time, there are many other types of file systems in the industry that not only address the challenges of traditional storage architectures, but also improve storage utilization and data read and write performance, replacing HDFs as the underlying file system/data store for the Hadoop architecture. Different technical ideas are biased, because of a wide range of public security services, large data application scenarios, in addition to the establishment of various types of basic large data resource database, but also to predict the police in advance, the real Time information analysis and event analysis, and visual query statistics, It is suggested that public security users should construct the high level application based on intelligent Fusion, and introduce advanced technology in large data field to push public security work into the stage of large data development. Huawei's large data solution services the public security Huawei brings together large data experts at home and abroad, build a world-class large data team, a comprehensive coverage of key technologies in the field of large data, the introduction of intelligent integration of large data solutions, has been in the global telecommunications market, the domestic financial industry and the Government-related departments have a lot of practice and success stories, such as China Merchants Bank , Construction Bank, ICBC, Shanghai Unicom and Jiangsu Mobile and so on, and with a number of public security organs and public security industry application manufacturers to work closely with the public security users to create services for the actual application of large dataProgramme。 498) this.width=498 ' OnMouseWheel = ' javascript:return big (This) ' src= ' http://s6.51cto.com/wyfs02/M02/53/EB/ Wkiom1rz8k7brnr2aabv81s_sne321.jpg "width=" 552 "border=" 0 "height=" "alt=" Huawei large data Solutions Service public security "/> Huawei's large data solution incorporates the infrastructure level of the Oceanstor 9000 large data storage, RH2288 common X86 Server, and the data management analysis level of fusioninsight Enterprise Hadoop, the program in the Public security field of value embodiment includes: To establish a public security information resource base which integrates massive heterogeneous multi-source data, to enlarge the data control surface of large scale and more kinds of data, to widen the information controlling face of public security, to achieve faster and more accurate intelligent Search, association query and collision alignment, and to improve the efficiency of analysis and evaluation Dealing with more complex analytical and predictive models, to enhance the detection ability of the case, to maintain the openness of the large data platform 100%, to support the integration and to carry the public security service more robustly; Huawei's strategic investment, strong research and development team has long been engaged in large data field of research, can help police large data practice and exploration.