Design of cloud-based log analysis system in security management platform

Design of cloud-based log analysis system in security management platform

Chen Jie Yu Yonggang Liu Mingheng Pansheng and Tsui Hark paid

The Security Management platform (SMP) is a technical support platform to realize the normal operation of security management, and it needs to deal with the mass log information from the security equipment in real time application. In order to solve the problem of inefficient mass log query in SMP, an SMP log storage analysis system based on cloud computing is designed. Based on the Hive Task transformation model, the paper uses the Distributed file system and MapReduce parallel programming model of Hadoop architecture to realize the efficient storage and query of the massive SMP log. The experimental results show that compared with the Multiple Table association query method based on relational data, the system can improve the average query efficiency of SMP log by about 90%, and accelerate the overall response speed of SMP centralized control.

Design of cloud-based log analysis system in security management platform