In-depth Big Data security Analytics (1): Why do I need big data security analytics?

"Foreword" After our unremitting efforts, at the end of 2014 we finally released the Big Data Security analytics platform (Platform, BDSAP). So, what is big Data security analytics? Why do you need big Data security analytics? When do I need it? Who needs it? What is the application scenario? Solve what problem? What is the value and meaning? How will big Data security analytics reshape cybersecurity technologies? How to build a big data security analysis platform at present? Starting this issue, I'm going to start a new series of articles-in-depth big data security analytics. If I had started at the end of 2011, when the network security encounter Big Data Analysis series was the beginning of a systematic study of the industry's Big data security analysis, then this new series of articles is what we have learned after years of research. Let's go into big data security analytics and explore how he's going to change our cyber security landscape.

Why do you need big Data security analytics?

Last Updated @ 2015-01-06 by Yepong

"Key words" big data security analytics, big Data

"Summary" Big data changes all aspects of us, and security analysis is no exception. The security element information presents the big data characteristic, but the traditional security analysis method faces the big challenge, the information and the network security needs to base on the big data security analysis.

There is no doubt that we have entered the Big data ( Big Data) era. Human productive life produces a lot of data every day, and it produces more and more rapidly. According to IDC and EMC 's joint survey, the total global data will reach 40ZB by 2020 year. Gartner has ranked big data as the top trend in the development of future information architectures . Gartner predicts a cumulative production of 2320 billion in the year to $.

Big data has long existed, but there has been insufficient basic implementation and technology to make a valuable dig of the data. As the cost of storage continues to fall, and as analytics continues to evolve, especially cloud computing, many companies have discovered the great value of big data: They reveal new trends that other tools cannot see, including demand, supply, and customer habits. For example, banks can have a more in-depth understanding of their customers, to provide more personalized customized services, banks and insurance companies can find fraud and deception, retail companies more accurately detect changes in customer demand, for different segments of the customer group to provide more targeted choices; pharmaceutical companies can use this as a basis for the development of new drugs, Track drug efficacy in detail and monitor potential side effects; security companies can identify more covert attacks, intrusions, and violations.

Current network and information security areas, are facing a variety of challenges. On the one hand, the enterprise and organization security architecture is increasingly complex, various types of security data more and more, the traditional analysis ability is obviously inadequate; on the other hand, the emergence of new threats, internal control and compliance in-depth, the traditional analysis method has many shortcomings, more and more need to analyze more security information, and to make decisions and responses more quickly. Information security also faces the challenge of big data.

1      Big data for secure data

The big data of safety data is mainly embodied in the following three aspects:

1) More and more data: The network has moved from gigabit to million, network security equipment to analyze the amount of data packets increased sharply. At the same time, with the advent of NGFW, the security gateway to the Application layer protocol analysis, the analysis of the data volume is greatly increased. At the same time, with the deepening of security defense, the content of security monitoring is continuously refined, in addition to the traditional attack monitoring, there are compliance monitoring, application monitoring, user behavior monitoring, performance testing, transaction monitoring, etc., which means to monitor and analyze more data than ever before. In addition, with the emergence of new types of threats such as APT, the whole-packet capture technology is gradually applied, and the problem of mass data processing becomes more and more obvious.

2) faster and faster : Packet processing and forwarding speeds are faster for network devices, and for Anguiping, event analytics platforms, the event sending rate (EPS,event per second The number of events per second) is getting faster.

3) More and more varieties: In addition to data packets, logs, asset data, security element information is also included in the vulnerability information, configuration information, identity and access information, user behavior information, application information, business information, external intelligence information and so on.

The big data of security data, naturally lead people to think about how to apply big Data technology to security domain.

2      traditional security analysis faces a challenge

The rapid expansion of the number, speed and type of security data not only brings the problem of the fusion, storage and management of the massive heterogeneous data, but also shakes the traditional method of security analysis.

Most of the current security analysis tools and methods are designed for small data volumes and are unsustainable in the face of large data volumes. New attacks have emerged, with more data to be detected and the existing analysis technology overwhelmed. How can we perceive the network security posture more quickly in the face of the security element information of the day quantity?

traditional analysis methods mostly adopt rules and features based analysis engine, must have the Rule Library and feature library to work, and rules and features can only describe the known attacks and threats, do not recognize unknown attacks, or is not yet described as a regular attack and threat. In the face of unknown attacks and complex attacks such as APT , need more effective analysis methods and techniques! How do you know the unknown?

the traditional centralized security analysis platform (e.g. SIEM, security management platform, etc.) also encountered many bottlenecks, mainly in the following areas:

• Acquisition and storage of high-speed, massive security data becomes difficult

• Storage and management of heterogeneous data becomes difficult

• Small threat data source, resulting in limited system judgment

• The ability to detect historical data is weak

• The investigation of security incidents is too inefficient

• Security systems are independent and work together without effective means

• Fewer methods to analyze

• More difficult predictions for trending things, less ability for early warning

• The system has limited interaction ability, the data display effect needs to be improved

Since the birth and establishment of intrusion detection technology in the last century, security analysis has developed for a long time. At present, there are two basic development trends in information and network security analysis: Situational awareness security analysis and intelligent security analysis.

"Future information security will be situational-aware and adaptive," Gartner said in a report in the year. Situational awareness is the ability to improve security decisions by taking advantage of the comprehensive analysis of more relevant element information, including asset perception, location awareness, topology awareness, application awareness, identity awareness, content awareness, and so on. Situational perception greatly expands the depth of security analysis, incorporates more security element information, pulls up the space and time range of analysis, and inevitably challenges the traditional method of security analysis.

also in the year,another Gartner report said, "Be prepared for the rise of enterprise Security Intelligence". In this report,Gartner puts forward the concept of security intelligence, emphasizing the need to integrate and correlate disparate security information from the past, and to integrate independent analytical methods and tools to create interactions that enable intelligent security analysis and decision making. The integration of information, the integration of technology will inevitably lead to the rapid growth of security elements information, intelligent analysis must require the machine learning, data mining and other technologies to apply to security analysis, and to make security decisions faster and better.

3      Information and network security need big Data security analysis

The big data of security data, as well as the challenge and development trend of traditional security analysis, all point to the same technology-big data analysis. As Gartner clearly stated in the year, "information security is becoming a big data analysis problem".

as a result, the industry has a technology to apply big data analysis technology to information security--Big Data security analysis ( Big Data security analysis, referred to as BDSA), is also known as a security-oriented analytics for large datasets.

With the help of big data security analysis technology, we can better solve the problem of collecting and storing the information of the day quantity security elements, and with the help of the machine learning and data digging algorithm based on big data analysis technology, we can more intelligently understand the situation of information and network security, and respond to the new complex threats and unknown changeable risks more actively and flexibly.

"To be Continued" next article-"What is big Data security analysis?" 》

This article is from the "focus on the Pipe platform" blog, please be sure to keep this source http://yepeng.blog.51cto.com/3101105/1599937

In-depth Big Data security Analytics (1): Why do I need big data security analytics?