Big Data brings big puzzles

Recently, discussions on large data have been growing, and companies are increasingly concerned about the management of large data. But in fact, many companies do not really understand what is large data, nor deploy the tools to effectively manage them.

"Many people do not understand what is the big data, because there is no clear definition, we all feel very confused." "Loglogic company http://www.aliyun.com/zixun/aggregation/35685.html" > chief Marketing Officer Mandeep Khera said. Recently, loglogic with the IT Security research company Echelon one to complete a large data management survey. The survey found that 49% of companies are somewhat or very concerned about big data management issues, while 38% of companies do not understand what big data is, and another 27% say they have a smattering of big data. In addition, the survey found that 59% of enterprises did not deploy the tools to manage the data in the IT system, but instead switched to stand-alone systems and other systems, even using spreadsheets. The survey was targeted at 207 executives from all walks of life or individuals from above.

"Large data is a large amount of unstructured data." If properly managed, we can unearth effective information from large data to help businesses address security, operations, and compliance issues. All enterprises are collecting more and more data from various sources and cloud infrastructures within the enterprise, and many enterprises have not used the right tools and processes to manage the data. If this pattern continues, we will see that the companies that do so fall far behind people because they are unable to gain insight into the system and thus help the business make informed decisions. Khera explained.

In the survey, 62% of respondents said they were already managing more than 1TB of data and that more data would be available later. Global data volumes are growing at an insane rate. It is said that 90% of the data currently in existence comes from sensors, transactions, images and videos, social media, journals, etc., from the past two years.

This is the big data. If you use large data correctly, it will provide you with the coveted intelligence and insight. In terms of security, it allows you to see what is happening on your network to protect your enterprise from advanced persistent threats and malware. It also improves operational efficiency by optimizing server and supply chain management. It can even help you deal with compliance issues.

Khera says one of the keys to controlling large data is log management, which consolidates all logs from the enterprise, builds an indexed repository, and displays it in a common user interface. This data requires the ability to normalize and correlate data and to report and send alarms.

In February 2012, Loglogic commissioned Ians to conduct an information security investment analysis for its log data management and compliance products (Isia). "The main difference between the log management of large data and the log management of general data is how to manage a large amount of log information," Ians said after visiting a number of Loglogic customers who had large processing problems. Without an Easy-to-use user interface or a fast-access index repository, it is almost impossible to find effective intelligence from a data vortex. Large data management solutions must keep up with the speed at which new information is generated. This is particularly important for alarms, which can cause serious problems if the indexing time is too long and critical alert messages are delayed. ”

The reality is that only 54% of respondents currently manage log data using a log management solution. Many businesses use system logs (syslog) or spreadsheets for log management, and Loglogic's survey showed that 33% of respondents did not manage the logs. Big Data management is a long way off.

(Responsible editor: Lu Guang)