Machine Big data can not be separated from Hadoop

According to the data source, the big data mainly includes three kinds: the data from the business operation, the data from the human behavior and the machine data. At present, most people talk about the processing and analysis of the first two kinds of data. Founded in 2004, Splunk company unique, from the beginning of the company has been focused on machine data processing and analysis. Splunk Company Product Marketing vice President Sanjaymehta in the interview with the reporter said that the machine large data in the future has a very broad prospects for development.

Large machine data available for

What is machine data? Each activity of the people leaves traces in the machine data, which contains a clear record of customer behavior, the use of transaction processing, application behavior, service levels, and so on, such as log files, sensor data, and so on, that are machine data. "Machine-generated data is the fastest, most complex and most valuable part of the big data," says Sanjaymehta. However, the existing data analysis, management and monitoring solutions are rarely designed for this type of data. ”

The difficulty of machine data processing lies in the following three aspects: the machine is derived from different sources, and it is very complicated to relate these different sources; machine data is mainly unstructured, it is difficult to deal with a predefined architecture; machine data is very demanding for real-time processing. Splunk's products are called Machine data engines, which can effectively deal with the challenges of machine data, collect unstructured time series machine data, and make use of indexes. Sanjaymehta says Splunk can read data from any source that people can think of, such as network traffic, Web servers, custom applications, application servers, virtual machine management programs, GSP systems and even stock market sources, social media, and structured databases. And through them real-time grasp of the business situation, in-depth analysis of the entire IT system and infrastructure what happened to make the right decision.

Enhance the ease of use of Hadoop

Some of our customers tell us that they want to use Hadoop to store data at a lower cost. The problem is that it's not easy to deploy Hadoop and get more value based on it. The amount of manpower and services deployed to deploy Hadoop may be 20 times times that of deploying generic software. If you want to maximize the role of Hadoop, you need to integrate at least 13 projects with Hadoop. Many other customers reflect that the amount of data on the Hadoop platform is too large to migrate at will. "In October 2012, we introduced Splunkhadoopconnect, allowing users to easily and easily transfer data between splunkenterprise and Hadoop," Sanjaymehta said. The

Splunkhadoopconnect the transfer channel between Hadoop and Splunk product platforms, and allows users to store data on the Splunk platform for long-term storage on the Hadoop platform. Data on Hadoop can also be transmitted in real time to splunk for analysis and visualization.

For many customers, the trickiest problem is that the amount of data on Hadoop is too large to move at will.

June 22, 2013, Splunk released the Hunk Beta--splunkanalyticsforhadoop, which provides interactive data exploration analysis and visualization for the Hadoop platform. This provides more convenience for users using the Hadoop platform. The

Splunkanalyticsforhadoop is a full-featured integrated product that provides interactive data exploration, analysis, and visualization on the same platform for the data on Hadoop three essential features. "Splunkanalyticsforhadoop provides users with a simple, easy-to-use interface that not only professionals can use, but even ordinary managers can use to access and analyze data." It may take a few months to understand and analyze the data, and using Splunkanalyticsforhadoop is now possible in just an hour or even a few minutes. "Sanjaymehta said. The

Splunkanalyticsforhadoop is the first product to use the Splunk virtual indexing technology (which is applying for a patent). It allows users to seamlessly use all of Splunk's technologies, including Splunk search Processing language (SPL). It enables interactive exploration, analysis, and visualization of data stored anywhere, just as these data are stored in Splunkindex. Sanjaymehta said: "In the future, we will put more technical innovation feedbackto the Hadoop community. Currently, we are inviting specific users to participate in hunk testing. The