Analyst: The survival rule of the "Big Data Age"

At the Talend Connect conference, an IT industry analyst pointed out that companies would likely be eliminated from their peers if they did not grasp the opportunities offered by large data.

Jeff Kelly is Wikibon.org's chief researcher and editor of Siliconangle. Big data technologies such as Hadoop and MapReduce are just beginning, he says, and many people still shut them out because of limited technology or outdated ideas. However, in the near future, companies that have adopted large data technologies will once again occupy the Highlands when the threshold for software use is low and a large number of companies begin to adopt them. By then, they will have richer information to make decisions and more money than other companies.

As a journalist who worked for TechTarget, Kelly said: "We've been studying big data for two or three years, and we've come to the conclusion that big data is the most powerful weapon in the industry to win a competitive advantage." When we came to this conclusion, we realized the sensation of it. It is true that we cannot think of any industry which will not be affected by large data. ”

The word "big data" is used today in the description of machine-generated mass structured and unstructured information, social media sites, and mobile devices. In addition, it is used to describe the storage, management, and analysis techniques that can extract useful business insights from information. The more familiar large data management technologies include: Apache Hadoop Distributed File Systems, MapReduce, Hive, Pig, and Mahout.

The advantages of data directivity

At Talend Connect conference talking about the business data directivity problem in the decision-making process more than Kelly, Tony Fraser is one of them. Fraser is the founder and technical partner of Neo@ogilvy, a marketing company of Ogilvy & Mather (a large PR advertising company).

Fraser said: "What we do is digital advertising." As long as you can think of areas that we all have involved. ”

Fraser, citing examples in his statement, illustrates the help that data-oriented decisions bring to their company's success. One is that Paris hotels and Las Vegas casinos work with Neo@ogilvy to attract more customers. Neo@ogilvy first collected customer positive reviews of Paris hotels through social media and travel website Needles, and found that most of them were related to the view of the Bellagio Hotel fountain and the casinos across the street. Based on this information, the company's team has launched an electronic advertisement for the Paris Hotel next to the fountain.

Fraser said: "The number of hotel bookings is soaring as the advertising is put out." ”

Barriers to the use of large data

The concept of big data management technology has already spread to the streets, Kelly says, but why are there few companies that actually use it? There are two main reasons.

First, the use of Hadoop and other large data software is extremely difficult, and training for proper use of skills has not yet been introduced. At present, the enterprise only has the high salary hires the related specialized doctor to be able to carry on the analysis to the big data equation.

"An Oracle DBA does not necessarily have the skills to manage, deploy, and monitor Hadoop," Kelly said. For example, a data engineer at the analysis level needs to write MapReduce, which can be completely different from the SQL query writing. ”

Second, most enterprises still lack the concept and plan of implementing large data.

Many large enterprises today have become accustomed to obtaining business information through data warehousing and bi reporting techniques. However, Kelly believes that the BI or Data warehouse model evaluates the past through data analysis, and that large data technologies predict the future through data analysis.

"For these companies, big data use requires a conceptual shift," he said. "You need to trust the data and follow the direction it refers to." The significance of large data is to look forward, make predictions, and then act. ”

The popularity of large data management

Kelly argues that big data management and analytics, unlike other emerging technologies, will eventually become popular or popular. However, this also requires a process.

These new applications and software technologies need to be simplified because the complexity of large data technologies is a challenge to many of the companies that are just contacted. Companies such as Talend, Hortonworks and Cloudera are now simplifying the difficulty of large data technology. Big Data technology also needs a lot of innovation to make it easier for users to deploy and manage, protect the Hadoop cluster and create integration between the process and the data source, Kelly said.

"Now that you want to be a top-tier data handler, you have to have complex techniques for writing mapreduce, SAS, or other language programs." So, we need to develop tools that eliminate some of the professionalism, so that even if you don't have a ph. D., you can write large data programs. ”

The popularity of large data involves a lot of skills training for users, including large data architecture, Hadoop deployment management, data integration and MapReduce programming.

"We need to address the problem in a comprehensive way," Kelly said. Simplify tools and technology on the one hand, and strengthen personnel training on the other hand, so that DBAs and business analysts can do the work of ' Big Data age '. ”