Forbes: Big Data brings high cost Hadoop needs to be perfected

Today we have entered a large data age, because the emergence of innovative data management technologies enables organizations to analyze all http://www.aliyun.com/zixun/aggregation/18278.html "> Data types." It also allows businesses to discover new business opportunities every day.

With the development of Internet technology, a great amount of information is produced every day in the network, which includes semi-structured and unstructured data. Organizations can find out what their customers really need and why they need it through an analysis of massive amounts of information. But the real cost of the new business model has yet to be fully understood.

Diversity of

From an IT perspective, information structure types have roughly gone through three of waves. It must be noted that the new wave does not replace the old wave, which is still evolving, with three types of data structures always exist, but one type of structure is often dominant in other structures:

Structured information-This information can be found in relational databases and has dominated it applications for years. This is the key task OLTP system business depends on the information, in addition, the structure of database information can be sorted and queried;

Semi-structured information-This is the second wave of it, including e-mail, word processing files, and information stored and posted on the web. Semi-structured information is based on content, can be used for search, which is the reason for Google's existence;

Unstructured information-This information can be considered essentially a bit-mapped data in its essential form. Data must be in a perceptible form (such as being able to be heard or seen in audio, video, and multimedia files). Many large data are unstructured, and their sheer size and complexity require advanced analysis tools to create or leverage a structure that is easier to perceive and interact with.

Market leaders are not only gaining competitive advantage by analyzing the data stored in the format. The data analysis allows them to gain insight into the customer's behavior patterns, which directly affect their business.

Two specific industries-telecoms and retailing-have invested heavily in data warehousing solutions. Over time, both the telecommunications and retail industries have been studying the cumulative number of customer transactions and interactive data to identify key performance metrics. For example, annual income, the cost of each customer's access to promotional information over the network, and the peak of sales.

However, with the proliferation of data, even the market leader can not afford, the traditional data Warehouse has been unable to store and manage petabytes of the original detailed data. Businesses tend to back up data to offline tapes, but that's not easy to access. Business challenges are everywhere, for example, when Christmas coincides with Saturday, businesses need to analyze data from 7 years ago (coinciding with Christmas and Saturday) to understand specific patterns. Importing a large amount of historical data into a data warehouse is not only challenging, but also costly.

(Responsible editor: The good of the Legacy)