Large data will become an important enterprise asset

Large data will evolve into important corporate assets

In my opinion, the concept of large data can be explained in the following two ways:

1. From a technical point of view, large data refers to the large amount of data and complexity to be unable to use traditional database technology for governance and value discovery. In this kind of demand, various NoSQL, Newsql, open source technology or commercial platform are emerging and evolving.

2. From a commercial point of view, large data will spawn many new business models. First, as long as the technology is sufficient, enterprises can integrate internal and external large data (such as social data, app data), better optimize their business, to achieve the traditional business model beyond. Second, as long as the technology is sufficient, the enterprise will collect and manage the large data that has not been able to collect and manage, and then discover the great commercial value. Large data will evolve into important corporate assets.

I think the current hot big data technology mainly includes the following content:

1. Hadoop HDFS for large data storage, and derived database hbase;

2. Batch processing large data items have Hadoop MR, the latest version of which is yarn. The derivative projects include Data Warehouse hive and machine learning mahout;

3. Real-time processing of large data items are: Spark and derived Data Warehouse Shark,cloudera Impala;

4. The project for flow calculation has Apache Storm.

5. Commercial large data products have one machine such as Puredata, Exadata, Hana; MPP DW such as Vertica, Aster Data, GP; MPP DM For example Yonghong DM.

The future is more optimistic about real-time large data technology because real-time large data technology can enable enterprises to explore the large data and interactive analysis, compared to the previous flexibility and dynamic batch processing large data technology, it will greatly enhance the value of large data mining efficiency and possibilities.

I think the domestic big data development momentum is good, but the following three areas still need improvement:

1. Commercial large data products or technology should be at least 1/2, and should not meet the big data on the open source. We see more and more powerful business companies in America with large numbers of data, such as Cloudera, Hortonworks, MapR, 1010Data. But domestic enterprises either large data can not get up, or the technology is relatively strong on their own churn open source, which is not conducive to resource integration, complementary advantages.

2. Two extremes. Domestic large data items are dumbbell-type, or on the super expensive one machine, or open source large data items. In fact, when you're on a big data project, you can choose a better business product or business service than the two extremes.

3. The current major data products and technologies are in the United States. As one of the most important countries of the 21st century, China should have its own mainstream data products and technology, not just to move bricks or blindly follow. I hope to work with colleagues in the field of real-time large data.