Cloud era: The value and challenge in the development of large data

The volume of corporate data is rising today, is being mentioned by more and more people and is often associated with cloud computing. As enterprise users become more mature in their understanding of large data, the size of large data has grown from dozens of TB to more than 10 PB-scale data. For enterprises, the current large data has 4 V characteristics worth Attention: http://www.aliyun.com/zixun/aggregation/13584.html "> Mass data volume, large-scale velocity, Diverse produced and data value values.

Large data will undoubtedly bring great value to human society. Research institutions can help with large data services, such as environment, resources, energy, meteorology, aerospace, life and other fields of exploration. Large data is affecting the transformation of business model, the analysis of data, optimization is becoming the effective way to enhance the core competitiveness.

IDC's Digital Universe study, "Extracting value from Chaos," points out that global data volumes are doubling every two years, and the amount of data created and replicated in 2011 is 1.8ZB (1ZB=10^6PB), an increase that is still accelerating and is expected to reach nearly 8ZB by 2015. In fact, it is not easy to define large data on the scale of data, but to consider the complexity of data processing and analysis to meet the needs of users. For simple user needs, the data is terabytes to petabytes, which can be called large data, and for complex user needs, the data volume is GB to terabytes to become large data.

According to the characteristics of the era of information explosion, the industry summed up the large data as "4V" Volume (Volume), diversity (produced), velocity (velocity) and value. Volume refers to the amount of data that is aggregated together for analysis to be very large. Ubiquitous mobile devices, wireless sensors are generating data every minute, and hundreds of millions of users ' internet services are generating huge amounts of interaction at all times.

Diversity (produced) refers to the complexity of data types. It's all kinds of data, the past data is more structured, and now more and more data is semi-structured, or even completely unstructured data, from the enterprise, from the Internet, from the user to a variety of data into our servers, access to the data center, So there are a lot of challenges, so much data how to turn it into information, how to turn information into knowledge, turn knowledge into decision-making, which requires more good data processing capabilities.

Speed (velocity) means that the speed of data processing must meet the requirements of real-time. The data volume grows faster, needs to process the speed and the response time is more and more fast, the system's delay request is quite high.

IDC research shows that there are 1.8 trillion gigabytes of data in the digital field. Corporate data is growing at a rate of 55% per cent a year. It is expected that in the next 5 years, most data warehouses will use the column-type storage technology.

(Responsible editor: The good of the Legacy)