The ability to foresee "big data" is now emerging

With the rise of mobile internet and cloud computing, the proliferation of data has made big data concepts hot in the Internet industry. According to IDC's 2011 Digital Universe Study, the total amount of global information will grow by one fold every two years. In 2011 alone, the total amount of data created and replicated globally was 1.8ZB (1.8 trillion GB). Compared with a 2010 increase of more than 1ZB, by 2020 this value will increase to 35ZB, which is the creation and production of large data.

Large data is an important information asset

"Big Data" is not a new thing, it has been in the fields of physics, biology, environmental ecology, automatic control, and military, communications, finance and other industries for many years. In recent years, with the rapid development of the Internet industry, large data are getting more and more attention, beginning to glow agitatedly vitality. In early 2012, the New York Times wrote that the "Big data age" had already arrived. The article points out that "big data" is affecting all areas. For example, in business, economics and other areas, more and more decisions will depend increasingly on the collection and analysis of data, and the judgment based on experience and intuition will be challenged, while the ability to foresee "big data" has emerged in areas such as public health, economic development and economic forecasts.

Gartner defines "Big data" as a huge, high growth rate and diverse information asset that requires new processing patterns to be more decision-making, Insight discovery and process optimization. This definition points out that big data will become an important asset for business and society!

In a way, large data is the frontier technique of data analysis. In short, the ability to quickly get valuable information from a variety of data is a big data technology. It is important to understand this and it is this that has the potential to move to many businesses.

Large data can be divided into large data technology, large data engineering, large data science and large data application fields. At present, most people talk about large data technology and large data application. Engineering and scientific issues have not been taken seriously. Large data engineering is a systematic project for the planning and construction of large data. Large data science focuses on the discovery and validation of large data during the development and operation of large data networks and its relationship with nature and social activities.

Large data has four levels of characteristics, they are: first, the volume of data is huge: From TB level, jumped to the PB level, and even EB level; second, a wide range of data types: data, blogs, videos, pictures, geographical information, etc., almost all-encompassing. Third, the value density is low: the amount of irrelevant information is huge, which needs deep excavation analysis. For example, in a continuous video monitoring process, the data that may be useful is only two seconds. The fast processing speed: "1-Second Law", that is, real-time analysis rather than batch-type analysis, immediate results rather than hindsight, and this is fundamentally different from the traditional data mining technology. The industry summed it up as 4 "V" Volume, produced, Value, Velocity.

With the development of Internet technology, the view of "data as assets" has reached a consensus in the industry. If cloud computing provides a storage and access place and channel for data assets, then how to activate the data assets for national governance, enterprise decision-making and even personal life services is the core issue of large data, but also the inherent soul of cloud computing and the inevitable direction of escalation. From the point of view of technology development, large data is still in the high speed growth period.

Large data is a new industry with national strategic significance, and is being highly concerned by the government. "Twelve-Five" national Strategic Emerging Industry Development plan, the support of mass data storage, the research and development of the processing technology and industrialization; "Twelve-Five" in "The Internet of Things", the information processing technology is also listed as one of the four key technological innovation projects, including mass data storage, data mining, image video intelligent analysis, Another three key technological innovation projects, including information sensing technology, information transmission technology, information security technology, are also important components of large data industry, closely related to the development of large data industry.