It's blind it's no pressure big data must be known

During the Spring Festival, we all witnessed the world's largest "human migration" flow of traffic, logistics, flow of people, they are also the flow of data, is business opportunities. and large data, is such a need to use a proprietary platform to achieve value extraction to help decision-making analysis of the mass data set. Given the universality and importance of large data in cloud computing, mobile, social and other trends, this article will provide a broad range of readers with the basics of big data to help more people understand big data and tap into more business opportunities.

Alternative "v Vendetta Team"

When it comes to big data, it's natural to think of the 4V features of Big Data: Volume (large data), Velocity (strong real-time), produced (variety), veracity (authenticity). In addition, usually large data also has the characteristics of value (value), which is one of the main drivers of attention to large data. The "V vendetta" here refers to the redefinition and excavation of values in the age of large data, which fills every corner of society.

Large data is multidimensional and highly complex. The value of large data includes, but is not limited to: Data organization and management, infrastructure, decision support, and automation interface and analysis. With the rise of new data sources such as social data, enterprise content, transaction and application data, the limitations of traditional data sources are broken, and enterprises need effective information governance to ensure their authenticity and security.

Four elements and challenges of

Volume

Data volume is huge. From TB level, to PB level. As of now, all the printed materials produced by human beings are 200PB (1PB=1024TB), and the amount of data in all the words that mankind has said in history is about 5EB (1EB=1024PB). Currently, the typical personal computer hard disk capacity of TB level, and some large enterprise data is already close to EB level.

Velocity

Fast processing speed, 1 second law. According to IDC's "Digital Universe" report, global data usage is expected to reach 35.2ZB by 2020. In the face of such a huge amount of data, the efficiency of processing data is the life of the enterprise.

Produced

A wide range of data types, log, video, pictures, geographical information and so on. The diversity of types also allows data to be divided into structured and unstructured data. Compared with the traditional text-oriented structured data, which is easy to store, unstructured data is more and more, these multiple types of data have higher demands on the processing ability of data.

Veracity

Only real and accurate data can make the control and governance of data really meaningful.

Large data: Beholder

Big data is an opportunity and a challenge for everyone. In large academic fields such as Big Data Science, blog, RFID (radio Frequency Identification Technology), sensing network, social network, social data, network documentation, internet search, call center, astronomy, Meteorology, geography, biology and other data, civil, military, video, E-commerce, etc. are widely used.

Large Data Science

The Large Hadron Collider (SCM Hadron Collider) is a collision-type particle accelerator at CERN, a European nuclear research organization located in the outskirts of Geneva, Switzerland, mainly as an international high energy physics Study. The lab distributes 150 million sensors, averaging up to 40 million times times the data per second, with 600 million collisions per second. 99.999% of the data is filtered and not recorded, which means that only 100 collisions (per second) of data are most valuable.

Therefore, the data that really needs to be collected and processed is only 0.001% of the sensor data. Data for the LHC lab is growing at 25PB a year (regardless of data backup).

If all the sensor data needs to be recorded and processed, the workload will be extremely large and unsustainable. In that case, the annual data growth will reach 150 million PB, which is equivalent to 500EB per day.

Government Departments

Last year, Barack Obama's administration announced the establishment of a large data research and development initiative (big and Development initiative), which is dedicated to helping government departments solve major problems with big data. The initiative includes 84 different large data project projects and 6 departments. The federal government also has six of the top ten supercomputers in the world today. The NASA department responsible for meteorological simulations also stores 32PB meteorological observations and analog data in its discovery supercomputer cluster. These, in fact, also illustrate the importance of government departments to large data, and the application for this purpose.

Business Area

In business, large data solutions and applications are all over the world. The famous Facebook social platform has already launched data mining and decision analysis based on user behavior analysis to analyze 50 billion photos of all its users. Wal-Mart handles more than millions of customer transactions per hour, with volumes up to 2.5PB (2560TB) equivalent to 167 times times the amount of library books in the United States Congress.

Facebook data information

Application development

According to the broad sense of ICT for development (ICT4D) effective applications, large data can play an important role in socio-economic development. On the one hand, the use of large data to provide cost-effective decision-making analysis, such as medical, recruitment, economic development, crime prevention, natural disasters, resource management and other fields. On the other hand, privacy, interoperability challenges, the refinement of algorithms, and a new digital divide due to the lack of technical infrastructure and human resources: the information gap brought about by data-based decision support.

From here we can see that large data in various industries and fields, due to the application and business of their different and create different challenges or opportunities, but anyway, large data is a trend, a short-term pain will bring the opportunity. And that's why we also need to be particularly aware of some of the current solutions and successful application cases for large data.

Value

Low value density. The value density is inversely proportional to the size of the total data. How to complete the value of the data more quickly through the powerful machine algorithm is a difficult problem to be solved in the background of large data. With the rise of new data sources such as social data, enterprise content, transaction and application data, the limitations of traditional data sources are broken, and enterprises need effective information governance to ensure their authenticity and security.

Large data: Beholder

Big data is an opportunity and a challenge for everyone. In large academic fields such as Big Data Science, blog, RFID (radio Frequency Identification Technology), sensing network, social network, social data, network documentation, internet search, call center, astronomy, Meteorology, geography, biology and other data, civil, military, video, E-commerce, etc. are widely used.

Large Data Science

The Large Hadron Collider (SCM Hadron Collider) is a collision-type particle accelerator at CERN, a European nuclear research organization located in the outskirts of Geneva, Switzerland, mainly as an international high energy physics Study. The lab distributes 150 million sensors, averaging up to 40 million times times the data per second, with 600 million collisions per second. 99.999% of the data is filtered and not recorded, which means that only 100 collisions (per second) of data are most valuable.

Therefore, the data that really needs to be collected and processed is only 0.001% of the sensor data. Data for the LHC lab is growing at 25PB a year (regardless of data backup).

If all the sensor data needs to be recorded and processed, the workload will be extremely large and unsustainable. In that case, the annual data growth will reach 150 million PB, which is equivalent to 500EB per day.

Government Departments

Last year, Barack Obama's administration announced the establishment of a large data research and development initiative (big and Development initiative), which is dedicated to helping government departments solve major problems with big data. The initiative includes 84 different large data project projects and 6 departments. The federal government also has six of the top ten supercomputers in the world today. The NASA department responsible for meteorological simulations also stores 32PB meteorological observations and analog data in its discovery supercomputer cluster. These, in fact, also illustrate the importance of government departments to large data, and the application for this purpose.

Business Area

In business, large data solutions and applications are all over the world. The famous Facebook social platform has already launched data mining and decision analysis based on user behavior analysis to analyze 50 billion photos of all its users. Wal-Mart handles more than millions of customer transactions per hour, with volumes up to 2.5PB (2560TB) equivalent to 167 times times the amount of library books in the United States Congress.

Facebook data information

Application development

According to the broad sense of ICT for development (ICT4D) effective applications, large data can play an important role in socio-economic development. On the one hand, the use of large data to provide cost-effective decision-making analysis, such as medical, recruitment, economic development, crime prevention, natural disasters, resource management and other fields. On the other hand, privacy, interoperability challenges, the refinement of algorithms, and a new digital divide due to the lack of technical infrastructure and human resources: the information gap brought about by data-based decision support.

From here we can see that large data in various industries and fields, due to the application and business of their different and create different challenges or opportunities, but anyway, large data is a trend, a short-term pain will bring the opportunity. And that's why we also need to be particularly aware of some of the current solutions and successful application cases for large data.