"Big Data" in cloud computing

The data center is becoming the "information power plant" of the new era and becoming the infrastructure of knowledge economy. Over the past year, "Big Data" is becoming a hot topic. The development of information technology in half a century mainly solves the storage, processing and application of "structural" data in cloud computing. The characteristics of "structural" data are like you go to the bank to access money, the bank's computer system records your name, after the name is your access to the amount of money, time, type and other information. These data are characterized by "strong logic" and each "cause" has "fruit".





However, in reality, large amounts of data do not have "manifest" causal relationships, such as a moment of traffic jams, weather conditions, human state (psychological and physical), it is characterized by time, mass and elasticity, such as a catastrophic weather analysis contains hundreds of PB (Petabyte, 1PETABYTE=1024TB) data. A social event, such as the data on the Internet (microblogs, Memorials, articles, videos, etc.) that occurred immediately after the death of Steve Jobs, was also a sudden outbreak.





Traditional computer design and software are mainly to solve the "structural" data. A new computational framework is required for this new type of "unstructured". The age of the Internet, especially social networks, e-commerce and mobile communications, has brought human society into a new era of structural and unstructured data information based on "PB", which is the era of "big Data".




Enterprise and Technology

large data




an era of mass production, sharing, and application data is opening, and each of us is the creator and user of data, and Weibo and social networks are the best examples.




After the Industrial Revolution,
books such as the carrier of knowledge can be doubled about every ten years; After 1970, knowledge can be doubled in about every three years; today, the total amount of global information can be doubled every two years; the amount of Internet data in 2010 is more than the sum of all previous years. Today, people can produce a number of petabytes of data every day, from logs, microblogs, share photos, transfer video, a variety of formats of data real-time and constantly updated. In medical and health, geographic information, E-commerce, film and television entertainment industry, every day is also creating a large number of data.





data is becoming an important feature of the transformation from industrial economy to knowledge economy, and it has become the most critical production factor and product form in the new era.





Companies such as Apple, Facebook and Amazon, which represent the big data age, are becoming the driving force of this change. At the same time, new businesses are emerging, such as the 2007 Dropbox company, the founder of less than 27 years old, the valuation has more than 4 billion U.S. dollars, this is a provider of file backup and sharing services, allowing users to sync and share files between different platforms and devices, Dropbox more than 25 million users, the number of files stored daily more than 200 million, Apple had to bid 800 million dollars to buy it did not succeed.





It is worth mentioning that the company was the first to use Amazon's S3 cloud computing platform, to Low-cost rapid start. Amazon's cloud computing data storage Service, which was designed to take advantage of idle server resources, can now generate nearly 1 billion dollars a year, and in short supply. Early this year, Amazon S3 cloud storage services stored 262 billion copies, the number has recently become 566 billion, doubled 1 time times more. Amazon now calls its own S3 data storage service, worrying that it is not the data storage costs, but more important data processing problems.





's large data in cloud computing has several core elements, such as the collection and sharing of data in the cloud, the seamless connection of personal data (at any time, anywhere, synchronization), and data tracing analysis and mining.





's large data systems, derived from Yahoo's Hadoop, are increasingly important as open source distributed data-processing system architectures, which are primarily geared toward storing and processing hundreds of terabytes of structured, semi-structured, or unstructured data, up to PB levels. The mapreduce provided by Hadoop can decompose large data problems into multiple child problems, assign them to hundreds of processing nodes, and then assemble the results into a small dataset, making it easier to analyze the final results.





Hadoop has become the main solution for large data analysis in companies such as AOL, Facebook, Twitter and Netflix. For example, Facebook has more data a day than many big companies do in a year, collecting and storing millions of of files per day from Hadoop, and using open source Apache hive Data Warehouse tools to focus on the data.





Opera FX's innovative company provides more compelling services: Customers upload data to opera platform, Opera will be based on the user data pool in the relevant "signal" analysis, according to each customer's personalized needs, Opera employs experts from various industries to help them with data analysis, and Opera FX has more than 100 million dollars in annual turnover.





new startups like MapR, Zettaset, Cloudera, hstreaming and Hadoop-related big data companies are favored in the capital markets. Its rapid growth will be the next power to change information technology.




The economic significance of
large data





large data provides a space for the computation of cloud computing in large scale and distributed, which solves the problems that traditional computers cannot solve. At the same time, the computing standards and software in this field have just started, providing unprecedented opportunities for new software, hardware and application innovations around the world.





large amounts of data need to be stored to accommodate it, fast, low-cost, green Data Center deployment becomes the key. Over the past year or so, companies such as Google, Facebook and Rackspace have been building a new generation of data centers, mostly using more efficient, energy-efficient, customized cloud servers for large data storage, mining and cloud computing operations.





Data Center is becoming the "information power plant" of the new era and becoming the infrastructure of knowledge economy. Extracting valuable information from massive data, data analysis makes the data more meaningful, and will affect the government, finance, retail, entertainment, media and other fields, bring revolutionary changes. "Big data is the strategic direction of information technology's future development that will spawn trillions of of billions of dollars of software companies in the next generation," said Accel, a leading venture capitalist who invests in Facebook. ”





Big Data will enrich our understanding of the world. From the quantitative, structural world, to the uncertain, unstructured world. This transformation enables us to understand the real information, improve the level of decision-making, and when the community has a more complete and analytical capability for natural data, our ability to grasp and predict events will increase. Cloud-based information storage, sharing and mining tools provide a tool for knowledge production, which is particularly important for China at this stage by analyzing and predicting large data to make decisions more accurate.





China has a huge population and application market, complex, full of changes, such a large group of users, making China the world's largest data country. To solve this problem caused by large-scale data and to explore the solution based on large data is an important means to upgrade and improve the efficiency of China's industry.





"Data Bank" and "cloud storage"





The concept of "data bank" gradually becomes the pilot of application. Companies will store the data we generate, like money assets, in the data bank.





enterprise Computing in large data environment can transfer existing data and documents to cloud computing environment, accelerate data management, data mining and other software applications in cloud environment, business model exploration and digital decision-making. Through data sharing and business synergy, the government can improve the efficiency of Office intelligence and decision-making through the storage and sharing of massive data, and solve many problems such as urban traffic, population management, public safety, medical and health, and so on.





data not only represents productivity, will also become an important asset, perhaps in the future, we leave the next generation of assets, not the amount of deposits in the bank, but information assets, perhaps 10 years, 15 years later, there will be the national data banks, relative to today's wealth assets, inside the preservation of our information assets.





large data on a variety of basic and application software, hardware products will be gradually introduced, and this aspect of Chinese entrepreneurial enterprises and Silicon Valley distance is also shortened. The Beijing Super Cloud computer company, which we invested at the end of this year, will also launch the world's first "Hadoop" server to solve big data problems in Beijing and a Silicon Valley company.





"Cloud storage" will become popular because of Apple's "ICloud". Traditional home appliance enterprises to provide "cloud home appliances" will become a hot application. Large-scale, massive "cloud data" center construction will become global and China's next round of infrastructure investment focus.





whether cloud computing or large data technology and applications, today is the early stage of development, equivalent to the early 80 's PC. We have seen its broad application prospects and the power to change the world economy. But we still can't predict exactly what business model and what kind of business, entrepreneurs will achieve the ultimate success. Exploration, Learning and trial and error are the only keys to the door of this new world.