Big data is the killer app for cloud computing

The relationship between big data and cloud computing has caused some confusion. In order to facilitate the discussion of the relationship between the two, from the "calculation" and "data" of the historical relationship. Because cloud computing is first a kind of "calculation", large data is first a kind of "data", and computer is used to "calculate" "data".

Computers are software-and hardware-separated, a software-defined electronic product (programmable). An important issue in computer design is how to effectively manage hardware resources such as CPU, memory, and I/O, and how to make these resources reasonably available to applications. These two tasks were first embedded in a variety of applications, completed by the application itself, with the disadvantage of being laborious, complex and error-prone, difficult to upgrade and migrate, and repetitive work.

In the 60 's, these common functions began to be separated from the application, gradually forming a common software package, which is the operating system. The operating system is a "middleware" between hardware and applications, allowing applications and hardware to be separated and developed independently, developing into the core computer system software and making Microsoft a great company.

The common Unix-like modern operating systems are Android, BSD, IOS, Linux, OS, QNX, and the original Microsoft Windows, Windows Phone, and IBM z/OS. The operating system also extends from the original computer to mobile phones, game controllers, TV set-top boxes, smart cars and smart glasses, as well as Web servers closely related to cloud computing.

In the 70 's, the rapid development of computer has made the digital data explode, and the "massive" data management has become a new challenge. When using the file management of the common operating system for data management, no matter scalability, efficiency and convenience, it is not suitable for the management of "massive" data, and the application software is forced to embed its own design data management system. Similarly, "massive" data management is done by each application itself, and the disadvantages are laborious, complex and error-prone, difficult to upgrade and migrate, and repetitive work.

Then a special for "massive" data management of the general software came out, that is, database management system (DBMS), an application system software. DBMS includes database definition, creation, query, update and management functions, which are necessary for data management and are not in the operating system's file management system.

The famous DBMS are MySQL, PostgreSQL, SQLite, Microsoft SQL Server, Microsoft Access, Oracle, Sybase, DBASE, FoxPro, and IBM DB2, are relational DBMS. Of course, there are non-relational no SQL mode, just not so popular.

DBMS and word processing software, and so on, become the most important application software, but also the achievement of a great application software company Oracle. About less than 20 years ago, the technology and market future of operating systems and databases seemed predictable. One is Microsoft's world, one is Oracle's world.

But the internet is coming, especially when the web is starting to pop.

The operating system used by the Web server was originally designed for stand-alone, and extended for LAN-wide management of multiple servers. But when the internet giants rise and need the Web server's operating system to manage millions of Web servers, the traditional operating system is reluctant and requires a "technological revolution". The result of "revolution" is cloud computing.

Cloud computing Umbrella has many concepts, one of the core technology is virtualization. Virtualization has "1 virtual n" and "N Virtual 1" two modes, the former is mainly to save money, Amazon AWS as the representative, the latter is mainly for large data processing, Google Gae as the representative.

Cloud computing "n Virtual 1" mode, can be multiple physical computers into a supercomputer, to provide the application resource pool scheduling management services, and the traditional operating system almost exactly the same function, so often referred to as "cloud computing operating system." It's just a cloud-computing operating system that extends to the data center and even across the internet, treating each computer as a resource.

With cloud computing operating systems, cloud applications and hardware (computer resources) can be separated and developed independently. History repeats itself, cloud computing and the rapid development of SNS, microblogging, mobile Internet and IoT, with 3V characteristics of data explosion, the challenge of large data management comes first. Similarly, the general-purpose cloud computing system for computing design faces new challenges in terms of scalability, efficiency and convenience in large data management.

Historically, computers have been confronted with the challenge of "massive" data, separating data applications from data management, and creating a common DBMS. Now cloud computing facing the challenge of large data, but also will make large data applications and large data management separation, the birth of a "large database management system", and gradually towards universal and platform.

ATM (asynchronous transmission mode) is the product of the era of scarce communication resources, and TCP/IP is the product of the rich era of communication resources. Similarly, traditional DBMS is the product of the scarcity of IT resources, and the large data management system is the product of the rich era of IT resources.

Computing is the tool that can be industrialized to provide; data is a resource and a personalized asset. If office, games and so is the killer application of the PC, browser, search, SNS and so on is the killer application of the Internet, so large data is the killer application of cloud computing.

(Responsible editor: Fumingli)