The core technology of cloud computing

The

Cloud computing system employs many technologies, including programming models, data management technologies, data storage technologies, virtualization technologies, and cloud computing platform management technologies.

The

(1) programming model

MapReduce is a Java, Python, C + + programming model developed by Google, a simplified distributed programming model and an efficient task scheduling model for parallel operations with large datasets (greater than 1TB). The rigorous programming model makes programming in cloud computing environments simple. The idea of the MapReduce mode is to decompose the problem into a map (map) and reduce (simplify) by the map program to cut the data into unrelated chunks, allocate (dispatch) to a large number of computers, and achieve the results of distributed operations. The result is then remitted through the reduce program.

(2) mass data distributed storage technology

Cloud computing system consists of a large number of servers, while for a large number of users, so the cloud computing system uses distributed storage to store data, using redundant storage methods to ensure the reliability of data. The widely used data storage system in cloud computing systems is the open source implementation HDFs of GFS developed by Google's GFS and Hadoop teams. GFS, the Google file system, is an extensible Distributed file system for large, distributed applications that access large amounts of data. The GFS design idea differs from the traditional file system, and is designed for large-scale data processing and Google application features. It runs on inexpensive, common hardware, but it can provide fault tolerance. It can provide a large number of users with a higher overall performance of the service. A GFS cluster consists of a master server (master) and a large number of block servers (Chunkserver), which are accessed by many clients (client). The primary server stores the file system's metadata, including the namespace, access control information, mapping from file to block, and the current position of the block. It also controls system-wide activities such as block lease (lease) management, orphan block garbage collection, block migration between block servers. The primary server periodically communicates with each block server through Herat beat messages, passing instructions to the block server and collecting its status. Files in GFs are cut into 64MB blocks and are stored in redundant storage, with each data holding more than 3 backups in the system. The customer's exchange with the primary server is limited to the operation of the metadata, and all data communication is directly connected to the block server, which greatly improves the efficiency of the system and prevents the primary server from overloading.

(3) massive data management technology

Cloud computing requires the processing and analysis of distributed, massive amounts of data, so data management techniques must be able to efficiently manage large amounts of data. The data management technology in cloud computing system is mainly Google's BT (big Table) data management technology and HAOpen source data management module developed by Doop team HBase. BT is a large distributed database built on Gfs,scheduler,lock service and map reduce, which, unlike traditional relational databases, processes all data as objects and forms a huge table for distributing and storing large structured data. Many of Google's projects use BT to store data, including Web queries, Google and Google Finance. These applications have different requirements for BT: The size of the data (from URLs to Web pages to satellite images) differs in response rates (from the back-end bulk processing to the real-time data services). For different requirements, BT has successfully provided a flexible and efficient service.

(4) Virtualization Technology
　　
Virtualization technology enables software applications to be isolated from the underlying hardware, which includes splitting patterns that divide individual resources into multiple virtual resources, as well as aggregation patterns that integrate multiple resources into a single virtual resource. Virtualization technology can be divided into storage virtualization, computational virtualization, network virtualization and so on, and computing virtualization is divided into system-level virtualization, application-level virtualization and desktop virtualization.
　　

(5) Cloud computing platform management technology
　　
Cloud computing resources are large, the number of servers and distributed in different locations, while running hundreds of of applications, how to effectively manage these servers, to ensure that the entire system to provide uninterrupted service is a huge challenge. Cloud computing System Platform management technology can enable a large number of servers to work together to facilitate the deployment and opening of the business, rapid detection and recovery of system failures, through automated, intelligent means to achieve a reliable operation of large-scale systems.