The core technology of cloud computing

The advent of cloud computing has made many people see it as a new technology, but in fact its embryonic form has been in place for many years, only in recent years it has only begun to make relatively rapid progress. To be exact, cloud computing is the product of the evolution of large scale distributed computing technology and its supporting business model, and its development mainly depends on the common development of virtualization, distributed data storage, data management, programming mode, information security and other technologies and products. In recent years, the evolution of managed, back-to-charge, on-demand delivery and other business models has also accelerated the cloud computing market. Cloud computing not only changes the way information is delivered, it also overturns the delivery model of traditional ICT systems. Cloud computing is not so much a technological innovation as a shift in thinking and business models.
Let's talk a little more about the core technologies of cloud computing.

Cloud computing is a data and processing power-centric intensive computing model, which integrates many ICT technologies and is the product of "smooth evolution" of traditional technology. Among them, virtualization technology, distributed data storage technology, programming model, large-scale data management technology, distributed resource management, information security, cloud computing platform management technology, green energy-saving technology are the most critical.

1. Virtualization Technology

Virtualization is one of the most important core technologies of cloud computing, which provides infrastructure support for cloud computing services, and is the main driver of the rapid trend of ICT services to cloud computing. It can be said that there is no virtualization technology will not be the landing and success of cloud computing services. As cloud computing applications continue to rise, the industry's focus on virtualization technology has also raised a new level. At the same time, our survey found that many people have misconceptions about cloud computing and virtualization, and that cloud computing is virtualization. In fact, virtualization is an important part of cloud computing, but not all.
Technically, virtualization is a form of computing that simulates computer hardware in software and provides services to users with virtual resources. The aim is to rationalize the provision of computer resources to provide services more efficiently. It breaks the physical division between hardware of the application system, and realizes the dynamic of the architecture, and realizes the centralized management and use of the physical resources. The greatest benefit of virtualization is to increase the elasticity and flexibility of the system, reduce costs, improve services, and increase resource utilization efficiency.

In terms of performance, virtualization is divided into two application modes. One is to virtual a powerful server into a number of separate small servers, to serve different users. The second is to virtual multiple servers into a powerful server, complete the specific functions. The core of the two models is unified management, dynamic allocation of resources, improve resource utilization. In cloud computing, both of these models have more applications.

2. Distributed Data storage technology

Another big advantage of cloud computing is the ability to process massive amounts of data quickly and efficiently. This is critical in the data explosion today. To ensure high data reliability, cloud computing typically uses distributed storage technology to store data in different physical devices. This mode is not only free from the limitations of hardware devices, but also more scalable, can quickly respond to changes in user needs.

Distributed storage is not the same as traditional networked storage, the traditional networked storage system uses centralized storage server to store all the data, the storage server becomes the bottleneck of system performance, and cannot meet the needs of large-scale storage applications. Distributed networked storage system uses the extensible system structure, uses multiple storage servers to share the storage load, and uses location server to locate and store information, which not only improves the system reliability, availability and access efficiency, but also is easy to expand.

In the current realm of cloud computing, Google's GFS and Hadoop-developed open source system HDFs are two of the more popular cloud computing distributed storage systems.
GFS (Google File System) technology: Google's non-open-source GFs (Googlefile System) cloud computing platform meets the needs of a large number of users and provides services in parallel to a large number of users. The data storage technology of cloud computing has the characteristics of high throughput rate and high transmission rate.

HDFs (Hadoop distributed File System) technology: Most ICT vendors, including Yahoo and Intel's "cloud" program, use HDFS data storage technology. Future developments will focus on ultra-large-scale data storage, data encryption and security assurance, and the continued increase in I/O rates.

3. Programming Mode

In essence, cloud computing is a multi-user, multi-tasking, concurrent processing system. Efficient, simple and fast is the core concept, it is designed to easily distribute powerful server computing resources to end users through the network, while guaranteeing a low cost and good user experience. In this process, the choice of programming patterns is critical. Distributed parallel programming patterns in cloud computing projects will be widely adopted.
The purpose of the distributed parallel programming model is to make more efficient use of software and hardware resources, allowing users to use applications or services more quickly and easily. In the distributed parallel programming model, the background complex task processing and resource scheduling is transparent to the user, so the user experience can be greatly improved. MapReduce is one of the mainstream parallel programming models of cloud computing. The MapReduce pattern automatically divides tasks into subtasks, and enables the height and allocation of tasks in large compute nodes through map and reduce two steps.

MapReduce is a Java, Python, and C + + programming model developed by Google, primarily for parallel operations with large datasets (larger than 1TB). The idea of MapReduce mode is to decompose the problem into map (map) and reduce (reduction), first, through the map program to cut the data into unrelated chunks, allocate (dispatch) to a large number of computer processing, to achieve the effect of distributed computing, The result is then exported through the reduce program.

4. Large-scale data management

Processing massive amounts of data is a big advantage of cloud computing. So how to handle it involves many levels of things, so efficient data processing technology is also one of the key technologies of cloud computing. For cloud computing, data management poses a huge challenge. Cloud computing is not only to ensure the storage and access of data, but also to be able to carry out a specific search and analysis of large amounts of data. Because cloud computing needs to process and analyze massive amounts of distributed data, data management technology must be able to efficiently manage large amounts of data.
Google's BT (BigTable) data management technology and the open source data management module, developed by the Hadoop team, are the industry's most typical large-scale data management technologies.
BT (BigTable) Data management technology: BigTable is a non-relational database, a multi-dimensional sort of distributed, persistent storage map.bigtable built on GFS, Scheduler, Lock service and MapReduce, Unlike traditional relational databases, it handles all of the data as objects, creating a huge table for distributing storage of large-scale structured data. The bigtable is designed to reliably process petabytes of data and can be deployed on thousands of machines.

The Open source data management module Hbase:hbase is a subproject of Apache's Hadoop project, located in a distributed, column-oriented, open-source database. HBase differs from the general relational database, which is a database suitable for unstructured data storage. The other difference is that HBase is column-based instead of row-based patterns. As a highly reliable distributed storage system, HBase performs better in terms of performance and scalability. With HBase, you can build large-scale structured storage clusters on inexpensive PC servers.

5. Distributed Resource Management

Cloud computing uses distributed storage technology to store data, so it is natural to introduce distributed resource management technology. In a multi-node concurrent execution environment, the state of each node needs to be synchronized, and in the event of a single node failure, the system needs an effective mechanism to ensure that the other nodes are unaffected. and distributed resource management system is just such technology, it is the key to ensure the state of the system.
In addition, the cloud computing system is often handled by a large number of resources, less than hundreds of servers, tens of thousands of units, and may leap across multiple regions. And there are thousands of applications running in the cloud platform, how to manage the resources effectively, ensure their normal service delivery, and need strong technical support. Therefore, the importance of distributed resource management technology is conceivable.

The world's major cloud computing solutions/service providers are actively engaged in the development of relevant technologies. Among them, Google's internal use of Borg technology is well-praised by the industry. In addition, the Microsoft, IBM, Oracle/sun and other cloud computing giants have corresponding solutions proposed.

6. Information Security

Survey data show that security has become one of the most important reasons hindering the development of cloud computing. Data shows that 32% of organizations that already use cloud computing and 45% of organizations that have not yet used cloud computing are using cloud security as the biggest barrier to further cloud deployment. Therefore, in order to ensure the long-term stability and rapid development of cloud computing, security is the first problem to be solved.
In fact, cloud computing security is not a new problem, the traditional internet has the same problem. Just after the advent of cloud computing, security issues became even more pronounced. In the cloud computing system, security involves many aspects, including network security, server security, software security, system security and so on. Therefore, some analysts believe that the development of the cloud security industry, the traditional security technology will be referred to a new stage.

Today, both software security vendors and hardware security vendors are actively developing cloud computing security products and solutions. Security vendors, including traditional anti-virus software vendors, soft and hard firewall vendors, and ids/ips vendors, have joined the Cloud security arena. I believe that in the near future, cloud security issues will be well resolved.

7. Cloud Computing Platform Management

The large scale of cloud computing resources, the large number of servers and distribution in different locations, while running hundreds of applications, how to effectively manage these servers, to ensure uninterrupted service of the entire system is a huge challenge. The platform management technology of cloud computing system needs the ability to efficiently allocate large amount of server resources and make it work better together. The key to cloud computing platform management technology is to conveniently deploy and open new business, quickly discover and recover system failures, and realize the reliable operation of large-scale systems through automated and intelligent means.
For the provider, cloud computing can have three deployment models: public, private, and hybrid. The requirements for platform management vary greatly from three modes. For users, the scale and manageability of cloud computing systems required by enterprises vary greatly due to the control of ICT resource sharing, the requirements for system efficiency, and the cost-to-invest budget of ICT. Therefore, the cloud computing platform management solution should take more account of the customization requirements, can meet the application needs of different scenarios.

Many vendors, including Google, IBM, Microsoft, Oracle/sun, etc., have a cloud computing platform management solution to launch. These programs can help enterprises to realize the integration of infrastructure, realize the unified management of enterprise hardware resources and software resources, unify distribution, unified deployment, unified monitoring and unified backup, and break the monopoly of application to resources, so that the enterprise Cloud computing platform value can be fully realized.

8, green energy-saving technology

Energy conservation and environmental protection are the major themes of the world. Cloud computing is also known for its low cost and high efficiency. Cloud computing has enormous economies of scale, while increasing the efficiency of resource utilization, saving a lot of energy. Green energy-saving technology has become an essential technology for cloud computing, and more and more energy-saving technologies will be introduced into cloud computing in the future.

Carbon Disclosure Project (Carbon Disclosure Project, CDP) recently released a study on cloud computing that will help reduce carbon emissions. U.S. companies migrating to the cloud can reduce carbon emissions by 85.7 million tonnes a year, the report says, which is equivalent to the total amount of carbon emitted by 200 million barrels of oil.

in short, cloud service providers need to continually improve their technology to make cloud computing greener.

The core technology of cloud computing