Cloud key technology and cloud computing model based on Hadoop

Introduction

Cloud computing is developed by the business community before it is brought to the attention of academia, which is the opposite of grid computing. After statistical analysis of the relevant academic papers on cloud computing to date, it shows that the research on cloud computing is mainly focused on the key technologies of cloud technology. The key technologies of cloud computing include virtual machines, security management, data management, cloud monitoring, energy management and computational models. The computational model of cloud computing is to study how to put forward a more efficient programming method for some kinds of application features, the current cloud computing model is numerous, and Hadoop is an open source distributed system infrastructure, and is becoming a more practical development platform. Taobao is one of the first companies in the country to use Hadoop.

1 Key technologies for cloud computing

1. 1 Virtual machines

Virtual machines are one of the key technologies in cloud computing. One of the main virtual machines currently used in cloud computing is VMware infrastructure. It is a virtual data center operating system that can unify discrete hardware resources to create a shared platform. The advantages are: ① consolidate servers to reduce IT costs, ② Anshao planned and unplanned outages to improve business continuity, ⑧ running fewer servers and dynamically shutting down unused servers.

1. 2 Security Management

Cloud computing is the integration of computer resources, and any privacy information can be found through any computer in the cloud computing facility. Cloud computing security has become an urgent problem to be solved. Among them, Siani persion and so on proposed in the Cloud computing service design process to protect the user privacy some design principles: ① Send as little personal information as possible to the cloud, or analyze the system to collect and gather only a small amount of personal information; ② use security measures to prevent unauthorized access, replication, Use or modify personal information to protect personal information in the cloud: ③ maximizes user control. In the cloud computing environment, it is more difficult to give users full control over personal information. To strengthen the control of personal information: one can allow users to control the most important personal information; The second can be entrusted to the trust of the third party to manage: ④ allows users to choose the use of personal information, that is, join, exit mechanism, ⑤ clear and limit the purpose of data use. Personal information must be used and handled by an identifiable person; ⑥ has a feedback mechanism. The designer and the interface clearly indicate the security measures taken in the cloud service and use it to provide security hints to the user.

1. 3 Data Management

Cloud computing has 3 features, such as variable computing power, data storage on untrusted hosts, and remote replication of data. From these 3 features, only two data management applications may be suitable for deployment to cloud computing: one is the data management system related to transaction processing: The other is the analysis of related data management systems. The former does not take a shared architecture, it is difficult to meet acid requirements for remote data replication, and there is a greater risk of storing data on untrusted hosts. Acid requirements are not necessary for a data management system based on analysis, while ensuring that sensitive data is not analyzed to ensure its safety. Therefore, the analysis based data management system should be suitable for deployment to the cloud computing environment.

1.4 Cloud monitoring and energy management

Cloud monitoring is the ability to monitor virtual machines with the spread of cloud computing, the increasing cloud facilities, and the ability to better reflect the powerful computing power of cloud computing: how energy management saves the energy needed for computing facilities in cloud facilities, effectively consolidates resources, and lowers costs.

1. 5 Computing model of cloud computing

The current distributed computing system can provide powerful computing power. But non-professional users are not effective, and a huge task is likely to be a significant drop in performance due to the occasional beginner's operations. In order to solve these problems, we should provide users with a highly abstract product system. These are the computational models of cloud computing. When it comes to cloud computing models, you have to say an open source framework, which is Hadoop.

Hadoop is a basic architecture of distributed systems. Developed by Apache to allow users to develop distributed programs without understanding the distributed underlying details. To put it simply. Hadoop is a software platform that makes it easier to develop and run large-scale data processing.

Hadoop implements a Distributed File system (Hadoop distributedfile System), referred to as HDFs. HDFs is characterized by high fault tolerance and is designed to be deployed on inexpensive hardware. And it provides high transmission rates to access application data for applications with large datasets. HDFs relaxed the requirements of the POSIX standard. This lets you access the data in the file system in the form of streaming.

Hadoop uses a master/slave structure (Figure 1), which has an MAS. ter, mainly responsible for namenode work and jobtracker work, job. The primary responsibility of tracker is to initiate, track, and dispatch the task execution of each slave. There will also be multiple slave, each slave usually has datanode functions and is responsible for tasktracker work. Tasktracker performs map tasks and reduce tasks based on application requirements in conjunction with local data.

Figure 1 Hadoop structure

(Responsible editor: The good of the Legacy)