Best Practice of Cloud Computing Course for Beginners

This article introduces distributed computing to cloud computing beginners  in cloud computing courses and how to quickly learn distributed computing.

Want to get more courses, please refer to this Academy.

Chapter One Distributed Scheduling System

First, the main role of distributed scheduling:

Use cloud computing like a desktop computer, combine the computing power of thousands of hardware, and provide reliable cloud computing services. In the desktop era, CPU is the most important part, and in cloud computing, distributed scheduling plays the role of CPU.

The main content of the first chapter:

1) Alibaba cloud Feitian distributed system

2) Task scheduling system

3) Resource scheduling

4) Fault tolerance mechanism

5) Scale challenge

6) Safety and performance isolation

7) Development direction of distributed scheduling

Section 1 Distributed scheduling system

Two tasks of distributed scheduling:

(1) Task scheduling: When different business departments share a cluster, there are a large number of computing tasks, how to divide the tasks, how to divide the massive data, perform operations on different nodes, and monitor the operation status of each node.

(2) Resource scheduling: The supply side of the supply side balances the use of resources between different business departments and needs to support priority preemption

Comparison of distributed scheduling systems:
MR master-slave architecture:

Jod Tracker is responsible for resource scheduling and task scheduling. When a client user submits a job to the resource manager, Job Traceker will distribute the job to different nodes. A task tracker named Task Tracker is set on each node to be responsible for the task. Scheduling and execution. This master-slave architecture has natural concentration defects:

1): There is a bottleneck in scale expansion: the continuous increase of nodes such as Task Tracker, in the case of Job Tracker, it is constantly adding memory modules, and there will be memory limit control.

2): Poor fault tolerance: Job Tracker is a single-node process. If Job Tracker crashes or the entire machine goes down, all the operating conditions and resource allocation results in all clusters cannot be recovered.

3): Not conducive to function expansion: different business departments have different scheduling strategies (different scheduling and usage strategies for resources), and also require support for hot plugging (changing the system scheduling without stopping the process).

(2) YARN (improvements made on the basis of Hadoop 1.0 version in response to the three shortcomings mentioned above are essentially a resource scheduler);
The biggest difference between YARN and Hadoop: the task scheduling and resource scheduling are distinguished, which can support a larger computing scale

However, YARN currently only supports resource scheduling in the memory dimension, and resource scheduling like disk, CPU, and network does not. Increased resource interaction links reduce YARN performance.

(3) Mesos;

(4) Alibaba cloud-Fuxi;