Hadoop open-source scheduling system Zeus (2)

Keep up with the previous hadoop open-source scheduling system Zeus (1)

This section describes the architecture of Zeus:

First give a Zeus architecture Diagram

Both the master and worker have a set of web UIS. The results are the same regardless of the source. In fact, they only look at the master

MASTER: The scheduling kernel. It starts a TCP Service at startup, reads all tasks to the memory at the same time, adds the tasks to the running queue when the tasks can run, and delivers them to the client.

WORKER: connects to the master after startup, and regularly sends heartbeat to the Master. After receiving the master task, encapsulate the Shell Running the task and notify the master of the task running result.

Master single point: in Zeus, every node of the entire machine may become a master. The entire cluster is uniquely identified by a distributed lock. After the Master obtains the distributed lock, the Distributed Lock status will be updated regularly, and all workers will check the Distributed Lock status. If the master node finds that the lock status is not updated on time, each worker will compete to obtain the lock. After obtaining the lock, make yourself a master.

A worker that fails to obtain the lock will kill the job that is running and connect to the new master again. The Master will send the job that is running to the worker again.