An overview:
-
(1) Hadoop MapReduce uses the master/slave structure. *master: The entire cluster is the only global manager, functions include: job management, status monitoring and task scheduling, namely, the Jobtracker in MapReduce. *slave: Responsible for the execution of the task and the return of the task status, that is, the Tasktracker in MapReduce.
Two Jobtracker analysis:
-
(1) Overview: Jobtracker is a background service process, after startup, will always listen and receive the
heartbeat information sent from each tasktracker, including the
use of resources and
task running condition, etc. Information. (2) Main functions of Jobtracker: 1.
Job Control : in Hadoop each application is represented as a job, each job is divided into multiple tasks, and Jobtracker's job control module is responsible for the decomposition and status monitoring of the job. * The most important is the status monitoring: mainly including tasktracker condition monitoring, job status monitoring and task status monitoring. Main role: Fault tolerance and provide decision-making basis for task scheduling. 2.
Resource Management .
Three Tasktracker Analysis:
-
(1) Tasktracker Overview: Tasktracker is a bridge between Jobtracker and task: On the one hand, receives and executes various commands from Jobtracker: Running tasks, submitting tasks, killing tasks, etc. on the other hand, the local node The status of each task is reported periodically to jobtracker through the
heartbeat . The
RPC protocol is used to communicate between the Tasktracker and the Jobtracker and the task. (2) Features of Tasktracker: 1.
reporting Heartbeat : Tracker periodically reports various information on all nodes through the heartbeat mechanism to jobtracker. This information includes two parts: * Machine-level information: node health, resource usage, etc. * Task Level Information: Task execution progress, task run status, and so on. 2.
Execute command : Jobtracker will give Tasktracker a variety of commands, mainly including: Start Task (launchtaskaction), submit task (committaskaction), kill the Task ( killtaskaction), kill Job (Killjobaction), and Reinitialize (tasktrackerreinitaction).
-
(ext.) Jobtracker and Tasktracker overview
