Introduction to the design idea and functional components of yarn

The design idea of yarn

A Yarn (yet Another Resource negotiator)
B. The basic idea of yarn is
    to separate the main functions of jobtracker into separate components, a global ResourceManager corresponding to each application Applicationmaster

Hadoop1. X and hadoop2.x frame contrast diagram:

Hadoop2. X Frame Chart:
Yarn Components: A. ResourceManager

A A pure scheduler
B) strictly limits the system's available resources according to the application's resource requirements
C) Optimize the utilization of cluster resources and make full use of all resources to ensure capacity, fairness and service levels
By the pluggable scheduler to apply different scheduling algorithms, such as pay attention to capacity scheduling or pay attention to fair scheduling

B. Applicationmaster

(a) is responsible for negotiating resources with ResourceManager and working with NodeManager to implement and monitor container and their resource consumption
B) Have the responsibility to negotiate with ResourceManager and get the right resources container, track their status, and monitor their progress
C) in real environments, each application has its own Applicationmaster instance, But it can also provide a applicationmaster for a group, such as pig or hive applicationmaster.

C. Resource Model

A) yarn provides a generic Application resource model
B) An application can request very specific resources via Applicationmaster
    I. Resource names (including host name, rack name, and complex network topology)
    Ii. memory
    Iii. CPU
    IV Other resources: IO for hard disk and network

D. Resourcerequest and Container

A an application Applicationmaster request specific resourcerequest to meet resource requirements
B Scheduler assigns a container to respond to resource requests
C) Container is a form of resource allocation that gives the application the right to use resources (such as memory, CPU) on a particular host
D) Applicationmaster take container and give it to NodeManager, NodeManager will use the appropriate resources to start the container task process