one. Yarn produces a background:
1. the problem with MapReduce 1.0: 1) Jobtracker performance problem, 2) Jobtracker single point problem, 3) only support MapReduce a computational framework
2. Resource utilization:
3. Operation and maintenance cost and data sharing:
operation and maintenance costIf you use the "one-frame-one-cluster" pattern, you may need tomultiple administrators to manage these clusters, thereby increasing operational costs,
Shared mode typically requires a small number of administrators to complete multiple frameworksthe Unified management
data sharing as data volumes increase, data movement across clusters takesand hardware costs will increase significantly while
sharing
cluster mode allows multiple frameworks to share data and hardware resources,large reduction in the cost of data movement
4. Summary:
two. Yarn Basic architecture:
Resourcemannager
- The entire cluster has only one (ha, backup), responsible for the unified management and scheduling of cluster resources
- Function:
- Handling client Requests
- Start/Monitor Applicationmaster
- Monitoring NodeManager
- Resource allocation and scheduling
NodeManager
- The entire cluster has multiple, responsible for single node resource management and use
- Function:
- Resource management and task management on a single node
- Handling commands from the ResourceManager
- Handling commands from the Applicationmaster
Applicationmaster
- Each application has one, responsible for the management of the application
- Function:
- Data segmentation
- Request resources for the application and further assign to internal tasks
- Task monitoring and fault tolerance
Container
- Abstraction of the task run environment
- Describe a range of information:
- Task Run resource (node, memory, CPU)
- Task Start command
- Task Run Environment
three. Yarn Running process:
From for notes (Wiz)
5. Yarn
