Comparison between Fair Scheduler and capacity Scheduler

1, Fair Scheduler

Facebook has developed a scheduler for shared environments that supports multi-user multi-group management, each of which can configure the amount of resources, or limit the number of concurrent runs in each user and per group, and each user's job has priority, and the higher the priority, the more resources are allocated.

2, Capacity Scheduler

Yahoo developed a suitable for the shared environment of the scheduler, support multi-user multi-queue management, each queue can configure the amount of resources, can also limit the number of concurrent running jobs per user and per queue, or limit the amount of memory used by each job, each user's job has priority, in a single queue, The job is dispatched on a first-come-first-served basis (in fact, by priority, the same priority, and then by job submission time).

3, the same point

[1] All support multi-user multi-queue, that is applicable to multi-user shared cluster application environment

[2] Single queue support priority and FIFO scheduling mode

[3] All support resource sharing, that is, a queue of resources in the remaining, can be shared to other resource-scarce queue

4, different points

[1] The core scheduling strategy is different. The scheduling strategy of the computing power Scheduler is to select the queue with low resource utilization, and then consider the FIFO and memory constraint factors in the queue, while the Fair scheduler only considers fairness, and fairness is manifested through the job vacancy. The scheduler selects the job with the most vacancies (the amount of resources in the queue, the job priority, etc. only for the calculation of job vacancies)

[2] memory constraints. The compute Power Scheduler schedules the job to account for the memory limits of the jobs, which may be assigned to multiple slots in order to meet the special memory requirements of some special job, while the fair Scheduler can only kill the task if it is powerless against this particular job.

This article is from the "Wipe" blog, be sure to keep this source http://qishi23.blog.51cto.com/4820716/1599138

Comparison between Fair Scheduler and capacity Scheduler