Analysis of hadoop fair scheduler algorithm Scheduling

The following describes the hierarchical scheduling algorithm of the fair scheduler. The big idea is similar to capacity scheduler. First, select a pool and then select a job from the pool, finally, select a locality task from the job.

Among them, the pool and job policies are the same, both adopt the fair‑comparator comparator to sort the pool or job, and then scan the queue from start to end to select the appropriate pool or job.

Method for Selecting a pool: When we
Sort pools to choose which ones to schedule next, we place pools below their min share ahead of pools above their min share. we order the pools below their min share by how far they are below it as a percentage of the share. (sort by percentage lower than the minimum share amount)

Method for Selecting a job:
Comparator orders jobs by distance below min share and then by runningtasks/weight.

Deficit sorts the job deficit (the difference between the calculation time that the job deserves on the ideal scheduler and the actual calculation time), and tests in sequence, until the tasks that meet the conditions are selected.