Linux Kernel load Balancing summary __linux

The Linux operating system supports SMP and also runs multiple processes, how they are scheduled across multiple CPUs, and how load is balanced.

The process decides on which CPU to run at the following time

1. When a process is awakened, such as when a process receives a network packet, or completes a disk IO operation

2. The process has just been fork out

Linux uses the SELECT_TASK_RQ function to determine which CPU the process is running on. This function is implemented in./kernel/sched/core.c.

int Select_task_rq (struct task_struct *p, int cpu, int sd_flags, int wake_flags)

Select_task_rq Time of Call:

Wake_up_new_task: The process has just been fork out

TRY_TO_WAKE_UP: When the process is awakened

The SELECT_TASK_RQ function takes into account the factors that determine which CPU the process is running on.

static int Select_task_rq_fair (struct task_struct *p, int prev_cpu, int sd_flag, int wake_flags)

{

int cpu = SMP_PROCESSOR_ID ();

For_each_domain (CPU, TMP) {
if (!) ( Tmp->flags & Sd_load_balance))
Break If the dispatch domain does not support sd_load_balance, exit directly.

/*
* If both CPU and PREV_CPU are part of this domain,
* CPU is a valid Sd_wake_affine target.
*/
if (Want_affine && (Tmp->flags & Sd_wake_affine) &&
Cpumask_test_cpu (PREV_CPU, Sched_domain_span (TMP))) {
AFFINE_SD = tmp;
Break
//Find the most recent dispatch domain where PREV_CPU and CPUs are in common

                if (Tmp->flags & Sd_flag)
      &NBSP ;                 SD = tmp;                                   &NB Sp      //According to Sd_flag find the Sd_flag maximum corresponding dispatch domain
                else if (!w Ant_affine)
                        break;    &N Bsp   

}


if (AFFINE_SD) {//If AFFINE_SD is found, the current CPU is used as the scheduled CPU
SD = NULL; /* Prefer wake_affine over balance flags * *
if (CPU!= prev_cpu && wake_affine (AFFINE_SD, p, sync))
NEW_CPU = CPU;
}

if (!SD) {//If no dispatch field is found that satisfies the sd_flags, the nearby empty Idle CPU.
if (Sd_flag & sd_balance_wake)/* XXX always? */
NEW_CPU = Select_idle_sibling (P, NEW_CPU);
else while (SD) {
struct Sched_group *group;
int weight;


if (!) ( Sd->flags & Sd_flag)) {
SD = sd->child;
Continue
}

Then find the least load CPU in SD as the Dispatch CPU

NEW_CPU = FIND_IDLEST_CPU (group, p, CPU);

}

}

The meaning of the Wake_flags parameter:

/*
* Wake Flags
*/
#define WF_SYNC 0x01/* Waker goes to sleep after wakeup * *
#define WF_FORK 0x02/* Wakeup after FORK * *
#define WF_MIGRATED 0x4/* Internal use, Task got migrated * *

The meaning of Sd_flag:

#define Sd_load_balance 0x0001/* do LOAD balancing in this domain. */
#define Sd_balance_newidle 0x0002/* BALANCE when about to become idle *
#define SD_BALANCE_EXEC 0x0004/* BALANCE on EXEC */
#define Sd_balance_fork 0x0008/* BALANCE on FORK, clone * *
#define Sd_balance_wake 0x0010/* BALANCE on Wakeup * *
#define Sd_wake_affine 0x0020/* WAKE task to waking CPU * *
#define Sd_prefer_local 0x0040/* Prefer to keep tasks local to this domain *
#define Sd_share_cpupower 0x0080/* Domain members SHARE CPU Power * *
#define Sd_powersavings_balance 0x0100/* BALANCE for power savings * *
#define Sd_share_pkg_resources 0x0200/* Domain members SHARE CPU PKG Resources * *
#define Sd_serialize 0x0400/* Only a single load balancing instance * *
#define Sd_asym_packing 0x0800/* Place busy groups earlier in the domain * *
#define Sd_prefer_sibling 0x1000/* prefer to place tasks in a sibling domain *

Meaning of sd_load_balance: if the corresponding dispatch domain supports sd_load_balance, then select the run queue again.

Sd_share_pkg_resources:select_idle_sibling whether to find out if the other CPUs in the Doman are idle when judging idle core.

Select_task_rq_fair need GDB to debug, look at the organizational structure of domain, find idle CPU when the time to stop.

In addition to the process being awakened or just fork done, what other mechanisms will cause the process to run on which CPU.

If the process can only wake up when the decision on which CPU, will inevitably lead to uneven CPU load, it is necessary to load balance.

Call timing for load balancing:

1. Periodic calls to ensure load balancing

OPEN_SOFTIRQ (SCHED_SOFTIRQ, run_rebalance_domains);

trigger_load_balance<-scheduler_tick<-periodic clock interrupts.

Triggers the call on each CPU through periodic scheduled interrupts.

There are two kinds of load balancing, one is the CPU idle trigger, the other is CPU busy trigger.

Rq->idle_balance = IDLE_CPU (CPU);
Trigger_load_balance (RQ);

Passed as a parameter to the load balancing function.

static void Rebalance_domains (struct RQ *rq, enum Cpu_idle_type idle)