Processor affinity for Advanced Process Management

Linux supports a single system with multiple processors. Except for the boot process, the process scheduling program is responsible for a large amount of work that supports multiple processors. In Symmetric Multi-processing (SMP) and above, the process scheduler must determine which processes are running on each CPU. Two challenges arise from this responsibility: the scheduler must make full use of all the processors in the system, because when one process is ready to run, one CPU is idle, this is obviously inefficient.

 

However, once a process is scheduled to run on a certain CPU, the process scheduler will schedule it to run on the same CPU. This is beneficial because migrating a process from one processor to another is costly.

 

These costs are related to the "cache effect. Due to the current SMP system design, the cache areas related to each processor are different and independent. That is to say, if the data is in the cache of one processor, it will not be in the cache of another processor. Therefore, if a process is migrated to a new CPU and new data is written to the memory, the data in the old CPU cache is outdated. If you use this cache area, the data will be damaged. To avoid this problem, when a new memory block is cached in a cache area, this will invalidate the data in each other cache area. Therefore, a specific piece of data can only appear in the cache of a processor at any time (assuming that all the data is cached ). When a process moves from one processor to another, it will pay two costs: the mobile process cannot access the cached data, in addition, data in the original processor cache must be voided. Because these costs must be paid, the process scheduler tries its best to assign a specific CPU for a process to run it.

 

Of course, the two goals of the Process scheduler may not be achieved. If one processor processes load much more than the other processor (or worse, if one processor is busy while the other processor is idle ), it is wise to reschedule some processes so that they can run on a less busy CPU. Determining when to move a process to respond to such imbalance is called load balancing, which is very important for SMP and its performance.

 

Processor affinity indicates the possibility that a process can be uniformly scheduled to run on the same processor. Soft affinity refers to the natural diligence that the scheduler continues to schedule a process to run on the same processor. The Linux scheduler will try to schedule the same process to run on the same processor, and will only move one process from one CPU to another when the load is extremely unbalanced. This allows the processor to minimize the cache effect of migration, but still ensures that the load of all processors in a system is balanced.

 

However, sometimes users or applications want to combine processes with processors. In this case, the process is highly sensitive to the cache and wants to stay on the same processor. Combining a process with a specific processor, the kernel acts in accordance with this relationship as a hard affinity (HAR affinity ).

 

Sched_getaffinity () and sched_setaffinity ()

A process inherits the CPU affinity of its parent process. By default, a process can run on any CPU. Linux provides two system calls that can be used to obtain and set the "hard affinity" of a process ":

 

# DEFINE _ gnu_source

# Include <sched. h>

Typedef struct cpu_set_t

Size_t cpu_set_size

 

Void cpu_set (unsigned long CPU, cpu_set_t * Set );

Void cpu_clr (unsigned long CPU, cpu_set_t * Set );

Int cpu_iiset (unsigned long CPU, cpu_set_t * Set );

Void cpu_zero (cpu_set_t * Set );

 

Int sched_setaffinity (pid_t PID, size_t setsize, const cpu_set_t * Set );

Int sched_getaffinity (pid_t PID, size_t setsize, cpu_set_t * Set );

 

Sched_getaffinity () can be used to obtain the CPU affinity of the Process PID and store it in a special cpu_set_t type, which can be accessed through a special macro. If the PID is 0, the call will get the affinity of the current process. The setsize parameter is the size of the cpu_set_t type. glibc may use this parameter to cope with future changes in this type of size. When the execution is successful, sched_getaffinity () returns 0; if the execution fails, it returns-1 and sets errno. See the following example:

 

 Cpu_set_t set; <br/> int ret, I; </P> <p> cpu_zero (& set); </P> <p> ret = sched_setaffinity (0, sizeof (cpu_set_t), & set); <br/> If (ret =-1) <br/>{< br/> perror ("sched_se "); <br/>}</P> <p> for (I = 0; I <cpu_setsize; I ++) <br/>{< br/> int CPU; <br/> CPU = cpu_isset (I, & set); <br/> printf ("CPU = % I is % s/n", I, <br/> CPU? "Set": "Unset"); <br/>}

 

Before calling, we will use cpu_zero to clear all bits in the set. Then we will iterate from 0 to cpu_setsize to process the set. Please note that cpu_setsize is not the set size. You should never pass it to setsize, but the number of processors that set may represent. Because the current implementation represents each processor in a single bit, cpu_setsize is much larger than sizeof (cpu_set_t. We will also use cpu_isset to check whether or not a specific processor I is bound to or bound to this process in the system. If 0 is returned, it indicates that it is not bound. If not 0 is returned, it indicates that it is bound.

 

An entity processor in the system will be set. Therefore, running the program code on a system with two processors will produce the following results:

 

CPU = 0 is set

CPU = 1 is set

CPU = 2 is unset

CPU = 3 is unset

...

CPU = 1023 is unset

 

All we care about is CPU #0 and CPU #1, because they are the only physical processor on this system. Maybe we want to ensure that our processes only operate on CPU #0, rather than on CPU #1. The following code completes this process:

 

Cpu_set_t set; <br/> int ret, I; </P> <p> cpu_zero (& set); <br/> cpu_set (0, & set ); <br/> cpu_clr (1, & set); </P> <p> ret = sched_setaffinity (0, sizeof (cpu_set_t), & set ); <br/> If (ret =-1) <br/> {<br/> perror ("sched_se "); <br/>}</P> <p> for (I = 0; I <3; I ++) <br/>{< br/> int CPU; <br/> CPU = cpu_isset (I, & set); <br/> printf ("CPU = % I is % s/n", I, <br/> CPU? "Set": "Unset"); <br/>}< br/>

 

As usual, we will first use cpu_zero to clear the set. Then we will use cpu_set to set CPU #0 and use cpu_clr to clear CPU #1. Cpu_clr operations are redundant, because we just cleared the entire set to ensure integrity.

 

Running this program on a system with two processors produces a slightly different output:

 

CPU = 0 is set

CPU = 1 is unset

CPU = 2 is unset

...

CPU = 1023 is unset

 

CPU #1 is cleared now. This process only runs on CPU #0.