Linux kernel: CPU private variable (per-cpu variable) __linux

Http://blog.chinaunix.net/uid-24148050-id-300576.html

Category: LINUX

First, Introduction

A new feature on the 2.6 kernel is the PER-CPU variable. As the name suggests, there is a copy of this variable on each processor.
The great advantage of PER-CPU is that access to it requires almost no locks, because each CPU works on its own copy.
Tasklet, timer_list and other mechanisms have used the PER-CPU technology.

Second, the API use

Note that the 2.6 kernel is preemptive.
So when accessing the PER-CPU variable, you should use a specific API to avoid preemption, that is, avoid switching to another CPU to be processed.

The PER-CPU variable can be declared at compile time, or it can be dynamically generated when the system is run

Example one:
Create a PER-CPU variable during compilation:
DEFINE_PER_CPU (INT,MY_PERCPU); Declaring a variable
DEFINE_PER_CPU (Int[3],my_percpu_array); Declaring an array

Using compile-time generated PER-CPU variables:
ptr = Get_cpu_var (MY_PERCPU); Using PTR
Put_cpu_var (MY_PERCPU); //

Of course, you can also use the following macros to access PER-CPU variables on a specific CPU
Per_cpu (MY_PERCPU, cpu_id); //

PER-CPU variables are exported for module use:
Export_per_cpu_symbol (Per_cpu_var);
EXPORT_PER_CPU_SYMBOL_GPL (Per_cpu_var);

Example two:
Dynamically allocating PER-CPU variables:
void *alloc_percpu (type);
void *__alloc_percpu (size_t size, size_t align);

To use dynamically generated PER-CPU variables:
int CPU;
CPU = GET_CPU ();
ptr = Per_cpu_ptr (MY_PERCPU);
Using PTR
Put_cpu ();
Third, realize

Why do you avoid preemption when using the API above, see the code implementation?

#define Get_cpu_var (Var) (* ({\
extern int simple_identifier_# #var (void); \
Preempt_disable (); \
&__get_cpu_var (VAR); }))
#define Put_cpu_var (Var) preempt_enable ()
#define GET_CPU () ({preempt_disable (); smp_processor_id ();}) #define PUT_CPU () preempt_enable ()
The key is to Preempt_disable and preempt_enable two calls, respectively, to prohibit preemption and open preemption
Grab the relevant stuff and see it later.

The introduction of the PER-CPU variable effectively solves the contention of the processor to lock in the SMP system, each CPU simply accesses its own local variables. This paper describes the implementation of the PER-CPU variable on the 2.6 kernel and the related operation. In the system compile phase we have manually defined a copy of all the PER-CPU variables defined by the macro define_per_cpu: one #define DEFINE_PER_CPU (type, name) \
__ATTRIBUTE__ ((__section__ (". Data.percpu")) __typeof__ (type) per_cpu__# #name

From the code above we can see that all of the PER-CPU variables that are manually defined are placed in the. DATA.PERCPU segment. Note that the macros above are defined only under the SMP architecture. If it's not an SMP-structured computer, then simply put all the PER-CPU variables where the global variables should be placed.

PER-CPU variable definition of single CPU: #else/*! SMP * *
28
#define DEFINE_PER_CPU (type, name) \
__typeof__ (type) per_cpu__# #name

After understanding the above code, we must also understand one point: the PER-CPU variable used in a single CPU computer is the PER-CPU variable that is placed in the global data area by the macro defined above. In the SMP architecture, we use variables that are not in the. DATA.PERCPU segment, and imagine which CPU to use if you use this variable. In fact, under SMP, each CPU uses copies of these PER-CPU variables in the. DATA.PERCPU segment, and several CPUs create several such replicas.

During system initialization, the Setup_per_cpu_areas () function is called in the Start_kernel () function to allocate space for each CPU's PER-CPU variable copy, noting that the alloc memory allocator is not yet established, and the function calls the ALLOC_ The BOOTMEM function assigns physical space to these variable copies during initialization. 332 static void __init setup_per_cpu_areas (void)
/* */
333 {
334 unsigned long size, I;
335 char *ptr;
336
337/* Copy section for each CPU (we discard the original) * *
338 size = ALIGN (__per_cpu_end-__per_cpu_start, smp_cache_bytes);
339 #ifdef Config_modules
(Size < Percpu_enough_room)
341 size = Percpu_enough_room;
342 #endif
343
344 ptr = alloc_bootmem (size * Nr_cpus);
345
346 for (i = 0; i < Nr_cpus; i++, ptr + = size) {
347 __per_cpu_offset[i] = ptr-__per_cpu_start;
348 memcpy (PTR, __per_cpu_start, __per_cpu_end-__per_cpu_start);
349}
350}
351 #endif/*!__GENERIC_PER_CPU * *

The above function, while allocating the physical space occupied by the PER-CPU variable copy of each CPU, also initializes the __per_cpu_offset[nr_cpus] array for subsequent copies of these PER-CPU variables for the specified CPU.

#define PER_CPU (Var, cpu) (*reloc_hide (&per_cpu__# #var, __PER_CPU_OFFSET[CPU))
#define __get_cpu_var (Var) per_cpu (Var, smp_processor_id ()) These two macros are used to obtain the PER-CPU variable for the specified CPU and the other PER-CPU variable for the local CPU being obtained. You can analyze it yourself.