Analysis on the implementation of Linux soft interrupt

Reference:

Http://bbs.chinaunix.net/thread-2333484-1-1.html

http://liu1227787871.blog.163.com/blog/static/20536319720129210112658/

1. Soft Interrupt

In general, the process of a single interrupt service can usually be divided into two parts. The beginning of the section often must be executed under the condition of the shutdown, in order to perform some key operations "atom" under undisturbed conditions, while this part of the operation is often very time-critical, must be executed immediately after the interruption request or at least within a certain time limit, And successive interrupts can not be combined to deal with the request. The second half, usually can and should be executed under the condition of interrupting, so as not to cause the loss of other interrupts due to the interruption of the shutdown too long, at the same time, these operations often allow delay until later to execute, and it is possible to combine the related parts of multiple interrupts together. These different properties often differentiate between the two halves of the interrupted service, which can and should be implemented differently. The latter part of the section is called "Bottom half", which is often written in the kernel code as a BF, and this part of the BF can be implemented by software interrupts. Because the activation of software interrupts is implemented through code, not hardware, it is possible to determine the timing of activation by itself or by the system!

1.1 Registration
Or the first of the two old friends I know best:


OPEN_SOFTIRQ (NET_TX_SOFTIRQ, net_tx_action);
OPEN_SOFTIRQ (NET_RX_SOFTIRQ, net_rx_action);


OPEN_SOFTIRQ registers a soft interrupt to the kernel, which essentially sets the corresponding slot of the soft interrupt vector table and registers its handler function:

void Open_softirq (int nr, void (*action) (struct softirq_action *)) {        softirq_vec[nr].action = action;}

The Softirq_vec is the entire soft interrupt vector table:

void Open_softirq (int nr, void (*action) (struct softirq_action*), void *data)  {      softirq_vec[nr].data = data;   Softirq_vec[nr].action = action;}; static struct softirq_action Softirq_vec[nr_softirqs] __cacheline_aligned_in_smp;

Nr_softirqs is the maximum number of soft interrupt vectors, and all the soft interrupts supported by the kernel are as follows:

enum{        hi_softirq=0,        Timer_softirq,        Net_tx_softirq,        Net_rx_softirq,        Block_softirq,        TASKLET_SOFTIRQ,        Sched_softirq,        Hrtimer_softirq,        RCU_SOFTIRQ,/        * Preferable RCU should always be the Last SOFTIRQ */        Nr_softirqs};

It seems to be a new one for RPS, but my kernel version is low.


1.2 Activation


When a soft interrupt needs to be called, call the RAISE_SOFTIRQ function to activate the soft interrupt, which uses the term "activate" rather than "call"
is because in many cases a soft interrupt cannot be called directly. So it can only be quickly flagged as "executable", waiting for a future call.
Why "soft interrupts cannot be called directly in many cases"? Imagine the idea that the lower half is introduced to make the top half more executed faster.
If the soft interrupt function is called directly in the interrupt program code, the difference between the upper and lower halves is lost, meaning that it loses its existence.


The kernel uses a bitmap named __softirq_pending to describe soft interrupts, each of which corresponds to a soft interrupt, and the bitmap is contained in the structure Irq_stat:

typedef struct {        unsigned int __softirq_pending;        ......} ____cacheline_aligned irq_cpustat_t;declare_per_cpu_shared_aligned (irq_cpustat_t, Irq_stat);

The macro or_softirq_pending is used to set the appropriate bit (bit or operation):

#define OR_SOFTIRQ_PENDING (x)        percpu_or (irq_stat.__softirq_pending, (x))

Local_softirq_pending is used to get the entire bitmap (not one):

#define Local_softirq_pending ()        percpu_read (irq_stat.__softirq_pending)

Macro __raise_softirq_irqoff is a parcel of or_softirq_pending:

#define __raise_softirq_irqoff (NR) do {or_softirq_pending (1UL << (NR)),} while (0)

Raise_softirq_irqoff activates the soft interrupt by invoking the __raise_softirq_irqoff, and its parameter nr is a soft interrupt corresponding to the bitmap slot:

/** This function must run with IRQs disabled!*/inline void Raise_softirq_irqoff (unsigned int nr) {//bitmap, which is marked as executable state        __raise_softirq_irqoff (NR); /* * If we ' re in an interrupt or SOFTIRQ, we ' re do * (this also catches softirq-disabled code).         We'll * actually run the SOFTIRQ once we return from * the IRQ or SOFTIRQ.         * * Otherwise We wake up ksoftirqd to make sure we * schedule the SOFTIRQ soon.        *///Set the bitmap, you can determine whether it is not in the context of the interrupt, if not, it is a good time to call the soft interrupt immediately.        In_interrupt another function is to determine whether a soft interrupt is disabled.        Wakeup_softirqd wakes the daemon Ksoftirq for soft interrupts. if (!in_interrupt ()) wakeup_softirqd ();}        The copy code can now look at all the meanings of the "activating" soft interrupt, the RAISE_SOFTIRQ function accomplishes this: void Raise_softirq (unsigned int nr) {unsigned long flags;        For all operations, the interrupt should be closed to avoid nesting calls to local_irq_save (flags);        Raise_softirq_irqoff (NR); Local_irq_restore (flags);}

Visible, the active operation, mainly two points:
<1>, the most important thing is to set the corresponding bitmap, waiting for the future to be processed;
<2>, if it is not in the interrupt context at this point, call immediately (actually the wake operation of the kernel thread), now is the future;


2. Timing of Dispatch
Yes, in addition to RAISE_SOFTIRQ, it is possible (well, importantly "probably") to awaken KSOFTIRQD through WAKEUP_SOFTIRQD, and also to understand the timing of other calls to soft interrupts.


A. Call Irq_exit when DO_IRQ completes an I/O interrupt:

#ifdef __arch_irq_exit_irqs_disabled# define INVOKE_SOFTIRQ ()        __do_softirq () #else # define INVOKE_SOFTIRQ ()        DO_SOFTIRQ () #endifvoid irq_exit (void) {        account_system_vtime (current);        Trace_hardirq_exit ();        Sub_preempt_count (irq_exit_offset);        if (!in_interrupt () && local_softirq_pending ())                Invoke_softirq ();                Call Soft Interrupt

B, if the system uses I/O APIC, when the local clock interrupt is processed:

void __irq_entry smp_apic_timer_interrupt (struct Pt_regs *regs) {        ...        Irq_exit ();        ......}

C, local_bh_enable

Local_bh_enable is to open the lower half, of course, in the middle of the soft interrupt: void local_bh_enable (void) {        _local_bh_enable_ip ((unsigned long) __builtin _return_address (0));} static inline void _local_bh_enable_ip (unsigned long IP) {        ...        if (Unlikely (!in_interrupt () && local_softirq_pending ())                Do_softirq ();        ......}

D, in SMP, when the CPU has finished processing functions triggered by interrupts between Call_function_vector processors:
Well, the communication between CPUs in Multicore is not familiar, and this mechanism is not very clear ...

3, DO_SOFTIRQ


Regardless of the invocation method, will eventually trigger the soft interrupt core handler function DO_SOFTIRQ, which handles all soft interrupts on the current CPU. The
kernel makes the soft interrupt design as platform-independent as possible, but in some cases there is still a difference, first of all, a x86 32-bit version of DO_SOFTIRQ:

asmlinkage void Do_softirq (void) {unsigned long flags;        struct Thread_info *curctx;        Union Irq_ctx *irqctx;        U32 *isp; Soft interrupts cannot be nested within the interrupt context.        The interrupt handler or the lower half uses the "activate" mode.        if (In_interrupt ()) return;        Disable interrupt, save interrupt flag local_irq_save (flags); The kernel uses a CPU bitmap, and indeed several soft interrupts can be run on different CPUs at the same time, including the same soft interrupts.        For example,//NET_RX_SOFTIRQ can run on multiple processors at the same time. The local_softirq_pending is used to determine whether all bitmaps for the current CPU are set.        That is, whether there is a soft interrupt waiting to be processed.        Recall the frequent network card receiving data processing: When the network card interrupt on which CPU, the corresponding soft interrupt function will be executed on it. From here, the essence is which NIC interrupt falls on the corresponding CPU, the CPU set its soft interrupt bitmap, here to do the corresponding detection (here local_softirq_pending only//is a total judgment, the back of the judgment of the bit), detected there is a corresponding bit, the implementation of I                F (local_softirq_pending ()) {//get thread Descriptor Curctx = Current_thread_info ();                Constructs the interrupt context structure, SOFTIRQ_CTX is the soft interrupt context for each CPU//static DEFINE_PER_CPU (Union Irq_ctx *, SOFTIRQ_CTX);                This first obtains the current CPU's soft interrupt context, then assigns it the initial value-saving the current process and the stack pointer irqctx = __get_cpu_var (SOFTIRQ_CTX); Irqctx->tiNfo.task = curctx->task;                Irqctx->tinfo.previous_esp = Current_stack_pointer; /* Build the stack frame on the SOFTIRQ stack *//construct interrupt stack frame ISP = (U32 *) ((char *) Irqctx + si                Zeof (*IRQCTX));                Call_on_stack switches the kernel stack and executes the function __do_softirq Call_on_stack (__DO_SOFTIRQ, ISP) on the interrupt context; /* * shouldnt happen, we returned above if In_interrupt (): */Warn_on_once (        Softirq_count ()); }//Recovery of local_irq_restore (flags);}

When Config_4kstacks is configured, the thread_union of each process is only 4 K, not 8K. When an outage occurs, the kernel stack does not use the kernel stack of the process, and the interrupt request stack for each CPU is used.
The kernel stack uses the interrupt request stack for each CPU, not the kernel stack of the process to perform the soft interrupt function:

static void Call_on_stack (void *func, void *stack) {        asm volatile ("Xchgl        %%ebx,%%esp        \ n"                                //Exchange stack pointer, The pointer stack that interrupts the stack frame is passed as an incoming parameter (%EBX), after which the ESP is the top of the Irq_ctx, EBX is the stack of the process kernel stack                     "                call *%%edi \ n"                                        //calls the soft interrupt function                     "MOVL        %%ebx,%%esp        \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ r \ n '                                        movl, not xchgl because the function is finished, the broken stack frame                     Stack),                       "D" (func)                     : "Memory", "CC", "edx", "ecx", "eax");}

PS: All of these executions should be based on the definition of the 4K stack:

#ifdef config_4kstacks/** per-cpu IRQ handling contexts (thread information and stack) */union irq_ctx {        struct thread_ Info      tinfo;        U32                     stack[thread_size/sizeof (u32)]; __attribute__ ((Aligned (page_size))); Static DEFINE_PER_CPU (Union Irq_ctx * , hardirq_ctx), Static DEFINE_PER_CPU (Union Irq_ctx *, SOFTIRQ_CTX), ... static void Call_on_stack (void *func, void *stack) ... ...

Yes, this version is relatively complex, but if you look at the complex, and then look at the simple, it is much easier, when the platform does not define the DO_SOFTIRQ function (__ARCH_HAS_DO_SOFTIRQ),
The kernel provides a common:

#ifndef __arch_has_do_softirqasmlinkage void Do_softirq (void) {        __u32 pending;        unsigned long flags;        if (In_interrupt ())                return;        Local_irq_save (flags);        Pending = Local_softirq_pending ();        if (pending)                __do_softirq ();        Local_irq_restore (flags);} #endif

No more explanation, it's very concise.


Regardless of the version, the __DO_SOFTIRQ function is called:

asmlinkage void __do_softirq (void) {struct softirq_action *h;        __U32 pending;        int max_restart = Max_softirq_restart;        int CPU;        Save Bitmap pending = Local_softirq_pending ();        Process Accounting Account_system_vtime (current); Close the lower half of the local CPU.        In order to ensure that a soft interrupt on the same CPU is executed serially.        __local_bh_disable ((unsigned long) __builtin_return_address (0));        Lockdep_softirq_enter (); Get local CPU CPU = SMP_PROCESSOR_ID (); restart:/* Reset the pending bitmask before enabling IRQs *//clear        Bitmap set_softirq_pending (0); The lock is interrupted just to keep the bitmap mutually exclusive and the bitmap is processed.        The following code can use the saved pending directly,//and the interrupt handler can safely use irq_stat.__softirq_pending when activated.        Therefore, can be opened interrupted local_irq_enable ();        Get soft Interrupt vector h = softirq_vec; Loop through all the soft interrupts do {//step through each bit of the bitmap and determine if there is a soft interrupt set on that bit. If so, deal with if (pending & 1) {//save preemption counter int prev_count = Pree                    Mpt_count ();    KSTAT_INCR_SOFTIRQS_THIS_CPU (H-softirq_vec);                        Trace_softirq_entry (H, Softirq_vec);                        Call Soft interrupt h->action (h);                        Trace_softirq_exit (H, Softirq_vec);                                Determine if the soft interrupt is preempted, and if so, output an error message if (unlikely (prev_count! = Preempt_count ())) {                                       PRINTK (Kern_err "Huh, entered Softirq%td%s%p" "With Preempt_count%08x," "Exited with%08x?\n", H-softirq_vec, SOFTIRQ                                _to_name[h-softirq_vec], h->action, Prev_count, Preempt_count ());                        Preempt_count () = Prev_count; } //？？                QSCTR, this is what East Rcu_bh_qsctr_inc (CPU);                }//point to the next soft-interrupt slot h++; Shift, remove aSoft interrupt bit pending >>= 1;        } while (pending);        When the soft interrupt processing is completed, because the front has been interrupted, it is possible that a new soft interrupt has been set up,//Soft interrupt Scheduler will try to re-soft interrupt, its maximum number of restarts determined by Max_restart.        So, here we have to shut down the interrupt again, one more time ... local_irq_disable ();        Take bitmap pending = Local_softirq_pending ();        There is a soft interrupt set, and no more than the maximum number of restarts, one more first if (pending &&--max_restart) goto restart; More than the maximum number of restarts, there are soft interrupts to be processed, call WAKEUP_SOFTIRQD.        It is either a wake-up soft interrupt daemon ksoftirqd.        if (pending) Wakeup_softirqd ();        Lockdep_softirq_exit ();        Account_system_vtime (current); Restore the lower half of _local_bh_enable ();}

Interrupt Tracking
If interrupt tracking config_trace_irqflags is defined, Lockdep_softirq_enter/lockdep_softirq_exit is used to increment/decrement the current process's soft interrupt context counter Softirq_context:
# define Lockdep_softirq_enter () do {current->softirq_context++;} while (0)
# define LOCKDEP_SOFTIRQ_EXIT () do {current->softirq_context--;} while (0)
Copy Code
The trace_softirq_entry is used in conjunction with Trace_softirq_exit to determine the delay of a soft interrupt.


It seems that the soft break is not too difficult, there is no more content. Welcome everyone to add Huitie.

Analysis on the implementation of Linux soft interrupt