Linux kernel Debugging technology--jprobe use and implementation

The previous post introduces the principle of kprobes and the use and implementation of Kprobe, this paper introduces the second detection technology in Kprobes Jprobe, which is based on kprobe implementation, can not be inserted at any point in the function of the probe, only at the entrance of the function to detect, Typically used to monitor the parameter values of a function. This article first introduces a simple example of how jprobe is used, and then analyzes the implementation process of jprobe in detail through the source code.

Kernel Source: linux-4.1.x

Experimental environment: FEDORA25 (x86_64), Raspberry Pi 1b

1. Jprobe Use Example

The kernel module needs to be written using the Jprobe probe function's entry parameter value. Like Kprobe, the kernel also provides the Jprobe instance program jprobe_example.c (located in the Sample/kprobes directory), which implements the function of detecting do_fork function entry. The user can use it as a template to detect other functions (not to say what functions can be detected, and to limit the same as kprobe, it is also important to note that a probe function can only register a jprobe). Familiarize yourself with the basic structure and API interface of Jprobe before analyzing jprobe_example.c.

1.1, Jprobe structure and API Introduction

The struct JPROBE structure is defined as follows:

/* * Special probe type that uses setjmp-longjmp type tricks to resume * Execution at a specified entry with a matching PR Ototype corresponding * to the probed function-a trick to enable arguments to become * accessible seamlessly by probe ha Ndling logic. * Note: * Because of the compilers allocate stack space for local variables * etc upfront, regardless of sub-scopes WI Thin a function, this mirroring * principle currently works with probes placed on function entry points. */struct jprobe {struct kprobe kp;void *entry;/* probe handling code to jump to */};

The structure is very simple, contains only a kprobe structure (because it is based on kprobe implementation) and a entry pointer, which holds the address of the probe execution callback function, when the invocation of the probe function is triggered, the address saved to the pointer will be executed as the destination address (probe Handling code to jump to), so the user-specified probe function is executed.

The relevant APIs are as follows:

int register_jprobe (struct jprobe *jp)      //to the kernel Register jprobe probe point  void Unregister_jprobe (struct jprobe *jp)   // Unload Jprobe probe Point  int register_jprobes (struct jprobe **jps, int num)     //register probe function vector with multiple different probe points  void Unregister_ Jprobes (struct jprobe **jps, int num)  //unload probe function vector, containing multiple different probe points  int disable_jprobe (struct jprobe *jp)       // Temporarily pauses the probe  int enable_jprobe (struct jprobe *jp) of the specified probe point        //Restores the probe for the specified probe point

1.2. Example Jprobe_example analysis and demonstration

As with KPROBE_EXAMPLE.C, the sample program still probes with Do_fork as the probe function. When the process is created, the probe function calls it to print out the parameter values of the Do_fork function. The following detailed analysis:

static struct Jprobe My_jprobe = {. entry= jdo_fork,.kp = {. Symbol_name= "do_fork",},};static int __init jprobe_init (void) { int ret;ret = Register_jprobe (&my_jprobe), if (Ret < 0) {PRINTK (kern_info "Register_jprobe failed, returned%d\n", R ET); return-1;} PRINTK (kern_info "planted jprobe at%p, handler addr%p\n",       my_jprobe.kp.addr, my_jprobe.entry); return 0;} static void __exit jprobe_exit (void) {Unregister_jprobe (&my_jprobe);p rintk (kern_info "Jprobe at%p unregistered\n" , my_jprobe.kp.addr);}

The program defines a struct Jprobe instance my_jprobe, which specifies that the name of the probed function is do_fork (which can be modified to detect other functions), and then detects the callback function as Jdo_fork. In the initialization function of the module, call the Register_jprobe function to register the My_jprobe with the Kprobe subsystem so that the Jprobe probe is enabled by default, and finally the Unregister_jprobe function is called in the Exit function to unload.

/* Proxy routine having the same arguments as actual do_fork () routine */static long jdo_fork (unsigned long clone_flags, u nsigned long Stack_start,      unsigned long stack_size, int __user *parent_tidptr,      int __user *child_tidptr) {pr_ info ("jprobe:clone_flags = 0x%lx, Stack_start = 0X%LX" "Stack_size = 0x%lx\n", Clone_flags, Stack_start, stack_size);/* A Lways end with a call to Jprobe_return (). */jprobe_return (); return 0;}

The Jdo_fork function simply prints out the three input parameters of the Clone_flags, Stack_start, and stack_size that were passed in when the Do_fork function was called, and the entire implementation was straightforward, but there were two points to note:

1) The parameters of the detection callback function must be consistent with the probe function, otherwise the purpose of the probe function can not be reached, such as the jdo_fork function here unsigned long clone_flags, unsigned long stack_start, unsigned long stack_size, int __user *parent_tidptr, and int __user *child_tidptr are exactly the same as the Do_fork function (note that the return value is fixed to a long type).

2) After the callback function is finished, you must call the Jprobe_return function (also highlighted in the note), or the execution process will not return to the normal execution process, which will be analyzed in detail later.

The following shows the actual effect of the program in the X86_64 environment (please refer to the previous blog post for Environment configuration):

<6>[15817.544375] Jprobe:clone_flags = 0x1200011, Stack_start = 0x0 Stack_size = 0x0
<6>[15817.551217] Jprobe:clone_flags = 0x1200011, Stack_start = 0x0 Stack_size = 0x0
<6>[15817.905328] Jprobe:clone_flags = 0x1200011, Stack_start = 0x0 Stack_size = 0x0
<6>[15822.684688] Jprobe:clone_flags = 0x1200011, Stack_start = 0x0 Stack_size = 0x0
<6>[15822.704001] Jprobe:clone_flags = 0x1200011, Stack_start = 0x0 Stack_size = 0x0

After loading the Jprobe_example.ko module, at the terminal casually knock a few commands to trigger the process to create, the kernel print out the above message, you can see the do_fork of the entry is very easy to obtain, and other functions of the probe is similar, no longer described in detail.

2, Jprobe realization Analysis

The implementation of Jpeobe is based on kprobe, so this will be in the previous post "Linux kernel debugging technology--kprobe use and implementation" based on the analysis of its implementation, including the Jprobe registration process and trigger the detection process, the part of the Kprobe is no longer detailed description.

2.1, Jprobe realization principle

The use of Kprobe,jprobe is a special form of Kprobe, which has its own pre_handler and Break_handler callback functions, of which pre_ The handler callback function is responsible for saving the original call context and preparing the environment for invoking the user-specified probe function jprobe->entry, and then jumps to jprobe->entry execution (the input parameter information for the probed function gets output here), The kprobe process is then triggered again, the original context is restored in the Break_handler function, and the normal execution process is returned.

2.2. Register a Jprobe instance

The Jprobe probe module calls the Register_jprobe function to register a Jprobe instance with the kernel, the code path KERNEL/KPROBES.C, and its main processes such as:

Figure 1 Jpobe Registration process

int register_jprobe (struct jprobe *jp) {return Register_jprobes (&JP, 1);} EXPORT_SYMBOL_GPL (Register_jprobe);

The Register_jprobe function is just a package of register_jprobes, and the primary registration function is done by the Register_jprobes function.

int Register _jprobes (struct jprobe **jps, int num) {struct Jprobe *jp;int ret = 0, i;if (num <= 0) return-einval;for (i = 0; i < Num i++) {unsigned long addr, offset;jp = Jps[i];addr = Arch_deref_entry_point (jp->entry);/* Verify Probepoint is a functio  N Entry point */if (Kallsyms_lookup_size_offset (addr, NULL, &offset) && offset = = 0) {Jp->kp.pre_handler = Setjmp_pre_handler;jp->kp.break_handler = Longjmp_break_handler;ret = Register_kprobe (&JP->KP);} Elseret =-einval;if (Ret < 0) {if (i > 0) unregister_jprobes (JPS, i); break;}} return ret;} EXPORT_SYMBOL_GPL (register_jprobes); 

The function is a loop that performs the same registration process for each jprobe, first removing the address of the probe callback function from the jp->entry and validating it. The role of the Kallsyms_lookup_size_offset function is to find the symbol of the addr address from the symbol table of the kernel or module, and when found, it returns the offset from the beginning of the symbol with the offset value of 0, which must be the entry of a function. If the conditions are met, set the Pre_handler and Break_handler of the Kprobe and the two callback functions Setjmp_pre_handler and Longjmp_break_handler, and finally call Register_ The Kprobe function registers the kprobe.

Visible Jprobe The registration process is very simple, its essence is to register a kprobe, using kprobe mechanism to implement detection, but the detection callback function is not the user's own definition, using jprobe private. After the registration is complete, the Jprobe (kprobe) mechanism starts and the Jprobe (Kprobe) probe is triggered when the function call process executes to the probed function.

Finally, it is important to note that Jprobe is not able to register multiple at the same detected point, and in the Kprobe registration process Register_kprobe->register_aggr_kprobe->add_new_kprobe will be judged:

if (P->break_handler) {if (Ap->break_handler) return-eexist;

2.3. Trigger Jprobe Detection

Based on the kprobe mechanism, after executing to the probe function, the CPU exception is triggered, according to the Kprobe execution process, the Kprobe_handler function is called to the Pre_handler callback function, namely Setjmp_pre_handler. The function schema is related, it performs some stack or register related operations according to the different schemas, saves the scene to recover after the call is finished, then jumps to the user-determined jprobe->entry, and after printing the information the user needs, return to the original normal process to continue execution. Main processes such as:

Figure 2 Jprobe Trigger process

2.3.1, ARM architecture implementation

int __kprobes setjmp_pre_handler (struct kprobe *p, struct pt_regs *regs) {struct Jprobe *jp = container_of (p, struct jprobe , KP); struct kprobe_ctlblk *KCB = get_kprobe_ctlblk (); Long sp_addr = Regs->arm_sp;long cpsr;kcb->jprobe_saved_ Regs = *regs;memcpy (Kcb->jprobes_stack, (void *) sp_addr, min_stack_size (SP_ADDR)); regs->arm_pc = (long) jp-> ENTRY;CPSR = REGS->ARM_CPSR | psr_i_bit; #ifdef config_thumb2_kernel/* Set correct Thumb state in CPSR */if (regs->arm_pc & 1) cpsr |= Psr_t_bit;el SECPSR &= ~psr_t_bit; #endifregs->arm_cpsr = Cpsr;preempt_disable (); return 1;}

First, it is clear again that the meaning of the Pt_regs *regs is the register value of the normal execution stream context that was saved before the CPU exception was triggered. The function first obtains the JPROBE structure instance that is triggered, and calls Get_kprobe_ctlblk to obtain the current KPROBE_CTLBLK structure global variable of the CPU, the struct KPROBE_CTLBLK structure definition has been seen in the Kprobe analysis. However, Jprobe uses the other two fields defined therein:

/* per-cpu kprobe control block */struct kprobe_ctlblk {unsigned int kprobe_status;struct prev_kprobe prev_kprobe;struct p T_regs Jprobe_saved_regs;char jprobes_stack[max_stack_size];};

Where Jprobe_saved_regs is used to hold the register information, Jprobes_stack is used to hold the stack information, which is used to restore the context of the pre-probe when the Jprobe returns, as can be seen from the first two lines of the Setjmp_pre_handler function. Ask a question first, why kprobe need not save the original context information and jprobe need?

The function next modifies the incoming ARM_PC value to the user-specified probe callback function address, noting that this value is intended to be set as the next instruction in the normal process in the normal kprobe process (after the Kprobe process is executed, it will go back to the original process to continue), This will not go back to the original process at the end of the kprobe process, but will go into the user-specified probe function execution.

The function then modifies the CPSR register value of the incoming parameter, psr_i_bit, which disables the interrupt, and finally disables preemption and returns 1. Back to the Kprobe_handler function, after returning 1, the next kprobe will not execute SingleStep and call Post_handler callback function, note that the Reset_current_ will not be called. The Kprobe function resets the currently executing kprobe to null:

if (!p->pre_handler | |!p->pre_handler (P, regs)) {kcb->kprobe_status = Kprobe_hit_ss;singlestep (p, regs, KCB); if (p->post_handler) {kcb->kprobe_status = Kprobe_hit_ssdone;p->post_handler (p, regs, 0);} Reset_current_kprobe ();}

After the Kprobe_handler process returns, the execution process enters the user-specified probe function execution, which is the Jdo_fork function for the Jprobe_example program in the previous article. In the second question, how is the parameter value of the probe function obtained?

As can be seen from the implementation of the Setjmp_pre_handler, the function only modifies the return address of the kprobe, and does not modify the stack and other register values, so when the CPU jumps to jdo_fork execution, its register and the contents of the stack are the same as the original call Do_ The fork function is almost identical (only the interrupt is disabled), so whether it is passed through the register or by means of a stack, the user needs to define the Jdo_fork function just as the same do_fork as the function in the parameter definition can easily get to the original parameter value. Another message from the implementation here is that the Jprobe callback execution context is the same as the context in which the original function was executed, unlike the context in which the Kprobe,kprobe callback function executes in the interrupt context of the CPU exception.

Finally, because the probe function (jdo_fork) is executed after the completion of the Kprobe_handler process execution, skipping the single_step process, it is said that it can not use the original kprobe mechanism to go back to the original execution process to execute, need to think of another method, In fact, the register stored in the Setjmp_pre_handler function Pt_regs is used for this purpose, but also explains the first question raised in the previous article, followed by a detailed analysis.

Back to the probe function Jdo_fork, the user obtains the required information, then enters the field recovery process, the key part is the Jdo_fork function last Call Jprobe_return function, it is implemented by the embedded assembly

void __kprobes Jprobe_return (void) {struct KPROBE_CTLBLK *KCB = get_kprobe_ctlblk (); __asm__ __volatile__ (/* * Setup an EMP Ty Pt_regs. Fill SP and PC fields as * they ' re needed by Longjmp_break_handler. * * We Allocate some slack between the original SP and start of * our fabricated regs. To being precise we want to having worst case * covered which are stmfd with all regs so we allocate 2 * * sizeof (Struct_pt_r EGS)). * * This was to prevent any simulated instruction from writing * over the regs when they was accessing the stack.    */#ifdef Config_thumb2_kernel #else "Sub sp,%0,%1\n\t" #endif "ldr r0, =" __stringify (jprobe_magic_addr) "\n\t" "Str  %0, [sp,%2]\n\t "" Str r0, [sp,%3]\n\t "" mov r0, sp\n\t "" BL kprobe_handler\n\t "/* * Return to the context saved by Setjmp_pre_handler * and restored by Longjmp_break_handler. */#ifdef Config_thumb2_kernel #else "Ldrr0, [sp,%4]\n\t" "msrcpsr_cxsf, r0\n\t" "ldmiasp, {r0-pc}\n\t" #endif:: "R" ( Kcb->jprobe_saved_regs. ARM_SP), "I" (sizeof (struct pt_regs) * 2), "J" (Offsetof (struct pt_regs, arm_sp)), "J" (Offsetof (struct pt_regs, arm_pc)), "J" (Offseto F (struct Pt_regs, ARM_CPSR)), "J" (offsetof (struct pt_regs, ARM_LR)): "Memory", "CC");

This simulates a fake pt_regs structure that fills only the SP and PC fields (as required by the Longjmp_break_handler function in the following), where the value of the PC is jprobe_magic_addr and then a long jump to Kprobe_ Handler execution, the Kprobe_handler function determines that there is currently kprobe running, so enter the following calling process:

} else if (cur) {/* We probably hit a jprobe.  Call it break handler. */if (Cur->break_handler && cur->break_handler (cur, regs)) {kcb->kprobe_status = KPROBE_HIT_SS; SingleStep (cur, regs, KCB); if (cur->post_handler) {kcb->kprobe_status = Kprobe_hit_ssdone;cur->post_handler (cur, regs, 0);}} Reset_current_kprobe ();

First Call Kprobe's Break_handler callback function, which is the Longjmp_break_handler function:

int __ Kprobes longjmp_break_handler (struct kprobe *p, struct pt_regs *regs) {struct KPROBE_CTLBLK *KCB = get_kprobe_ctlblk (); Long stack_addr = Kcb->jprobe_saved_regs. Arm_sp;long orig_sp = regs->arm_sp;struct Jprobe *jp = container_of (p, struct jprobe, KP); if (regs->arm_pc = = Jprobe _MAGIC_ADDR) {if (orig_sp! = stack_addr) {struct Pt_regs *saved_regs = (struct Pt_regs *) kcb->jprobe_saved_regs. ARM_SP;PRINTK ("Current SP%LX does not match saved SP%lx\n", orig_sp, Stack_addr);p RINTK ("Saved registers for Jprob E%p\n ", JP); Show_regs (Saved_regs);p rintk (" Current registers\n "); Show_regs (regs); BUG ();} *regs = kcb->jprobe_saved_regs;memcpy ((void *) stack_addr, Kcb->jprobes_stack, Min_stack_size (STACK_ADDR));p re Empt_enable_no_resched (); return 1;} return 0;} 

This function is very simple, the first will determine the SP value and the SP value is the same, if not the same as the report of the bug, or restore the register stored in the KPROBE_CTLBLK structure of the value and stack, and finally enable the kernel preemption, so that jprobe processing process is complete, The next step is to go back to Kprobe's kprobe_handler to complete this kprobe, perform a single-stepping single_step and Post_handler, and finally go back to the original process execution. Thus, at the end of the user-defined probe function, the Jprobe_return function must be called, otherwise the execution of the code "flies", and can no longer return to the original process.

2.3.2, x86_64 Architecture implementation

int Setjmp_pre_handler (struct kprobe *p, struct pt_regs *regs) {struct Jprobe *jp = container_of (p, struct jprobe, KP); Unsi gned long addr;struct kprobe_ctlblk *KCB = get_kprobe_ctlblk (); kcb->jprobe_saved_regs = *regs;kcb->jprobe_saved_ SP = STACK_ADDR (regs); addr = (unsigned long) (KCB->JPROBE_SAVED_SP);/* As Linus pointed out, GCC assumes EE * owns the argument space and could overwrite it, e.g. * Tailcall optimization. So, to is absolutely safe * We also save and restore enough stack bytes to cover * the argument area. */memcpy (Kcb->jprobes_stack, (kprobe_opcode_t *) addr, min_stack_size (addr)); Regs->flags &= ~X86_EFLAGS_IF  ; Trace_hardirqs_off (); regs->ip = (unsigned long) (jp->entry);/* Jprobes use Jprobe_return () which skips the normal return * Path of the function, and this messes the accounting of the * function Graph tracer to get messed up. * Pause function Graph tracing while performing the Jprobe function. */pause_graph_tracing (); rEturn 1;} Nokprobe_symbol (Setjmp_pre_handler);

The implementation of the x86_64 architecture is much the same as that of arm, where the function is first to save the scene, then closes the interrupt and sets the value of the IP register to Jp->entry, and finally returns 1, so that the Kprobe_int3_handler function skips single_step.

/* * If we have no pre-handler or it returned 0, we * continue with normal processing.  If we have a * Pre-handler and it returned Non-zero, it prepped * for calling the break_handler below on re-entry * for JP Robe processing, so get out doing nothing * more here. */if (!p->pre_handler | |!p->pre_handler (P, regs)) Setup_singlestep (p, regs, KCB, 0); return 1;

It then jumps to the user's probe function execution at the end of the kprobe call process. Look at the implementation of the Jprobe_return function:

void Jprobe_return (void) {struct KPROBE_CTLBLK *KCB = get_kprobe_ctlblk (); ASM volatile (#ifdef config_x86_64 "       xchg< C1/>%%rbx,%%rsp\n "#else"       xchgl   %%ebx,%%esp\n "#endif"       int3\n ""       . Globl jprobe_return_end\n "       " jprobe_return_end:\n ""       nop\n "::" B "(KCB->JPROBE_SAVED_SP):" Memory ");}

Unlike the implementation of ARM, where the int3 instruction is used to trigger the CPU3 exception again, and the address of the exception is no longer breakpoint_instruction, so it will go to the following process of Kprobe_int3_handler execution:

} else if (kprobe_running ()) {p = __this_cpu_read (current_kprobe); if (P->break_handler && p->break_ Handler (P, regs)) {if (!skip_singlestep (p, regs, KCB)) Setup_singlestep (p, regs, KCB, 0); return 1;}

The same is called Kprobe's Break_handler callback function execution, which is also the Longjmp_break_handler function.

int Longjmp_break_handler (struct kprobe *p, struct pt_regs *regs) {struct KPROBE_CTLBLK *KCB = get_kprobe_ctlblk (); U8 *add R = (U8 *) (regs->ip-1), struct jprobe *jp = container_of (p, struct jprobe, KP), void *saved_sp = Kcb->jprobe_saved_ Sp;if ((Addr > (U8 *) Jprobe_return) && (Addr < (U8 *) jprobe_return_end)) {if (STACK_ADDR (regs)! = Saved _SP) {struct Pt_regs *saved_regs = &AMP;KCB-&GT;JPROBE_SAVED_REGS;PRINTK (kern_err "Current SP%p does not match save D sp%p\n ", Stack_addr (regs), saved_sp);p rintk (kern_err" saved registers for Jprobe%p\n ", JP); Show_regs (Saved_regs) ;p RINTK (kern_err "Current registers\n"); Show_regs (regs); BUG ();} /* It's OK to start function graph tracing again */unpause_graph_tracing (); *regs = kcb->jprobe_saved_regs;memcpy (saved _SP, Kcb->jprobes_stack, Min_stack_size (saved_sp));p reempt_enable_no_resched (); return 1;} return 0;}

The Longjmp_break_handler function is basically consistent with the arm implementation, restoring the original context of the code, opening the kernel preemption, and finally returning to Kprobe to continue with the subsequent single_step and recovery process. However, the value of the note is the first judgment statement, because this INT3 exception is triggered in the Jprobe_return function, so the Longjmp_break_handler function of the struct Pt_regs *regs the parameter value is in the call Jprobe_ return function in the context of the register value, so addr must be in the address range of the Jprobe_return function, so as to determine the validity of this call, to prevent accidental entry.

3. Summary

The Jprobe detection technology is based on the Kprobe implementation, is the second of the Kprobes Three detection technology, the kernel developers can use it to detect the kernel functions of the call and the call when the parameter value, the use is very convenient. This paper introduces how to use the Jprobe detection tool and its principle, and analyzes the implementation mode of ARM architecture and x86_64 architecture through source code. The next post will introduce the last kretprobe detection technique in Kprobes to detect the return value of a function.

Linux kernel Debugging technology--jprobe use and implementation