Two remote debugging schemes for Embedded Operating Systems

Article Title: two remote debugging solutions for Embedded Operating Systems. Linux is a technology channel of the IT lab in China. Includes basic categories such as desktop applications, Linux system management, kernel research, embedded systems, and open source.
　　 One plug-in (stub)
The first solution is to add some functional modules in the target operating system and the debugger respectively, and the two are interconnected for debugging. The above problems can be solved through the following channels:
　　
Communication between the debugger and the program to be debugged
The debugger communicates with the target operating system by specifying the communication port (serial port, Nic, and parallel port) following the remote debugging protocol (see the http://rtos.ict.ac.cn/rtos/debugger/ for remote debugging protocol ).
Notify the debugger of exceptions caused by the debugged program in time
All exception handling operations of the target operating system must eventually be switched to the communication module to inform the debugger of the current exception number. The debugger then shows the user the exception type generated by the program to be debugged.
Control and access the program to be debugged by the debugger
Such requests of the debugger are actually converted into access to the address space of the program to be debugged or some registers of the target platform. The target operating system can directly process such requests. For a simple embedded operating system without the concept of virtual storage, it is very easy to complete these tasks.
The debugger identifies multitasking information about the program to be debugged and controls a specific task.
Related interfaces are provided by the target operating system. The target system calls this interface based on the multi-task request sent by the debugger to provide relevant information or control over a specific task, and returns the information to the debugger.
The debugger processes information related to the target hardware platform.
Article 21 The debugger shall be able to identify the types of exceptions that may occur on the Target Platform Based on the exception number. Such work can be completed independently by the debugger. Supporting Multiple Target platforms is a major feature of gnu gdb.
　　
To sum up, this scheme requires the target operating system to provide communication modules (including simple device drivers) that support remote debugging protocols and multi-task debugging interfaces, and rewrite the relevant parts of exception handling. In addition, a breakpoint setting function needs to be defined for the target operating system, because some hardware platforms provide debugging traps to generate specific exceptions (debug traps) breakpoint commands to support debugging (such as X86 INT 3), while other machines do not have similar commands, they are replaced by any illegal (Reserved) commands that cannot be interpreted and executed. These modules added to the target operating system are collectively referred to as "plug-ins" (see Figure). Resident modules are called ROM monitor. The general operating system also has such modules: If you enable the kgdb switch when compiling the Linux kernel running on the Alpha, platinum, or PowerPC platform, it is equivalent to adding a plug-in.
　　 　
　　
Applications running on the target operating system must call the breakpoint-setting function at the entrance to generate exceptions. The exception handling program calls the debug port communication module and waits for the host) information sent by the debugger on. After both parties establish a connection, the debugger waits for the user to issue the debugging command. The target system waits for the command generated by the debugger according to the USER command. Shows the process.
　　 　
　　
The essence of this solution is to use software to take over all exception handler and part of interrupt handling of the target system, insert the debugging port communication module into it, and interact with the host debugger. It can only take effect after the initialization of the target operating system, especially after the initialization of the debugging communication port is completed. Therefore, it is generally only used to debug applications running on the target operating system, it is not suitable for debugging the target operating system, especially the Startup Process of the target operating system. And because it must occupy a communication port of the target platform, the communication program of the port cannot be debugged. The most important thing is that it must change the target operating system, even if it does not adversely affect the performance of the operating system during debugging, at least one debugging version is not officially released for the target system.
　　
　　 On-Chip Debugging and Embedded PowerPC Background Debug Mode
In-chip debugging, additional control modules are embedded in the processor. When certain trigger conditions are met, some special status is entered. In this state, the debugged program stops running. The debugger of the host can access various resources (registers, memory, etc.) and execute commands through a special external communication interface of the processor. To match the signal of each pin of the host communication port and the target board debugging communication interface, the two are often connected through a simple signal conversion circuit board (as shown in ). Embedded control modules exist in the form of microcode monitor or hardware-only resources, including interfaces (such as breakpoint registers) provided to users ). Specific products include Motorola CPU16, CPU32, Coldfire series BDM (Background Debug Mode), Motorola PowerPC 5xx, 8xx series EPBDM (Embedded PowerPC Background Debug Mode ), JTAG (Joint Test Action Debug, IEEE Standard) of IBM and TI, as well as OnCE and MPSD. The following describes the on-chip debugging method using the epbdm of the MPC860 as an example.
　　
　　
The EPBDM operation is equivalent to taking over interrupt and exception handling with the debugging module embedded in the processor. You can set the debug enable register to specify which interrupts or exceptions occur before the processor enters the debugging status, rather than the operating system's processing program. After entering the debugging status, the embedded debugging module sends a signal to the external debugging communication interface, notifying the host debugger that has been listening on the communication interface, the debugger then allows the processor to execute arbitrary system commands (equivalent to privileged state) through the debugging module ). All commands are obtained through the debugging module. All load/store commands directly access the memory, and the cache and Storage Management Unit (MMU) are unavailable; the data register is mapped to a special register DPDR, which is accessed through the mtspr and mfspr commands. The debugger sends the rfi (return from interrupt) command to the processor to terminate the debugging. The program to be debugged continues to run.
　　
In contrast to the disadvantages of the plug-in method, OCD does not occupy the communication port of the target platform, and does not need to modify the target operating system. It can debug the Startup Process of the target operating system, greatly facilitating system developers. The downside is the increase in software workload: In addition to the need to add modules for multi-task Identification and Control of the target operating system on the debugger, you must also compile various ROM and RAM initialization programs for different development boards using the same chip.
　　
The following describes some key details about how to debug the startup of an operating system using OCD.
　　
First, the Linux kernel module resides in the target board ROM in the form of compressed zImage. After the target board is powered, run the program at the specified position in the ROM to move the kernel to RAM and decompress it, then jump to the kernel entry to run. To debug the kernel, you must obtain control of the system before executing the commands in the ROM after power-on, that is, entering the debugging status and setting breakpoints. The mp860 EPBDM provides this method.
　　
The MPC860 does not have commands similar to the X86 INT 3 that can generate specific debugging trap exceptions, while the operating system kernel often has exception handling for illegal commands; in order to minimize interference with normal kernel running, you should try to set hardware breakpoints when debugging, instead of using illegal commands to generate abnormal "soft" breakpoints.
　　
LINUX implements virtual storage management, which is also available in embedded LINUX. The address space conversion from real to virtual is completed during kernel startup. The debugger cannot bypass access to the virtual address space of the target system no matter whether the kernel or application is debugged, otherwise, when a breakpoint hits, the current command cannot be displayed based on the virtual address of the program counter, let alone the access variable. Because the Translation Lookaside Buffer cannot be used in the debugging status, it can only be used in the case of Linux kernel TLB failure, based on the index position of the page table in the virtual address, access the specific register to query the two-level page table to obtain the physical page number, so as to complete the conversion of the actual address. The MPC860 adopts the Harvard architecture and sets the separation of instruction and data cache (because the instruction and data segments of the program are separated, this structure can eliminate the conflict between the FETCH Command and the access data), and The TLB of the two is also set separately. However, when the TLB fails, the process of searching the page table to calculate the physical address is the same, because there is only one page table, there is no instruction or data separation problem. Although the task of real-world address translation falls entirely on the debugger side, due to the above reasons, coupled with the debugging object being an embedded system, there is generally no external storage device, and there is no need to consider the case of Memory Access missing pages, therefore, the increased workload is not large.
　　
　　 In-depth topic
The traditional debugging method can be summarized as follows: Set a breakpoint -- pause a program -- observe the program status -- continue running. If the debugging is a real-time system, even if the debugger supports batch processing commands to avoid the delay caused by user input commands and observation results, communication between the platform and the target system may also miss the response to the peripheral signals of the target platform. Therefore, the plug-in of the target side is improved based on the trace point provided by some debuggers (such as GDB), which is called the agent ). During debugging, you first set monitoring points in the debugger and specify the names of objects of interest in the form of source code expressions. To reduce the work of the proxy parsing expression, the debugger converts the expression to a simple bytecode and transfers it to the proxy. After the program runs, it hits the monitoring point and wakes up the agent. The agent records the user's required data and saves it to a specific Buffer Based on the bytecode (not only the final result of the expression, but also the intermediate result), so that the program continues to run; this step does not need to communicate with the debugger. When the debugger gets control again, you can issue a command to query previous monitoring records from the agent. Compared with the plug-in, the proxy adds an analysis module that connects to the received bytecode. The size of the target code is only about 3 kb. Of course, the monitoring record buffer also occupies the storage space of the target platform, however, the size of the buffer can be determined by the user when the proxy is generated. In short, this improvement provides a low-cost and feasible solution for real-time monitoring at the cost of limited target system resources.
　　
Debugging does not only mean that the process of breakpoint-program pause-observation-continues, but also requires profiling, trace, and other means, however, the technological advances in modern microprocessor have brought difficulties to the implementation of these debugging methods. Taking tracking as an example, the purpose is nothing more than to record the real program running stream. The instruction cache of modern processors can all be integrated into the chip (especially for the Proteus processor ), during command execution, the "Fetch" operation is mostly performed on the instruction cache inside the chip. On the chip's external bus, only the prefetch of multiple commands can be observed ), prefetch commands are not necessarily executed (due to jump and other reasons). In addition, commands are often executed in disorder in the pipeline after dynamic scheduling, and how to reproduce the original order is also a problem. There are roughly three solutions:
　　
In addition to normal operation, some processors can also run in serial mode. All the fetch cycles can be displayed on the off-chip Bus (equivalent to disabling cache and pipeline ). In this way, tracking is much easier, and the processor performance is greatly reduced. It is not applicable to systems with strict real-time requirements.
　　
The compiler automatically inserts write commands to a specific memory area at the specified branch and function entrances and exits (similar to profiling tools such as gprof ), they are written directly to the memory without caching, which can be reflected in the chip's external bus and thus recorded by the external logic analyzer.