UClinux system analysis

 

Introduction
Linux is a very popular operating system. It is compatible with Unix systems and has open source code. It was originally designed as a desktop system and is now widely used in the server field. The biggest influence is that it is gradually applied to embedded devices. UClinux is created in this atmosphere. In uClinux, the English word "U" indicates "micro". "C" indicates "control" and "control". Therefore, uClinux indicates "micro-control-Linux, literally, it is "a Linux system designed for the field of micro-control ".

UClinux miniaturization practices

Possible miniaturization of standard Linux

1. recompile the kernel
The Linux kernel adopts a modular design, that is, many functional blocks can be independently added or removed. developers can use these kernel modules as Optional options when designing the kernel, which can be specified when compiling the system kernel. Therefore, a common practice is to re-compile the Linux kernel, carefully select the function support module required by the embedded device during compilation, and delete unnecessary functions. By re-configuring the kernel, You can significantly reduce the number of kernels required for system operation, thus reducing resource usage.

2. Create a root file system image
The Linux system must load the root file system at startup. Therefore, the cropping system also includes the cropping of the root file system. In x86 systems, Linux can be started by loading the loadlin file in DOS.

The miniaturization method adopted by uClinux

1. uClinux kernel Loading Method
The uClinux kernel can be run in either flash or memory. This method can reduce memory requirements.
Flash running mode: Burn the executable image of the kernel to flash. When the system starts, it starts from an address of flash and runs it one by one. This method is actually used by many embedded systems.
Kernel Loading Method: stores the compressed file of the kernel in flash. When the system starts, it reads the compressed file and decompress it in the memory. This method is relatively complicated, however, the operation speed may be faster (the ram access rate is higher than that of Flash ). In addition, this is also the startup method used by the standard Linux system.

2. root file system of uClinux
UClinux uses the romfs file system, which requires less space than the general ext2 file system. Space saving comes from two aspects. First, the kernel supports romfs file systems and requires less code than ext2 file systems. Secondly, the romfs file system is relatively simple) requires less storage space. The romfs file system does not support dynamic write/write operations. The system uses the virtual ramdisk method to process the data to be dynamically saved (the ramdisk uses the ext2 File System ).

3. uClinux application library
Another way to reduce the size of uClinux is to rewrite the application library. Compared with the larger and more comprehensive glibc library, uclibc simplifies libc.
UClinux uses static connections to user programs. This method will increase the application, but the memory-based management problem occurs, we have to do this (this will be explained in the subsequent analysis of uClinux memory management), and this approach is closer to the practice of General embedded systems.

Development Environment of uClinux

GNU Development Kit
As a general Linux Open Kit, the GNU Development Kit includes a series of development and debugging tools. Main components:
GCC: compiler, which can be used as a cross-compilation form, that is, to develop and compile binary files that can run on the host machine.
Binutils: Some auxiliary tools, including objdump (binary files can be decompiled), as (assembly compiler), LD (connector) and so on.
GDB: a debugger that supports multiple cross-debugging methods, such as GDB-BDM (background debugging tool) and gdbserver (Ethernet debugging ).
UClinux printing Terminal
Generally, the default terminal of uClinux is a serial port. When the kernel is started, all information is printed to the serial port terminal (print using the printk function). At the same time, you can also use the serial port terminal to interact with the system.
When uClinux is started, telnetd (Remote Logon Service) is enabled. The operator can remotely log on to the system to control the running of the system. You can choose whether to enable remote logon by burning the romfs file system.

Cross-compilation debugging tool
Supports a new processor that requires compilation and assembly tools to form binary files that can run on this processor. The compilation tools used by the kernel are different from those used by the application. Before explaining the differences, you need to describe the GCC connection:
. LD (Link Description) file: The LD file indicates the memory image format during connection.
Crt0.s: the Startup File required for application compilation and connection. It is mainly used to initialize the application stack.
PIC: Position independence code, a location-independent binary format file, must include the reloc segment in the program segment, so that the code can be relocated during loading.
Ucsimm is used to compile the connection. LD file to form an executable file image. The code segment can be indirectly addressable (that is, using reloc segments for addressing) or absolute addressing. This gives the compiler more room for optimization. Because the kernel may use absolute addressing, the memory address space loaded by the kernel must be exactly the same as the memory space specified in the LD file.
The connection mode of the application is different from that of the kernel. The application is loaded by the kernel (the executable file loader will be discussed later). Because the memory space provided by the LD file of the application may be different from the memory location actually loaded by the application, in this way, a new position is required in the application loading process, that is, the reloc segment is corrected, so that the program does not encounter errors when indirectly addressing. (This problem varies with i386 and other advanced processors. This article will further analyze it later ).
As discussed above, at least two sets of compilation and connection tools are required. After discussing the memory management of uClinux, this article will show the workflow of the entire system and the space distribution of the system in flash and ram.

Executable File Format
First, I would like to explain some terms:
Coff (Common Object File Format): A Common Object File Format
Elf (excutive linked file): A common file format used by Linux system. It supports dynamic connection.
Flat: ELF format has a large file header. The flat file simplifies the file header and some segment information.
UClinux uses flat executable file format. GCC compilers cannot directly form this file format, but can form executable files in coff or ELF format, these two types of files require coff2flt or elf2flt tools for format conversion to form a flat file.
When you execute an application, the kernel's execution file loader will further process the flat file, it mainly modifies the reloc segment (for details about the executable file loader, see fs/binfmt_flat.c ). The reloc Section is further discussed below.
The root cause of the need for reloc segments is that the program running space assumed by the connector during connection is different from the memory space loaded by the actual program. Suppose there is such an instruction:
JSR app_start;
This command uses direct addressing and jumps to the app_start address for execution. After compilation, the connection program calculates the actual address of app_start (if the actual address is 0x10000 ), the actual address is calculated based on the LD file (because the connector assumes that the program will be loaded to the memory space specified by the LD file ). However, due to memory allocation, the operating system cannot guarantee that the program will be loaded according to the LD file during loading. If the program is still directed to the absolute address 0x10000 for execution, this is usually incorrect. One solution is to add a bucket to store the actual address of app_start. If the variable ADDR is used, it indicates the bucket. Then the above program will be changed:
Movl ADDR, a0;
JSR (A0 );
The added variable ADDR occupies a 4-byte space in the data segment, and the connector stores the absolute address of app_start to this variable. When an executable file is loaded, the executable file loader calculates the actual location of app_start in the memory based on the memory space to be loaded by the program and writes the ADDR variable. In actual processing, the system does not need to know the exact storage location of the variable (nor can it be known). The system only needs to process the entire reloc segment (the reloc segment has an identifier, the system can read it out ). Processing is simple. You only need to add an offset to the values stored in the reloc segment. (If the loaded space is higher than expected, the offset is actually subtracted ). The offset is calculated based on the subtraction of the actual starting value of the physical address and the starting value of the address specified by the LD file.
This reloc method is partly caused by memory allocation problems in uClinux, which will be explained in the memory management analysis of uClinux.

Real-Time Solutions
UClinux itself does not focus on real-time issues. It is not proposed for the real-time performance of Linux. In addition, there is a Linux-RT-Linux concern about real-time issues. RT-Linux execution manager treats the Linux kernel as a task and manages real-time processes. Rather than real-time processes are handed over to the common Linux kernel for processing. This method has been applied to many operating systems to enhance the real-time performance of the operating system, including some commercial Unix systems and Windows NT. One of the advantages of this method is its simplicity and ease of real-time performance testing. The second advantage is that non-real-time processes run on the standard Linux system and maintain great compatibility with other commercial Linux versions. The third advantage is that it supports hard real-time clock applications. UClinux can use patch of RT-Linux to enhance the real-time performance of uClinux, so that uClinux can be applied to industrial control, process control and other applications with high real-time requirements.

UClinux Memory Management
It should be said that the biggest difference between uClinux and standard Linux lies in memory management. At the same time, the memory management of uClinux also causes some problems that will not occur in standard Linux. This document compares uClinux memory management with the standard Linux memory management section.

1. virtual memory technology used in standard Linux
Standard Linux uses virtual memory technology, which is used to provide much larger memory than the physical memory actually used in computer systems. The user will feel like the program can use a very large memory space, so that programmers do not have to consider the actual capacity of physical memory in the computer when writing the program.
To support Virtual Storage Manager Management, Linux uses paging to load processes. The so-called paging separates the actual memory into segments of the same size. For example, each segment contains 1024 bytes, so the segment of the size of 1024 bytes is called a page ).
The virtual memory is supported by the memory management mechanism and a large-capacity Fast hard disk storage. Its implementation is based on the local principle. When a program is running, it is not necessary to load all the memory, instead, it only loads some pages or segments that are currently running into the memory for running (copy-on-write), and the rest are temporarily on the hard disk if the page (segment) it wants to access is) if a page (segment) does not exist, the operating system generates a page error (page fault ), this error causes the operating system to load the part to be run to the memory. If necessary, the operating system can also switch unnecessary memory pages (segments) to the disk. Using this method to manage memory, you can convert the memory used by a process to zero and load it in batches as needed, the core program uses the page number of each page to address each storage segment.
Standard Linux is designed for processors with memory management units. On such a processor, virtual addresses are sent to the memory management unit (MMU) to map virtual addresses to physical addresses.
By assigning different virtual-physical address translation mappings to each task, protection between different tasks is supported. The address translation function defines in each task that the virtual address space in a task maps to a part of the physical memory, and the virtual address space of another task maps to another area in the physical memory. The memory management unit (MMU) of a computer generally has a set of registers to identify the conversion table of the currently running process. When the current process abandons the CPU to another process (one context switch), the kernel loads these registers by pointing to the pointer to the address translation table of the new process. MMU registers are privileged and can only be accessed in kernel mode. This ensures that a process can only access the addresses in its own user space, rather than accessing and modifying the space of other processes. When an executable file is loaded, the loader loads the program to a virtual memory space based on the default LD file. For this reason, the virtual address space of many programs is actually the same, however, because the conversion functions are different, the actual memory area is also different. For multi-process management, when the processor switches the process and executes a new task, an important part is the conversion table for the new task switchover task. We can see that the memory management in Linux has at least implemented the following functions:
Run programs that are larger than the memory. Ideally, programs of any size can be run.
◇ You can run programs that only load part of the program and shorten the startup time of the program.
◇ Multiple programs can stay in the memory at the same time to improve CPU utilization
◇ You can run the relocation program. That is, the program can be anywhere in the memory and can be moved during execution.
◇ Write machine-independent code. The program does not have to specify the machine configuration in advance.
◇ Reduce the burden on programmers to allocate and manage memory resources.
◇ Can be shared-for example, if two processes run the same program, they should be able to share the same copy of the program code.
◇ Provides memory protection. processes cannot access or modify pages in an unauthorized manner. The kernel protects the data and code of a single process to prevent other processes from modifying them. Otherwise, the user program may accidentally (or maliciously) destroy the kernel or other user programs.
The virtual storage system does not have no cost. Memory Management requires address translation tables and some other data structures, and the memory reserved for the program is reduced. Address Translation increases the execution time of each command, and more serious commands with extra memory operations. When a process accesses a page that is not in memory, the system becomes invalid. The system processes the failure and loads the page to the memory. This requires a very time-consuming disk I/O operation. In short, memory management activities occupy a considerable amount of CPU time (about 10% of the BUSY systems ).

2. Special nommu processing in uClinux
For uClinux, it is designed for a processor without MMU, that is, uClinux cannot use the virtual memory management technology of the processor (it should be said that this kind of processor without MMU is quite common in embedded devices ). UClinux still uses paging Management of memory, and the system paging the actual memory at startup. The program is loaded by page when the application is loaded. However, since MMU management is not available, uClinux uses real memeory management ). This affects many aspects of system work.
The uClinux system directly accesses the memory (its access to the address does not need to go through MMU, but is directly sent to the address for online output ), the address accessed in all programs is the actual physical address. The operating system does not protect the memory space (which is actually a feature of many embedded systems). Each process actually shares a runtime space (without an independent address translation table ).
Before a process is executed, the system must allocate sufficient contiguous address space for the process and load all the data into the contiguous space of the primary memory. Correspondingly, the standard Linux system does not need to ensure that the actual physical storage space is continuous when allocating memory, but it only needs to ensure that the virtual storage address space is continuous. In another aspect, the program loading address is usually different from the expectation (indicated in the LD file), so the relocation process is necessary. In addition, the disk swap space cannot be used. If the system lacks memory during execution, it cannot be improved through disk swap.
UClinux reduces memory management and puts forward higher requirements for developers. In terms of ease of use, uClinux's memory management is a kind of backend, Which is returned to the Unix or DOS system era. Developers have to participate in system memory management. Starting from compiling the kernel, developers must tell the system How much memory the Development Board has (if you fool the system, it will be punished when running the program later ), in this way, the system will paging the memory during the initialization phase and mark the used and unused memory. The system will use the paging memory when running the application.
Because the application must be allocated continuous address space during loading, block each time for different hardware platforms (continuous address) the size limit of allocated memory is different (currently, uClinux for ez328 processors is 128 kb, but the system memory for ColdFire processors is not limited ), therefore, when developing an application, developers must consider the memory allocation and pay attention to the size of the running space required by the application. In addition, because the real-memory management policy is adopted, the user program is in the same address space as the kernel and other user programs. During program development, ensure that the address space of other programs is not infringed, so that the program does not disrupt the normal operation of the system, or cause exceptions in other programs.
From the perspective of memory access, the developer's rights have increased (the developer can access any address space during programming), but at the same time the system's security is also greatly reduced. In addition, the system will greatly change the management of multi-process, which will be described in the multi-process management of uClinux.
Although uClinux's memory management function is much different from the standard Linux system, it should be said that this is the choice of embedded devices. In embedded devices, due to the impact of cost and other sensitive factors, the general use of a processor without MMU, which determines that the system does not have enough hardware to support virtual storage management technology. From the perspective of functions implemented by embedded devices, embedded devices usually run in a specific environment. As long as they implement specific functions, their functions are relatively simple, the requirements for memory management can be fully considered by developers.

 
3. standard Linux system processes and threads
Process: A process is an entity that runs a program and provides the execution environment for it. It contains an address space and at least one control point. A process executes a single command sequence in this address space. The process address space includes a set of memory units that can be accessed or referenced. The process control point controls and tracks the sequence of process commands through a hardware register commonly called program counter (PC.
Fork: because the process is the environment of the execution program, you must first establish the environment that can "run" the program before executing the program. The Linux system provides a system call to copy the content of the current process to generate a new process. The process that calls fork is called the parent process, and the new process is called the child process. A child process inherits all the features of the parent process, but it has its own data segment. That is to say, although the child process changes its variable, it does not affect the variable value of the parent process.
Parent and Child processes share a program segment, but each has its own stack, data segment, user space, and process control block. In other words, two processes run the same program code, but each has its own program counter and its own private data.
When the kernel receives a fork request, it first checks whether the memory is sufficient and whether there are still vacancies in the progress table; the last step is to check whether the user has created too many sub-processes. If the above three conditions are met, the operating system will give the sub-process a process identifier, set the CPU time, and then set the segment shared with the parent process, at the same time, copy the inode of the parent process to the sub-process for use. In the end, the sub-process returns 0 to indicate that it is a sub-process. As for the parent process, it may wait for the execution of the sub-process to end, or do it with the sub-process.
Exec System Call: This system call provides the ability of one process to execute another process. Exec System Call overwrites the old process memory content, therefore, the stack, data segment, and program segment of the original program are modified, and only the user zone remains unchanged.
Vfork System Call: When fork is used, the kernel copies the parent process to the child process, but this is a waste of time, in most cases, the program calls exec immediately after calling fork, so that the copied process area is immediately overwritten by new data. Therefore, a Linux system provides a system to call vfork. vfork assumes that the system will execute exec immediately after vfork is called. Therefore, vfork does not copy the page of the parent process, only initialize the private data structure and prepare enough paging tables. In this way, after the vfork call is complete, the Parent and Child processes actually share the same memory (before the child process calls exec or exit). Therefore, the child process can change the data and stack information of the parent process, therefore, after the vfork system is called, the parent process goes to sleep until the child process executes exec. When a sub-process executes exec, because exec uses the data of the program to be executed, the Code overwrites the storage area of the sub-process, resulting in a write protection error (do_wp_page) (at this time, the sub-process actually writes the storage area of the parent process). This error causes the kernel to re-allocate the storage space for the sub-process. When the sub-process correctly starts execution, it will wake up the parent process so that the parent process continues to run later.

4. multi-process processing in uClinux
UClinux does not have MMU to manage memory. When multiple processes are implemented (fork calls to generate sub-processes), data protection is required.
Fork and vfork of uClinux: fork of uClinux is equal to vfork. In fact, multi-process management in uClinux is implemented through vfork. This means that after the fork in uClinux calls the process, either the child process is executed in place of the parent process (at this time, the parent process has sleep) until the child process calls exit to exit, or the exec is called to execute a new process, at this time, executable files are generated. Even if the process is only a copy of the parent process, this process cannot be avoided. After the child process executes exit or exec, the child process uses wakeup to wake up the parent process, and the parent process continues to run.
UClinux's multi-process implementation mechanism is closely related to its memory management. UClinux is developed for nommu processors, so it is forced to use a flat mode of memory management. when starting a new application, the system must allocate storage space for the application, and immediately load the application to the memory. Without the memory re ing mechanism of MMU, uClinux must process the executable file reloc In the executable file loading phase, so that the program can directly use the physical memory during execution.