On the analysis method of Linux kernel

This article from http://blog.csdn.net/ouyang_linux007/article/details/7422346

One of the biggest benefits of Linux is its open source. At the same time, the open core source code also attracts countless computer enthusiasts and programmers, they interpret and analyze Linux core source as their greatest interest, modify the Linux source code and transform the Linux system as their pursuit of computer technology the biggest goal. Linux Kernel source code is very attractive, especially when you understand an analysis of a long time did not understand the problem, or you have modified the kernel, smoothly through the compilation, everything is running normally. It's a feeling of accomplishment! Moreover, the analysis of the kernel, in addition to the fanatical pursuit of technology, the rewards of this daunting labor is very fascinating, and this is the main reason for its many followers:

• First of all, you can learn a lot of the bottom of the computer knowledge, such as the following will be talked about the guidance of the system and hardware to provide the interrupt mechanism, and other, such as virtual storage implementation mechanism, multi-tasking mechanism, system protection mechanism, and so on, these are all non-source code can not understand.
• At the same time, you will be from the overall structure of the operating system to understand the overall design in the software design of the weight and role, as well as some macro-design methods and techniques: The Linux kernel for the upper-level applications to provide a platform that is not related to the specific hardware, and inside the kernel, it also divides the code into architecture and hardware-related and portable parts; For example, although Linux is not a microkernel, he handles most of the device drivers into relatively independent kernel modules, which reduces the overhead of running the kernel and enhances the module independence of the kernel code.
• And you can also from the analysis of the kernel source, realize it in solving a specific detail problem, the method is ingenious: As the later analysis of the Linux through the botoom_half mechanism to speed up the system to interrupt processing.
• The most important thing is: in the source code analysis process, you will be 1.1 points, subtle specialization. A professional programmer, always put the code of clarity, compatibility, portability in a very important position. They always define a large number of macros to enhance the clarity and readability of the code without increasing the length of the compiled code and the efficiency of the code; they always encode and take into account future code maintenance and upgrades. Even if you analyze 1% of the code, you will have a deep understanding of what kind of code is written by a professional programmer, what kind of code is written by a hobbyist. And this is something that no one in the real analysis of standard code can appreciate.

However, because of the lengthy kernel code, and the complexity of the kernel architecture, the analysis of the kernel is also a difficult, very need of perseverance, in the absence of guidance and communication, especially in the case. Only by the right way can you do more. Based on this consideration, the author hopes that this article can give us some reference and enlightenment.

Since my analysis is based on the 2.2.5 version of the kernel, if not specifically described, the following analysis is based on the i386 single processor 2.2.5 Linux kernel. All source files are relative to the directory/usr/src/linux.

one way: Where to start

To analyze the Linux kernel source code, you must first find the location of each module, that is, to understand the source code of the file organization. Although this is not difficult for an experienced master, it is necessary for many beginner Linux enthusiasts, and those who are interested in source analysis but have little contact.

1, the Linux core source program is usually installed under/usr/src/linux, and it has a very simple numbering convention: Any even-numbered core (the two number is an even, For example 2.0.30) are the core of a stable distribution, and any odd core (such as 2.1.42) is the core of a development.

2, the core source files are organized according to the tree structure, at the top of the source program tree, that is, directory/usr/src/linux there are some directories and files:

COPYING:GPL Copyright statement. A program created by a change in source code that has GPL copyright, or a program produced using the GPL, has the obligation to use the GPL, such as public source;

CREDITS: Hall of Fame. Information about some of the people who have made a great contribution to Linux;

Maintainers: Maintenance Personnel list, who is responsible for the current version of the kernel parts;

Makefile: The first Makefile file. Used to organize the kernel of the various modules, recorded a module of the relationship between each other and rely on relationships, compile-time use; it is very helpful to read the makefile files in each sub-directory to clarify the relationship and reliance of each document.

ReadMe: The core and its compiler configuration method is briefly introduced;

Rules.make: Some common rules used by various makefilemake;

Reporting-bugs: Some content about reporting bugs;

arch/: The Arch subdirectory includes all the core code related to architecture. Each of its subdirectories represents a supported architecture, such as i386, which is a subdirectory of the Intel CPU and its compatible architecture. PC machines are generally based on this directory;

include/: The Include subdirectory includes most of the header files needed to compile the core. Platform-Independent header files in the Include/linux subdirectory, the Intel CPU-related header files are in the include/asm-i386 subdirectory, and the INCLUDE/SCSI directory is the header file directory for the SCSI device;

init/: This directory contains the core initialization code (note: Not the system's boot code), contains two files main.c and VERSION.C, which is one of the good starting points for studying how the core works.

mm/: This directory includes all memory management code that is independent of the CPU architecture, such as page storage Management memory allocation and deallocation, and architecture-related memory management code in arch/*/mm/, such as arch/i386/mm/fault.c;

kernel/: The main core code, the file in this directory implements the kernel functions of most Linux systems, the most important of which is sched.c; Similarly, the architecture-related code is in Arch/*/kernel;

drivers/: Place all device drivers for the system, and each driver consumes one subdirectory: For example,/block is a block device driver, such as the IDE (IDE.C). If you want to see how all the devices that might contain the file system are initialized, you can look at the Device_setup () in DRIVERS/BLOCK/GENHD.C. It not only initializes the hard disk, but also initializes the network, because the NFS file system needs to be installed on the network;

documentation/: Document directory, no kernel code, just a set of useful documents, unfortunately, are Chinese, see should be useful oh;

fs/: All file system codes and various types of file manipulation codes, each of which supports a file system, such as fat and ext2;

ipc/: This directory contains the code for the core interprocess communication;

lib/: Placing the core library code;

net/: Core and network-related code;

Modules/: Module file directory, is an empty directory, used to store the module target file generated at compile time;

scripts/: Description file, script, for the configuration of the core;

Generally, in each subdirectory, there is a Makefile and a readme file, carefully read the two files, the kernel source of understanding is very useful.

The Linux kernel source code analysis, there are a few good entry points: One is the system's boot and initialization, that is, from the machine power to the core of the system operation, the other is the system call, the system call is the user program or operation call core functions provided by the interface. For those who are familiar with the hardware, from the guidance of the system to analyze, it may come easier, and from the system call down, it may be more appropriate for those who have a DOS or Uinx, Linux under the C programming experience of the master. These two points will also be introduced later.

Method Two: Take the procedure as the clue, the first line bead

From the surface, the Linux source code is like a group of disorderly mess, in fact, it is an organized web of orderly. To analyze the whole structure clearly, in addition to finding the thread, but also to straighten out the relationship between the various parts, methodical 1.1-point analysis.

The so-called procedural process as a clue, a line of beads, refers to the process of execution according to the procedure, the code involved in the execution of the program is clearly analyzed. The most typical applications of this method are two: one is the initialization process of the system, and the other is the execution process of the application: from the loading of the program to the running, until the program exits.

For the sake of simplicity, follow the principle of gradual progress, and now the system initialization process to specifically introduce this method. The initialization process of the system includes: System boot, initialization in real mode, and initialization in protected mode, total three parts. Here's a description.

There are two common ways to boot Linux systems: Lilo boot and Loadin boot, and the Linux kernel comes with a bootsect-loader. Since it can only implement Linux boot, unlike the first two, which has a lot of flexibility (Lilo can achieve multi-boot, Loadin can boot Linux under Dos), so in general applications rarely use Bootsect-loader. Of course, Bootsect-loader also has its own advantages: short no redundant code, attached to the kernel source, is the core source of organic components, and so on.

Bootsect-loader and the corresponding program in the source code is/arch/i386/boot/bootsect. S The following will be primarily an analysis of this file.

2. Several related documents:
   
   <1>/arch/i386/boot/bootsect. S
   
   <2>/include/linux/config.h
   
   <3>/include/asm/boot.h
   
   <4>/include/linux/autoconf.h
   
3. Guided Process Analysis:
   
   For an Intel x86 PC, when power is turned on, the machine starts performing a series of system test actions for the ROM BIOS, including checking for Ram,keyboard, monitors, hard disks, and so on. After performing the system test of the BIOS, the control is then transferred to the boot program (Rom bootstrap routine) in the ROM, which reads the No. 0 track No. 0 sector on the disk (called the boot sector or MBR, where the system's bootloader is placed) into memory. and place it at 512 bytes from the beginning of the 0x07c0:0x0000; then the processor jumps here to start executing the boot program, and after loading the bootstrapper in the MBR, Cs:ip = 0x07c0:0x0000. The power-on processor operates in a real mode compatible with 8086.
   
   If you want to use Bootsect-loader for system boot, you must put bootsect. s compile the connection after the corresponding binary code is placed in the MBR; when the Rom BIOS puts the bootsect. After compiling the connection, the binary code is loaded into memory, and the control of the machine is completely forwarded to Bootsect, that is, Bootsect will be the first program to be read into memory and executed.
   
   After the bootsect takes over control of the machine, the following actions are performed in turn:
   
   1. First, Bootsect moves its "self" (512 bytes from position 0x07c0:0x0000) to 0x9000:0000 from the address 0x07c0:0x0000 the ROM BIOS loads; This task is made by Bootsect. The first 10 instructions of s are completed, and the 11th instruction "Jmpi go,initseg" then jumps the machine to the "new" Bootsect "Jmpi go,initseg" after the instruction "Go:mov di, #0x4000 12"; Continue execution of Bootsect's remaining code; in Bootsect. Several constants are defined in S:
   
   Bootseg = 0x07c0 The address of the agreed location of the BIOS loading the MBR;
   
   initseg = 0x9000 bootsect. s the first 10 instructions to move themselves here (Section address)
   
   Setupseg =0x9020 loading SETUP.S segment Address
   
   Sysseg =0x1000 System Segment Address
   
   See definitions in/include/asm/boot.h for these constants, which will be used frequently in the following analysis;
   
   2. Set up its own stack area with 0x9000:0x4000-12 as the bottom of the stack, where the 12 bytes 0x9000:0x4000-12 to 0x9000:0x4000 are reserved as disk parameter table area;
   
   3. Create a new disk parameter table in the 12 reserved bytes from 0x9000:0x4000-12 to 0x9000:0x4000, which is called the "new" Disk parameter table, relative to the disk parameter table established by the BIOS. Because the designers consider that some older BIOS does not accurately identify the disk "number of sectors per track", causing the BIOS to establish a disk parameter table to hinder the maximum performance of the disk, so the designer in the BIOS set up the disk parameter table based on the enumeration method to test, to establish an accurate "new" Table of disk parameters (this is done in subsequent steps), and the position of the parameter table is moved from the original 0x0000:0x0078 to 0x9000:0x4000-12, and the old disk parameter table area is modified to point to the new disk parameter table;
   
   4. The next step is the Load_setup process, which invokes the 2nd service of 0x13 interruption, and reads the contiguous setup_sects (constant 4) sectors starting with the No. 0 2nd sector to the memory area adjacent to Bootsect, which is 0x9000:0 X0200 begins with 2048 bytes, and the contents of these four sectors are/arch/i386/boot/setup. s binary code after the connection is compiled, that is, if you want to use Bootsect-loader for system boot, you must not only bootsect. The binary code is placed in the MBR after the connection is compiled and the setup is also put in place. The binary code corresponding to the S compile connection is placed in a contiguous four sectors immediately following the MBR; Of course, due to setup. The executable code for S is loaded by bootsect, so in our project we can modify the Bootsect to arbitrarily place setup as needed. s corresponding executable code;
   
   5. The only exit of the Load_setup process is the Probe_loop sub-process, which tests the disk "number of sectors per track" by enumeration method;
   
   6. The next few sub-processes are relatively clear: print our familiar "Loading", read into the system to 0x1000:0x0000; Turn off the floppy drive motor, determine the disk type according to the "number of sectors per track" measured in 5 steps, and finally jump to 0x9000:0x0200, which is setup. s the entry of the corresponding executable code, transferring machine control to setup. S; The entire bootsect code runs;
5. Memory impression graph After the boot process has finished executing:
7. Memory impression graph After the boot process has finished executing:

For the sake of simplicity, in this analysis, I overlooked the analysis of the processing of the large kernel, because the processing of the large kernel is only a small part of this boot process and does not affect the overall grasp. After the system is booted, the system will go into the initialization processing stage. The initialization of the system is divided into two parts: Real mode and protection mode.

II, initialization in real mode

In real-mode initialization, it refers to some of the processing that the system has done since the kernel booted successfully into protected mode. The corresponding program in the kernel source code is/arch/i386/boot/setup. S; The following sections are primarily analysis of this file. This part of the analysis is mainly to understand its processing flow and the INITSEG (9,000:0000) section of the parameter table, this parameter table contains a lot of hardware parameters, these are in the protection mode after the initialization, and the core of the foundation of the establishment.

1. several other related documents:

<1>/arch/i386/boot/bootsect. S

<2>/include/linux/config.h

<3>/include/asm/boot.h

<4>/include/asm/segment.h

<5>/include/linux/version.h

<6>/include/linux/compile.h

2. initialization process Analysis in real mode:





INITSEG (9,000:0000) Section parameter table: (see INCLUDE/LINUX/TTY.H)

Name of parameter	Offset (0x9000 for all segments)	Length byte	Reference files
Param_cursor_pos	0x0000	2	Arch/i386/boot/video. S
Extended Mem Size	0x0002	2	Arch/i386/boot/setup. S
Param_video_page	0x0004	2	Arch/i386/boot/video. S
Param_video_mode	0x0006	1	Arch/i386/boot/video. S
Param_video_cols	0x0007	1	Arch/i386/boot/video. S
Useless	0x0008	2	Include/linux/tty.h
Param_video_ega_bx	0x000a	2	Arch/i386/boot/video. S
Useless	0x000c	2	Include/linux/tty.h
Param_video_lines	0x000e	1	Arch/i386/boot/video. S
Param_have_vga	0x000f	1	Arch/i386/boot/video. S
Param_font_points	0x0010	2	Arch/i386/boot/video. S
Param_lfb_width	0x0012	2	Arch/i386/boot/video. S
Param_lfb_height	0x0014	2	Arch/i386/boot/video. S
Param_lfb_depth	0x0016	2	Arch/i386/boot/video. S
Param_lfb_base	0x0018	4	Arch/i386/boot/video. S
Param_lfb_size	0x001c	4	Arch/i386/boot/video. S
Temporarily unused ①	0x0020	4	Include/linux/tty.h
Param_lfb_linelength	0x0024	2	Arch/i386/boot/video. S
Param_lfb_colors	0x0026	6	Arch/i386/boot/video. S
Temporarily unused ②	0x002c	2	Arch/i386/boot/video. S
Param_vesapm_seg	0x002e	2	Arch/i386/boot/video. S
Param_vesapm_off	0x0030	2	Arch/i386/boot/video. S
Param_lfb_pages	0x0032	2	Arch/i386/boot/video. S
Keep	0x0034--0x003f		Include/linux/tty.h
APM BIOS Version③	0x0040	2	Arch/i386/boot/setup. S
BIOS Code Segment	0x0042	2	Arch/i386/boot/setup. S
BIOS Entry Offset	0x0044	4	Arch/i386/boot/setup. S
BIOS-bit Code SEG	0x0048	2	Arch/i386/boot/setup. S
BIOS Data Segment	0x004a	2	Arch/i386/boot/setup. S
Supports 32-bit flag ④	0x004c	2	Arch/i386/boot/setup. S
BIOS Code SEG length	0x004e	4	Arch/i386/boot/setup. S
BIOS Data seg Length	0x0052	2	Arch/i386/boot/setup. S
hd0 parameters	0x0080	16	Arch/i386/boot/setup. S
hd0 parameters	0x0090	16	Arch/i386/boot/setup. S
PS/2 Device Flag ⑤	0x01ff	1	Arch/i386/boot/setup. S

* Note : ①include/linux/tty.h:cl_magic and Cl_offset here

1. Include/linux/tty.h:

unsigned char rsvd_size; */0x2c * / unsigned char rsvd_pos; */0x2d */

③0 indicates no APM BIOS

④0x0002 to support 32-bit mode

⑤0 said no, 0X0AA said there was mouse

III, initialization in protected mode

The initialization in protected mode refers to some of the processing that the system does during the process of the first kernel program running the system after the processing machine enters the protection mode. Protection mode initialization in the kernel source code corresponding to the program is/arch/i386/boot/compressed/head. S and/arch/i386/kernel/head. S; The following sections are mainly for the analysis of these two files.

1. Several related documents:
   
   <1.>/arch/i386/boot/compressed/head. S
   
   <2.>/arch/i386/kernel/head. S
   
   <3.>//ARCH/I386/BOOT/COMPRESSED/MISC.C
   
   <4.>/arch/i386/boot/setup. S
   
   <5.>/include/asm/segment.h
   
   <6.>/ARCH/I386/KERNEL/TRAPS.C
   
   <7.>/include/i386/desc.h
   
   <8.>/include/asm-i386/processor.h
   
   
2. Initialization process Analysis in protected mode:
   
   First,/arch/i386/kernel/head. S process:
   
   
3. Initialization process Analysis in protected mode:
   
   First,/arch/i386/kernel/head. S process:

Second,/arch/i386/boot/compressed/head. S process:

1. As you can see from the flowchart, the initialization in protected mode is mostly done with a few things:
   1. Unzip the kernel to 0x100000,
   2. Set up the page directory and Pg0 page table and start the paging function (that is, the virtual Storage management function),
   3. Save the hardware information measured in real mode to Empty_zero_page, initialize the command buffer,
   4. Detect the CPU type, check the coprocessor,
   5. Re-establish GDT global descriptor table, and interrupt description schedule IDT;
2. From the page directory and Pg0 page table can be seen,0& #0; 4M physical memory is used as the system area, it is mapped to the 0& #0 of the linear space of the system segment, 4M and 3g& #0; 3g+4m; The system can access the actual 0&# by accessing these two segments 0;4m physical memory, which is the region in which the system resides;
3. The global descriptor Descriptor GDT has been established in real mode, and the global descriptor descriptor GDT has been built here for two reasons: one is that if the kernel is a large kernel bzimag, the previously established GDT may have been overwritten by the decompression. In this source file are only the relative transfer instructions JXX NF or jxx nb; second, the previously established GDT was built in the field site mode, and now is established after enabling the protection of virtual address mode, that is, the current GDT is based on the logical address (i.e., the linear address);
4. Each time a new GDT is established and the protected virtual address mode is enabled, the system stack must be reloaded and the registers of each segment are reinitialized: Cs,ds,es,fs,gs;
5. Initialization in real mode and the initialization process in protected mode can be seen as follows: The process of Linux system entering into protected mode is as following.

6. Since the paging mechanism can only be started in protected mode and cannot be started in real mode, the first step is necessary, and since the GDT and IDT are built on the logical address (linear address) in 386 protected mode, the third step is also necessary;
7. After the initial in real mode and protected mode, the main system data is distributed as follows:

Initial post-primary system data distribution table

Position	System data	Size
0x101000	Page Catalog Swapper_pg_dir	4K
0x102000	Page Table Pg0	4K
0x103000	Empty_bad_page	4K
0x104000	Empty_bad_page_table	4K
0x105000	Empty_zero_page	4K
0x105000	System Hardware Parameters	2 K
0x105800	Command buffers	2 K
0x106000	Global Description Schedule Gdt_table	4192B

From the above analysis of the initialization process of Linux system can be seen, the program execution process as a clue, the first line beads, according to the order of execution of the program, to understand the process of implementation of the various stages of processing, and the interaction between the stages. The flowchart should be the most appropriate expression tool for this method of analysis.

In fact, taking the program execution process as a clue is the preferred method for analyzing any source code. Because of the particularity of the operating system, this method of light is far from enough. Of course, using this method to analyze the initialization process of the system or the execution process of the user process should be very effective. For your safety, please only open URLs with reliable sources

On the analysis method of Linux kernel