Exploring the origin of the source--deep understanding of the Linux kernel beautiful scenery of the system launch (i) __linux

From Get Linux3.1.1 version of the core source code and build a good reading environment began, to now around two months time, during the Google a large and small article, just clear a little thought and found a reading of the mouth. A good guide for kernel beginners is more important than anything, and the methodology for Linux kernel learning can refer to the Linux kernel practices that FUDAN_ABC writes, and the author speaks to the reader with his deep core and humorous writing. This kind of appeal makes me read the whole column almost in one breath, and I believe that it will be helpful for any learner who has a strong interest in the kernel. In addition, for beginners, the methodology is not enough, especially for the Linux kernel such a large maze, so a good map in the core learning is equally important, Kconfig makefile is playing such an important role, This is especially true for learners who want to focus on a particular module or subsystem, see the specific article in the above column- kernel map: kconfig and Makefile , and for makefile Kbuild system introduced more specific can see Yunsong written makefile Preparatory Knowledge/kbuild system , of course, the most authoritative information naturally from the kernel document. According to my own reality, I do not think it is necessary to analyze every makefile file in detail, because the final result is obvious--a kernel image consists of hundreds or even thousands of files, such a large amount of anatomical work, especially for the like I have not built a large system, Even writing the largest makefile files are only a few lines of beginners, is undoubtedly a huge challenge, so through Google search for the previous written on a module relied on the source files of the article can let us put time on more important source code analysis.

Kbuild/makefile/kconfig
According to personal experience, reading the Linux kernel for beginners First hurdle is the source file large and small config_xxxx logo, which for the vast number of people like me without access to drive development/file System/network protocol learners are undoubtedly the first major impact on self-confidence, Fortunately, the Linux kernel distribution provides a wealth of documentation, it is common to encounter many unfamiliar things in the kernel learning process, so it will be easier for you to learn to find the right documents.

The documentation for Kbuild/makefile/kconfig can be found in the directory Documentation\kbuild, and the following is a brief overview of these three in the document: Makefile
The makefile consists of a total of five sections: ① top-level makefile files, ② kernel configuration files, ③ makefile in various architectures, arch/$ in directory ④ (ARCH), and a series of common rules for kbuild makefile. These files are mainly in the scripts directory, ⑤kbuild makefiles, there are about 500 of such files.
The top layer makefile reads the. config file, which is generated primarily during kernel configuration. For specific configuration of the kernel, seehere。 The top-level makefile is primarily used to construct two of the most important files: vmlinux--fixed kernel images, and arbitrary modules, which are accomplished by recursively entering subdirectories of the kernel source tree. The list of subdirectories that need to be accessed depends on the configuration of the kernel. The top-level makefile contains a makefile file in a specific architecture that provides information about the specified architecture primarily to the top-level makefile.
Each subdirectory contains Kbuild Makefile, which is used to perform some of the commands passed by the parent directory. Kbuild Makefile constructs a variety of file lists by using the information contained in the. config file, and ultimately kbuild constructs any built-in or modular target based on these file listings.
Some makefile.* files contained in the scripts directory contain a series of definitions or rules that construct the kernel according to Kbuild makefile.

Kbuild
This is the build system that starts with the Linux2.6 version of the kernel. In short, a built-in or modular target is constructed from the list of files generated by makefile.

Kconfig
A kconfig file is a configuration database that is used to store a set of configuration options that are organized into a tree structure. These configuration options are organized into the form of a menu entry, a simple example of the following:[CPP]View Plain copy config modversions bool "Set version information on all module symbols" depends on MODULES  Help usually, modules have to is recompiled whenever you switch to a new kernel.   ... This is a simple configuration menu entry, however, this also illustrates the characteristics of all configuration entries: "Config" leads to a new configuration option, and the next line defines a series of properties that can be configured with a configuration option, input hints, dependencies, help documents, or default values. It is important to note that the same configuration menu entry can be defined multiple times, but there is only one input hint for each definition, and the type definition cannot conflict. The vast majority of entries define a configuration option, and other entries serve as dependencies for these entries. Given the importance of kconfig files, here are some of their common syntax:

--Type definition: "bool"/"TriState"/"string"/"hex"/"int", which indicates the type of input when the user configures this option. There are two most basic types: TriState and string, and other types based on both. A type definition allows you to give input hints, such as:[CPP]View Plain copy bool "Networking support" is equivalent to[CPP]View Plain copy bool Prompt "Networking support" both indicate that the input type of the configuration menu is bool, and there will be a prompt for "networking support" before entering.

--Default value: ' Defaults ' <expr> [' if ' <expr>]
A configuration option can have any number of default values. If more than one default value is visible, the default value is selected if and only if the first defined default is active, that is, if the user does not configure the option. The default value will not be restricted by the menu item, which means that the default value can be defined anywhere else or overwritten by the previous definition. You can increase the default values that exist under the specified conditions by adding an "if" condition statement.

--type definition + default value: "Def_bool"/"Def_tristate" <expr> ["if" &LT;EXPR&GT;]
This is the shorthand for the type definition and default value. You can also add a type and default value under a specified condition by adding an "if" statement.

--Dependencies: "Depends on" <expr>
This defines a dependency condition for this menu item, and if there are multiple dependencies, they can be connected through the symbol "&&". dependencies apply to all other properties that this menu item starts at its definition. For example:[CPP]View plain copy bool "foo" if bar default y if bar is equivalent to[CPP]View plain copy depends on BAR bool "foo" Default Y
--Help document: "Helping" or "---helps---"
This property defines a help document, and the end of the help document depends on its indentation level, in other words, a line that has a smaller indent than the first line at the beginning of the Help document indicates the end of the entire help document. "---help---" does not make a difference in effect, except that "---helps---" assists developers in separating configuration options from the entire menu item.

The above is a brief introduction to Kbuild/makefile/kconfig, in fact, the key to understanding these three is: because the building software is compiled from the source file, and the Linux kernel supports a variety of CPU architectures, This allows the entire kernel to contain thousands of each of the required source files under each schema, however, in the process of forming a kernel image, only one part of the source file is extracted from the source files, and the source files required under all architectures (such as memory management/process dispatch/network, etc.) can be compiled. , the fact that part of the Kbuild makefile implements this functionality is that the interdependence between the source files needed to build each goal in Makefile is complex and the upper makefile need to invoke the lower makefile. This can also lead to target coverage, which may also require script files in the scripts directory to assist in compiling, as well as a large set of predefined environment variables/automation variables and built-in functions in makefile for beginners. The last link order for some of the modules have a strict progressive relationship, otherwise there may be damage to the hardware, you can see that only the kernel image of the source files required to analyze the process will be a lot of effort, so this part of the work recommended through the search engine to complete. The kconfig related to our source files should be properly understood, or, as mentioned above, will cause a great deal of trouble with the specific parsing process of the code. ToPage_32_types.hMacros defined in the header file__page_offsetThe variables used in theConfig_page_offsetFor example, a variable that encounters such a configuration type first finds the Kconfig file in the directory under the corresponding architecture.Note that our explanations are always directed against the x86_32 architecture., so the Kconfig file is in Arch\x86, and the directory in which the variables of the configuration type are located will not be summarized in the profiling process, and the menu items for Config_page_offset are as follows:[CPP] View plain copy config page_offset       hex        default 0xB0000000 if VMSPLIT_3G_OPT       default  0x80000000 if vmsplit_2g       default 0x78000000 if  vmsplit_2g_opt       default 0x40000000 if VMSPLIT_1G       default 0xC0000000       depends on x86_32    you can see that the menu item is a type hex, that is, an integer that is displayed in the 16-in format and not a visual (non-visible) variable (because there is no input hint, which is displayed as a string with no double quotes after hex, and no prompt input prompt). If the expression vmsplit_3g_opt is predefined, then the value is 0xb000 0000, and then the three defaults are page_offset desirable values under the specified conditions, and the last default value without any criteria is 0xc000 0000, that is, when we use the default configuration, the value of Page_offset is 0xc000 0000, and finally we see the depends on X86_32 statement that the menu item has a dependency on the X86_32 configuration option, but we see no other attributes defined thereafter. So you can simply ignore the dependency. The meaning of the page_offset in the source file is the offset of the kernel within the linear address space of a single process, indicating that by default the Linux kernel will occupy a linear address space of up to 1G in the 0~4g virtual memory under the x86 architecture, It also shows that items starting from 0x300 (i.e. 768) in the page directory entry of any process correspond to the innerThe nuclear address space.

The above is a brief introduction to Kbuild/makefile/kconfig and would like to have a closer look at the contents of this section to refer to the information previously listed. We can see that the most closely related to our source code analysis is our kconfig configuration file, so we can roughly understand Kconfig file is the most basic reading kernel source requirements, in addition to the X86\configs directory of i386_ The Defconfig file also has a default configuration for part of the config__xxxx identity. The above description is just a prelude to the beginning of our journey to the core, and will then officially enter our core world.

Power up
First you need to know that memory (DRAM) is a type of volatile storage device, from the microscopic level, the leakage of current factors will cause the DRAM unit in 10~100 millisecond time loss of charge, So the storage system must periodically read out and then write back to refresh each bit of memory, this is different from SRAM, as long as there is electricity, SRAM stored in the information will not need to refresh, SRAM than the advantages of DRAM and access speed, light and electrical noise interference is not sensitive, The cost of all these advantages is that the SRAM unit uses more transistors than the DRAM unit, which is more expensive and consumes more power; In macro terms, the vulnerability is that once the power is off, the information stored in memory will be lost. So our kernel must be stored in a class of permanent media, even if the power loss can still save information, on the PC such media is our hard disk, it is by converting information into electrical signals, and then converting electrical signals into magnetic field to magnetization materials, so that the information will be permanently preserved, Of course, other nonvolatile storage devices such as U disk are also available. However, our program must be loaded into memory before it can be executed, because the instructions that are executed on the CPU are addressed in memory, this requires the presence of some kind of external force to "put" the kernel into memory and perform the corresponding initialization work, then transfer control to the kernel. Up to the user to provide attentive service, down can be effective management of hardware resources to make the entire wonderful machine world effectively running, and this "external force" is our basic input/output system (Input/output System,bios), It is called the BIOS because it includes a few interrupt-driven low-level processes. All operating systems need these processes to initialize computer hardware devices at startup. Because the BIOS process can only run in real mode, Linux will no longer use the BIOS when it enters protection mode, but instead provides its own device drivers for each hardware device on the computer.

At the beginning of the start, there is a special hardware circuit in the CPU on a PIN to generate a reset logic value, the CPU in the identification of the reset signal after the data bus set to high impedance state, the address line is forced to 1, and disable interrupts. After this, some registers of the processor are set to a fixed value, of which the most important two registers--cs segment register is placed as the 0XF000,EIP instruction pointer register as 0x0000 FFF0, because the PE bit in the CR0 register has not been put bit, so in real mode state, The address of the instruction is calculated by filling a 0 to the right of the 16-bit CS segment register to form a small segment, plus the IP instruction pointer register in the content (EIP register low 16-bit), with an image of the formula is expressed as: Cs*16+ip, finally get the address value of 0xf Fff0. Note that the first instruction for addressing this address is stored in the BIOS, and because the BIOS is solidified in ROM, and the instructions in the ROM to address the need to use 32-bit addresses (reason please refer to pc storage ), to contact the previous description, the CPU to the address line to force 1 , the final form of the physical address is 0xFFFF fff0. After the address is formed, the CPU gives it to the MCH (the bridge chip-the equivalent of a distributor, assigned to different devices depending on the address map), the MCH determines that the address is assigned to the ICH (North Bridge chip-decoder), which eventually resolves to the address in the BIOS ROM.

The BIOS actually does the following 4 things: Perform a series of tests on the hardware to detect which devices are present and whether they are working properly, a phase called post (Power-on-self-test, Power on self-test). Initializes the hardware device, which ensures that all hardware device operations do not cause an IRQ (interrupt request) line to conflict with the I/O port, and a list of all PCI devices installed on the system is finally displayed. Search for an operating system to start. This process can be accessed according to the order of the user to access the floppy disk in the system, the hard disk and the first sector of the CD-ROM (that is, the boot sector), usually on the power-on when the DEL key into the BIOS settings interface, but may also be other keys, depending on the specific PC. When a valid device is found in the above access order, the contents of the first sector are copied to the physical address 0x0000 7c00 in RAM, and then jumps to the address to begin executing the code just loaded.

For more detailed introduction of the BIOS boot process can refer to the BIOS boot process-hardware detection and initialization analysis of the article. You want to learn about BIOS programming can refer to here .

Boot mount process (boot loader)
The boot loader is loaded by the BIOS, a program used to load the operating system's kernel image into RAM, which is stored on the No. 0 track No. 0 Sector of the hard disk, and is also referred to as a boot record because it only takes up less (512 bytes) of storage space. In addition, the boot record includes a 64-byte partition table and a 2-byte label that identifies the end of a valid boot record (0x55 0xAA). However, this boot loader is no longer performed from Linux2.6, which can be learned from the Header.s file (the assembly syntax for the gas format under Linux can be referencedGas LearningSeries):[CPP] View Plain Copy bootseg        = 0x07c0         /* original address of boot-sector */   sysseg         = 0x1000        /*  historical load address >> 4 */           .code16  /* represents 16-bit mode */       .section  ". Bstext",  "Ax"   /* the following code in the. Bstext section, "Ax" indicates that the section can be allocated and executable */          .global  bootsect_start  /* defines the global label bootsect_start*/   bootsect_start:          # Normalize the start address        ljmp     $BOOTSEG,  $start 2      start2:        movw    %cs, %ax       movw    %ax, %ds        movw    %ax, %es        movw    %ax, %ss       xorw     %sp, %sp       sti   executes a long jump instruction after the global label Bootsect_start LJMP $ Bootseg, $start 2 because the bootseg value is 0x07c0 and is still in real mode, so the instruction represents a transfer to the segment base address 0x7c00 (SEG*16), offset to the Start2 Contact to boot boot when the contents of the first sector by the BIOS copy to the physical address 0x0000 7c00, so the long jump instructions actually transferred to the Start2 label at the end of the execution, you can see the Start2 after the label of the instructions to the CS section Register in order to fill the contents of the content to ds/es/ SS Register, then clear the SP stack pointer register to 0 and enable interrupts.

[CPP]View plain Copy CLD/* Clear direction flag, used in the string transfer instruction (in this case, LODSB) to indicate that the DI register is automatically increased 1*/MOVW $bugger _off_msg After the transfer is completed,%SI/* the bug Ger_off_msg's first address is moved into the SI register/* MSG_LOOP:LODSB/* To read the memory address Ds:si contents into the AL Register/andb%al,%al/* Test the contents of the AL Register is 0 * * JZ bs_die/* If you jump to the label Bs_die to continue to execute * * Movb $0xe,%ah MOVW $,%BX int $0x10/* Call BIOS0 X10 number interrupted, the function of the interrupt is to display the character/jmp Msg_loop
You can see that the preceding code snippet is used to invoke the BIOS interrupt routine to display a string content that is identified as bugger_off_msg on the screen, and the BIOS interrupt content can be referencedhere。 The definition of bugger_off_msg is as follows:
[CPP]View Plain copy bugger_off_msg:. ASCII "Direct booting from floppy is no longer supported.\r\n". ASCII Use a boot loader program instead.\r\n ". ASCII" \ ". ASCII" Remove disk and press any key to Rebo ot ... \ r \ n ". Byte 0 tells the user that" no longer supports a direct boot from the floppy disk, instead use a boot loader program to remove the disk and press any key to reboot. " Then, after displaying the character, jump to the Bs_die label to continue execution, as shown in the following code snippet:

[CPP]View plain copy Bs_die: # Allow The user to press a key, then reboot XORW%ax,%ax/* Will ax register clear 0 */int  $0x16/* Reads the characters from the keyboard, waiting for the user to press the key/int $0x19/* Reboot the System/# int 0x19 should never return. In case it does anyway, # invoke the BIOS reset code ... ljmp $0xf000,$0xfff0/* long jump to 0xf fff0, that is, to rerun a system in the BIOS The process of column initialization * *

The code above waits for the user to restart immediately after pressing the key. actually though header. s file with bootloader, but if the kernel boot from the hard drive this piece of code will not be executed anyway, and the kernel maintainer no longer maintain the boot record, this can also be seen from the settings of the linked script file.[CPP] View Plain copy Output_format ("elf32-i386",  "elf32-i386",  "elf32-i386")    Output_arch (i386)    ENTRY (_start)   /* Specify entry marking */      sections   {        . = 0;       .bstext         : { * (. Bstext)  }       .bsdata         : { * (. bsdata)  }          .  = 497;       .header        :  { * (. Header)  }       .entrytext    : {  * (. Entrytext)  }       .inittext    : { * (. Inittext)  }       .initdata    : { * (. InitdatA)  }       __end_init = .;        ......  }  

The above file is the setup.ld in the Arch\x86\boot directory, and you can find that the link script specifies the entry as the _start label rather than the Bootsect_start, which is analogous to specifying the main () function as the entry point in C + +.in fact, when the Linux kernel starts from the _start label, it never jumps into the Bootsect_start label execution .。 Here's a quick reference to the syntax of the link script: Set properties for each section of the resulting file within sections{}. = 0 indicates that the code and data in the. Bstext and the. Bsdata section are loaded at offset 0. The description format for each section is "section name: {compose}", for example. Bstext: {* (. bstext)}, To the left represents the. Bstext segment of the resulting file, and the. Bstext section on the right representing all the target files, which means the final generated file. Bstext section is a combination of the. Bstext section of each target file, for more detailed description of the link script.gnu-ld Link Script Analysis。 Contact the. Section ". Bstext", "ax" directive at the beginning of the boot loader, and we know that the above code is at the beginning of the entire kernel image being loaded into memory, and that although the instructions will not be executed, they will still be loaded into memory. And the data stored in this 512-byte boot record is still useful, and these parameters will be used to initialize the Setup_header structure (in the arch\x86\include\asm\bootparam.h file) as follows:
[CPP] View plain Copy     .section  ". Header",  "a"        . globl  hdr   hdr:   setup_sects:    .byte 0          /* Filled in by build.c */   Root_ flags: .word root_rdonly   syssize:    .long 0          /* Filled in by build.c */   ram_size:    .word 0         /* obsolete */    vid_mode:   .word svga_mode   root_dev:   .word 0          /* filled in by build.c */    boot_flag:  .word 0xaa55          #  offset 512, entry point   These parameters are in the. Header section, the previous link script file learns that this section has an offset of 497 for the entire boot record, and that the variables in them will occupy a total of 1+2+4+2*4= 15 bytes of space, thus eventually constituting a 512-byte boot record. In addition, an assembly instruction from a label that identifies the end of the entire valid boot recordboot_flag:. Word 0xaa55And the last line of comments. Also note that because the x86 architecture uses a small-end method for storing data, the 0xaa55 final deposit form is 0x55,0xaa from low to high, which is not in conflict with what we described earlier. And the padding of these parameters can be seen from the annotation is done by the Build.c file, we will put in the following article to introduce the specific settings of this parameter.

As mentioned earlier, the bootloader is loaded by the BIOS and is used to load the kernel image into memory, but the Linux2.6 is no longer executing the code, so who is responsible for the heavy responsibility of loading the kernel image. In the x86 system, this is known as the Linux LOader (LILO) or the widely used grand Unified Bootloader (GRUB).

In fact, because of the rapid development of hard disk capacity, it is usually divided into several "partitions" of a hard disk, so that a physical hard disk can be considered as several logical disks. This allows an operating system image to be stored on each logical disk, and the first sector of each logical disk remains as the boot sector. Obviously these boot sectors are physically no longer the first sector of the entire hard disk (that is, the No. 0 track No. 0 sector on the entire hard disk), in which case the first sector of the entire hard disk is independent and does not belong to any one logical disk. However, the BIOS is booted from this sector after the post operation and initialization, so that the bootstrapper stored in the sector is called the Master boot record (master boot record,mbr). More detailed information about MBR can refer to the operating system loading process and the BootSector assembly language implementation .

On the surface, the loading process is divided into two phases, first loading the boot process in the MBR block, and then loading the secondary boot process for the specific operating system that is located in the first sector under a partition. However, from the analysis of the above bootloader process, we do not execute any instructions except for some of the data, because the modern operating system puts all the boot load tasks into the MBR and executes them uniformly. The execution process is divided into two parts: The MBR includes a small boot program that is loaded by the BIOS into the physical address 0x0000 7c00 in RAM, which loads the second part of the MBR into memory by establishing a real-mode stack. The second part reads the available operating system mapping table from disk in turn, and provides the user with a prompt to copy the kernel image of the partition directly into memory (including the boot record of the first sector in the partition) when the user selects the kernel to be loaded (or automatically chooses a default value after a period of delay).

The step in which the boot sector mounts the kernel image is further subdivided as follows: Call a BIOS routine to display the "Loading" information call the BIOS routine to mount the initial portion of the kernel image, and the first 512 bytes of the kernel image are loaded into memory from address 0x0009 0000. It also loads the remaining kernel images by calling the BIOS routines and putting the kernel image into RAM from the ground address 0x0001 0000 (small kernel image) or from the high address 0x0010 0000 (the large kernel image).