Deep understanding of the segmentation mechanism of Linux memory addressing

First, the preface

Recently learning the Linux kernel, read the chapter on memory addressing in the deep understanding of the Linux kernel. Originally thought oneself to the subsection paging mechanism already understood, the result discovers actually is the scanty knowledge. So, look for a lot of data, finally straighten out the knowledge of memory addressing. Now write down my understanding, I hope to help the kernel learners, and I hope you point out the wrong place.

Second, what is going on in the end?

Believe that people who have learned the course of the operating system are aware of segmented pagination, but it is strange that the book does not mention the basic section of how the page is produced, which leads us to know that they do not know why. Now let's take a look at the history of the segmented mechanism.

The birth of Real mode (16-bit processor and addressing)

Before the 8086 processor was born, memory addressing was a direct access to the physical address. The 8086 processor extends the address bus select、read to 20 bits to address 1M of memory space. However, an awkward problem arises, the ALU is only 16 digits wide, that is, the ALU can not calculate the 20-bit address. In order to solve this problem, the segmented mechanism was introduced and boarded the stage of history.

To support fragmentation, the 8086 processor is set up with four segment registers: Cs,ds,ss, ES. Each segment register is 16 bits, and the address in the instruction that accesses the memory is also 16 bits. However, before the address bus select、read is fed, the CPU first adds it to the value in a segment register. here to note: The value of the segment register corresponds to the high 16 bits in the 20-bit address bus, so the addition is actually the high 12 bits in the memory bus added to the 16 bits in the segment register, while the lower 4 bits remain unchanged, thus forming a 20-bit actual address. It also implements the conversion from a 16-bit memory address to a 20-bit actual address, or "mapping."

Birth of Protection mode (32-bit processor and addressing)

80286 Processor Address bus is 24 bits, addressing space up to 16M, while introducing protection mode (access to memory segment is limited)

The 80386 processor is a 32-bit processor, and the ALU and address bus are 32-bit, addressing space up to 4G. That is, it can directly access 4G of memory space without a segmented mechanism. Although it is the new age of the Little prince, beyond its countless predecessors, however, it needs to bear the family's mission-compatible with the previous generation of processors. In other words, it must support real mode and protection mode. Therefore, 80386 constructs the protection mode on the basis of the segment register, and retains the 16-bit segment registers.

The architecture is basically similar to the processor after 80386, known collectively as IA32 (Bit Intel architecture).

Three, IA32 memory addressing mechanism

Addressing hardware

In 8086 real mode, a segment of the register left 4 bits, and then added to the address addr is sent directly to the memory bus, the addition of the address is the physical address of the memory unit, and the program of this address is called the logical address (or virtual address). In IA32 protection mode, this logical address is not sent directly to the memory bus but is sent to the memory snap-in (MMU). The MMU is composed of one or a set of chips, the function of which is to map the logical address to the physical address, that is, address translation, as shown in the figure.

Three kinds of addresses for IA32

Logical address: Machine language instructions still use this address to specify the address of an operand or the address of an instruction. This approach is particularly specific in Intel's segmented architecture, which allows MS-DOS or Windows programmers to divide the program into segments. Each logical address consists of a segment and an offset.

Linear address: A linear address is a 32-bit unsigned integer that can express up to 232 (4GB) addresses. A linear address is usually represented by a 16 binary, and its value range is 0x00000000~0xffffffff.

Physical Address: The actual address of the memory unit used for chip-level memory cell addressing. The physical address is also represented by a 32-bit unsigned integer.

MMU Address conversion process

MMU is a kind of hardware circuit, it contains two parts, one is segmented part, the other is the paging part, we call them piecewise mechanism and paging mechanism respectively to facilitate the understanding of the hardware implementation mechanism from a logical point of view. A piecewise mechanism converts a logical address to a linear address; then the paging mechanism converts a linear address into a physical address.

Mmu_translate
Segment Registers for IA32

There are six 16-bit registers in the IA32: Cs,ds,ss,es,fs,gs. Unlike the 8086 segment registers, these registers are no longer the base address of a segment, but rather a segment selector (Selector).

Iv. implementation of piecewise mechanism

Segment is the basic unit of virtual address space, and the segmentation mechanism must convert one address of virtual address space into a linear address of linear address space.

In order to achieve this mapping, it is not enough to use a segment register to determine a base address, at least to describe the length of a segment, and some other information, such as access rights, is required. So what's needed here is a data structure that includes three aspects:

1. The base address of the segment: the starting addresses in the middle of the linear address space.

2. Paragraph bounds (Limit): The maximum offset that can be used within a segment in a virtual address space.

3. Paragraph protection attribute: represents the attribute of a segment. For example, whether the segment can be read out or written, or whether the segment is executed as a program, the privilege level of the segment, and so on.

The above data structure, which we call the segment descriptor, is composed of multiple segment descriptors called the Segment descriptor table.

Segment Descriptor

A descriptor (descriptor) is a 8-byte storage unit that describes the properties of a segment. In real mode, the properties of a segment are nothing more than code snippets, stack segments, data segments, beginning addresses of segments, length of segments, and so on, and are more complex in protected mode. IA32 combines them in a 8-byte number, called a descriptor.

As you can see from the diagram, a segment descriptor indicates the 32-bit base address of the segment and the 20-bit boundary (that is, the section length). Here we only focus on the base address and the segment bounds, and the other attributes skip over.

1. Paragraph Description Chart

A wide variety of user descriptors and system descriptors are placed in the corresponding global descriptor, local descriptor, and interrupt descriptor tables. The description table (that is, the paragraph) defines the situation for all segments of the IA32 system. All descriptor sheets themselves occupy a memory space of a multiple of byte 8, with a space size of 8 bytes (at least one descriptor) to 64K bytes (at most 8K) descriptors.

2. Global Descriptor (GDT)

Global descriptor Table GDT, in addition to task doors, interrupt doors, and trap door descriptors, contains descriptors for those segments that are shared by all tasks in the system. Its first 8 byte position is not used.

3. Interrupt Descriptor IDT (Interrupt descriptor Table)

Interrupt Descriptor IDT (Interrupt descriptor Table), contains 256 door descriptors. The IDT can only contain task doors, interrupt doors, and trap door descriptors, although the IDT table may be the longest or 64K byte, but only a descriptor within 2K bytes, that is, 256 descriptors, to maintain compatibility with 8086.

Local Descriptor Chart (LDT)

The local descriptor Table, LDT (descriptor), contains descriptors associated with a given task, each of which has a LDT. With LDT, you can isolate the code, data, and other tasks for a given task. The local descriptor of each task, LDT itself, is also represented by a descriptor, called the LDT descriptor, which contains information about the local descriptor chart and is placed in the global descriptor GDT.

Summarize

The IA32 memory addressing mechanism completes the conversion from the logical address-linear address-physical address. Where the value in the segment register of the logical address provides the segment descriptor, then the segment base and segment bounds are obtained from the segment descriptor, then the linear address is obtained by adding the offset of the logical address, and the linear address gets the physical address through the paging mechanism.

First, we need to be clear that the segmentation mechanism is the addressing method provided by IA32, which is the hardware level. In other words, whether you are Windows or Linux, as long as the use of IA32 CPU access to memory, you have to pass through the MMU conversion process to get the physical address, that is, must go through the logical address-linear address-physical address conversion.

Implementation of segmentation in Linux

Said so much about the implementation of the segmentation mechanism, in fact, for Linux, there is no egg to use. Because Linux basically does not use segmented mechanism, or, the partition mechanism in Linux is only designed to be compatible with IA32 hardware.

The segment mechanism of the Intel microprocessor was proposed from 8086, when the introduced segment mechanism resolved the conversion from 16-bit addresses to 20-bit real addresses within the CPU. To maintain this compatibility, 386 still uses the segment mechanism, but is much more complex than before. As a result, the Linux kernel design does not all take the Intel provided segment scheme, only limited use of the segmentation mechanism. This not only simplifies the design of the Linux kernel, but also creates conditions for porting Linux to other platforms because many RISC processors do not support the segment mechanism. However, understanding the knowledge of the segment mechanism is the only way to enter the Linux kernel.

Starting with version 2.2, Linux makes all processes (or tasks) use the same logical address space, so there is no need to use local descriptor LDT. But LDT is also used in the kernel, which only runs wine in VM86 mode, because it is used when simulating programs that run WINODWS Software or DOS software on Linux.

Any address given on IA32 is a virtual address, that is, any address is given by means of "selector: Offset", which is the basic feature of the mode of storage and access of the segment mechanism. Therefore, the use of segment mechanism cannot be avoided when designing the operating system on IA32.  A virtual address is eventually converted to a linear address through the "segment base site + offset" approach. However, because most hardware platforms do not support the segment mechanism, only the paging mechanism, so in order to make Linux more portable, we need to remove the segment mechanism and use only paging mechanism.  Unfortunately, the IA32 mechanism is not prohibited, so it is not possible to bypass it directly give the address of the linear address space. Under the helpless, the Linux designers simply let the segment's base address is 0, and the section of the boundary is 4GB, when any given an offset, the equation is "0+ offset = linear address", that is, "offset = linear address." In addition, because the segment mechanism stipulates "offset <4GB", so the range of the offset is 0H~FFFFFFFFH, which is exactly the range of linear address space, that is, the virtual address directly mapped to the linear address, we refer to the virtual address and the linear address refers to the same address. It seems that Linux in the absence of evasive mechanism in the case of a clever way to get around the mechanism of the past.

In addition, since the IA32 segment mechanism also stipulates that different segments must be created for code snippets and data segments, Linux must create a segment descriptor with a base address of 0 and segment boundaries of 4GB for code snippets and data segments. Not only that, because the Linux kernel runs at privilege level 0, and the user program runs at privilege Level 3, according to the IA32 section protection mechanism, the privilege Level 3 program cannot access the segment of the privileged level 0, so Linux must create its code snippets and data segments for the kernel user program separately. This means that Linux must create 4 segment descriptors--privileged level 0 code snippets and data segments, and privileged 3 code snippets and data segments.

Vi. Summary

The segmentation mechanism is the characteristic of IA32 architecture CPU, and is not the inevitable choice of operating system addressing mode. Linux in order to cross the platform, clever around the mechanism, the main use of paging mechanism to address.