Segmentation mechanism of Linux memory addressing

Objective

Recently learning the Linux kernel, read the chapter on memory addressing in-depth understanding of the Linux kernel. Originally thought oneself to the subsection paging mechanism already understood, the result discovers actually is smattering. As a result, a lot of information was found, and finally the knowledge of memory addressing was smoothed out. Now to record my understanding, I hope that the core learners have some help, but also hope that people point out the error.

What the hell's going on in the segment?

I believe that the people who learn the operating system courses know the sub-page, but it is strange that the book basically did not mention how the sub-page is produced, which led us to know that it does not know why. Let's take a look at the history of the segmentation mechanism.

The birth of Real mode (16-bit processor and addressing)

Prior to the birth of the 8086 processor, the memory addressing method was to directly access the physical address. The 8086 processor expands the address bus to 20 bits in order to address 1M of memory space. However, an awkward problem arises, the Alu width is only 16 bits, that is, the ALU can not calculate the 20-bit address. In order to solve this problem, the segmentation mechanism was introduced and boarded the historical stage.

To support fragmentation, the 8086 processor has four segment registers: CS, DS, SS, ES. Each segment register is 16 bits, and the address in the instruction that accesses the memory is also 16 bits. However, before the address bus is fed, the CPU first adds it to the value in a segment register. Note here that the value of the segment register corresponds to the high 16 bits in the 20-bit address bus, so the addition is actually a 16-bit memory address (that is, the intra-segment offset value) of the high 12 bits and the segment register 16 bits are added, while the low 4 bits remain unchanged, thus forming a 20-bit physical address, It also implements a conversion from a 16-bit memory address to a 20-bit actual address, or "map."

The above description of the segmentation mechanism to calculate the memory address is difficult to understand, drawing a diagram to help understand

+-----------------+|        |  20-bit address bus +-----------------+ +------------+|    |       16-bit segment address +------------+    +-------------+    |  | 4  |  16-bit memory address (intra-segment offset)    +--------+----+ Physical Address = (Segment register address << 4) + (CPU-submitted address)

The birth of the protection mode (32-bit processor and addressing)

• 80286-processor Address bus is 24-bit, addressing space up to 16M, while protected mode is introduced (limited access to memory segments)
• The 80386 processor is a 32-bit processor, and the ALU and address bus are 32-bit, addressing spaces up to 4G. In other words, it can directly access 4G of memory space without the fragmentation mechanism. Although it is the new age of the little prince, surpassing its countless predecessors, however, it needs to bear the family's mission – compatible with previous generations of processors. That is, it must support real mode and protected mode. Therefore, 80386 constructs the protection mode on the basis of the segment register, and retains 16 bits of the segment register.
• From the processor after 80386, the architecture is basically similar, collectively referred to as IA32 (Intel Architecture).

IA32 addressing hardware with memory addressing mechanism

In the real mode of 8086, a section of the register to the left 4 bits, and then add to address addr is sent directly to the memory bus, the added address is the physical address of the memory unit, and the program is called the address of the logical address (or called virtual address). In IA32 protected mode, this logical address is not sent directly to the memory bus but is sent to the Memory Management Unit (MMU). The MMU consists of a chip or a group of chips, whose function is to map the logical address to a physical address, i.e. address translation.

Three types of IA32 addresses

• Logical Address:
  The machine language instruction still uses this address to specify the address of an operand or an address of an instruction. This type of addressing is particularly specific in Intel's segmented structure, which allows MS-DOS or Windows programmers to divide the program into several segments. Each logical address consists of a segment and an offset.
• Linear Address:
  The linear address is a 32-bit unsigned integer that can represent addresses up to 232 (4GB). A linear address is usually represented by a 16 binary, with a value range of 0x00000000~0xffffffff.
• Physical Address:
  That is, the actual address of the memory unit for chip-level memory unit addressing. The physical address is also represented by a 32-bit unsigned integer.

MMU Address conversion process

The MMU is a hardware circuit that contains two parts, one is a segmented part, and the other is a paging part, where we call the segmentation mechanism and paging mechanism to facilitate understanding of the hardware implementation mechanism from a logical point of view. The segmentation mechanism converts a logical address into a linear address, and then the paging mechanism translates a linear address into a physical address.

Segment Register for IA32

There are six 16-bit segment registers in the IA32: CS, DS, SS, Es,fs, GS. Unlike the 8086 segment registers, these registers are no longer the base address of a segment, but rather a selector (Selector) for a segment.

Implementation of the segmentation mechanism

The segment is the basic unit of the virtual address space, and the segmentation mechanism must translate an address of the virtual address space into a linear address of the linear address space.

In order to implement this mapping, it is not enough to use a segment register to determine a base address, at least to describe the length of a segment, and to require some additional information about the segment, such as access rights. So what's needed here is a data structure that includes three things:

1. Base Address: The beginning of the line in the middle of the linear address space.
2. Limit: The maximum amount of offsets that can be used within a segment in a virtual address space.
3. Protection properties for segments (Attribute): Represents the attributes of a segment. For example, whether the segment can be read out or written, or whether the segment is executed as a program, the privileged level of the segment, and so on.

The above data structure is called a segment descriptor, and a table consisting of multiple segment descriptors is called a segment descriptor .

Segment Descriptor

The so-called descriptor (descriptor) is a 8-byte storage unit that describes the properties of a segment. In real mode, the attributes of a segment are nothing more than code snippets, stack segments, data segments, the starting address of a segment, the length of a segment, and so on, while in protected mode. IA32 combine them in a 8-byte number representation, called a descriptor.

As you can see from the diagram, a segment descriptor indicates the 32-bit base address of the segment and the 20-bit segment bounds (that is, the section length). Here we focus only on the base address and the segment bounds, and the other properties are skipped.

Segment Descriptor Descriptor

A variety of user descriptors and system descriptors are placed in the corresponding Global descriptor table, local descriptor tables, and interrupt descriptor lists. The descriptor table (that is, the paragraph list) defines all the segments of the IA32 system. All descriptor characters themselves occupy a memory space of a multiple of 8 bytes, with a space size of 8 bytes (at least one descriptor) to 64K bytes (at most 8K) descriptors.

1. Global Descriptor Descriptor (GDT)
   Global descriptor Table GDT, which includes the task gate, break gate, and Trap Gate descriptor, contains descriptors for those segments that are common to all tasks in the system descriptor. Its first 8-byte position is not used.
2. Interrupt descriptor Table IDT (Interrupt descriptor)
   The interrupt descriptor Table, IDT (Interrupt descriptor), contains 256 gate descriptors. IDT can contain only the task gate, the interrupt gate and the trap Gate descriptor, although the IDT table can be up to 64K bytes long, but can only access the descriptor within 2K bytes, that is, 256 descriptors, this number is to maintain compatibility with 8086.
3. Local Descriptor Descriptor (LDT)
   The local descriptor Table, descriptor, contains the descriptors associated with a given task, each of which has a LDT for each task. With the LDT, you can isolate the code, data, and other tasks for a given task. The local descriptor for each task The LDT itself is also represented by a descriptor, called the LDT descriptor, which contains information about the local descriptor list and is placed in the global descriptor descriptor GDT.

Summarize

The IA32 memory addressing mechanism completes the conversion from the logical address – the linear address – to the physical address. Where the value in the segment register of the logical address provides the segment descriptor, and then obtains the segment base and segment bounds from the segment descriptor, and then adds the offset of the logical address, the linear address is obtained, and the linear address gets the physical address through the paging mechanism.
First, we need to make it clear that the fragmentation mechanism is the addressing method provided by IA32, which is hardware-level. That is, whether you are Windows or Linux, as long as you use IA32 CPU access to memory, you have to go through the MMU conversion process to obtain a physical address, that is, must go through the logical address-linear address-physical address conversion.

Implementation of fragmentation in Linux

There is so much to say about the implementation of the segmentation mechanism, in fact, for Linux, there is no egg. Because Linux basically does not use a fragmentation mechanism, or that the fragmentation mechanism in Linux is designed to be compatible with IA32 hardware.

The segment mechanism of the Intel microprocessor was proposed starting from 8086, when the segment mechanism introduced resolved the conversion from the internal CPU to the 16-bit address to the 20-bit real address. To maintain this compatibility, 386 still uses the segment mechanism, but is much more complex than before. Therefore, the design of the Linux kernel does not all use the Intel-provided segment scheme, only a limited use of the fragmentation mechanism. This not only simplifies the design of the Linux kernel, but also creates the conditions for porting Linux to other platforms because many RISC processors do not support the segment mechanism. However, understanding the knowledge of the segment mechanism is the only way to enter the Linux kernel.

Starting with version 2.2, Linux allows all processes (or tasks) to use the same logical address space, so it is not necessary to use the local descriptor descriptor for the LDT. However, the LDT is also used in the kernel, which only runs wine in VM86 mode, because it is used to simulate programs running WINODWS software or DOS software on Linux.

Any address given on the IA32 is a virtual address, that is, any address is given by the "selector: Offset" method, which is the basic feature of the memory access mode of the segment mechanism. Therefore, it is not possible to avoid using the segment mechanism when designing an operating system on IA32. A virtual address will eventually be converted to a linear address by the "segment base address + offset" approach. However, because the majority of hardware platforms do not support the segment mechanism, only support paging mechanism, so in order to make Linux more portable, we need to remove the segment mechanism and only use paging mechanism. Unfortunately, IA32 specifies that the segment mechanism is not forbidden, so it is not possible to bypass it to give the address of the linear address space directly. Helpless, the Linux designers simply let the base address of the segment is 0, while the boundary of the segment is 4GB, when any given an offset, the equation is "0+ offset = linear address", that is, "offset = linear address". In addition, because the paragraph mechanism stipulates "offset <4GB", so the range of offsets is 0H~FFFFFFFFH, which is exactly the linear address space, that is, the virtual address directly mapped to the linear address, we refer to the virtual address and the linear address refers to the same address. It seems that Linux has cleverly bypassed the segment mechanism without bypassing the segment mechanism.

In addition, because the IA32 segment mechanism also stipulates that different segments must be created for both the code snippet and the data segment, Linux must create a segment descriptor for both the code snippet and the data segment with a base address of 0 and a segment boundary of 4GB respectively. Not only that, because the Linux kernel is running at privileged level 0, and the user program is running at privileged 3, according to the IA32 segment protection mechanism, the privileged Class 3 program is unable to access the privileged 0 segment, so Linux must create its own code snippets and data segments for the kernel user program. This means that Linux must create 4 segment descriptors--privileged level 0 code snippet and data segment, privileged level 3 code snippet and data segment.

Summarize

The segmentation mechanism is the characteristic of IA32 architecture CPU, and is not the inevitable choice of the operating system addressing mode. Linux in order to cross-platform, clever bypass mechanism, mainly using paging mechanism to address.

Segmentation mechanism of Linux memory addressing