Linux Memory addressing

Directory

• 1 memory Address Type
• 2 "Memory-segment management" on hardware
  • 2.1 Segment selector and segment registers
  • 2.2 Segment Descriptor (Segment descritor)
  • Quick access to 2.3 segment descriptors
  • 2.4 Conversion of the logical address to the linear address
• 3 Memory-segment management in Linux
• 4 page-managed on the hardware
  • 4.1 General Paging
  • 4.2 Expand Paging
• 5 in-memory page management in Linux

1 memory Address Type

The programmer accesses the content stored in the memory unit by means of a memory address. On the X86 architecture, you need to differentiate between the following three types of addresses:

• Logical addresses (Logical address)

  The logical address is the address of the operand of the instruction operator in the machine language. The logical address consists of a segment and an offset of two parts.

• Linear addresses (Linear address)

  A linear address is also a virtual address, with a 32-bit unsigned number addressing up to 4 GB of space. Linear addresses are typically expressed in 16 notation, such as the addressing space for 32-bit unsigned numbers: 0x00000000 ~ 0xFFFFFFFF.

• Physical addresses (physical address)

  The physical address is used for addressing in the memory chip, and its address and the pin correspondence of the processor to the memory bus, typically expressed as an unsigned number of 32 or 36 bits.

  In other words, the physical address is the address used between the CPU and memory.

Conversion relationships between three addresses such as:

                +--------------+                   +--------+
  Logical Addr  | segmentation |   Linear Addr     | PAGING |  Physical addr
--------------> |     UNIT     |------------------>|  UNIT  |--------------->
                +--------------+                   +--------+

The segmentation unit is responsible for translating the logical address into a linear address, and the PAGING unit is responsible for translating the linear address into a physical address. The two UNIT is described in a later section.

Multiple CPUs share memory, and memory reads and writes must be serial, which is a hardware known as the Memory Arbitor (arbiter). In fact, on a single CPU machine, also use this arbiter, because the single CPU on the use of DMA also to deal with the memory of concurrent access problems. But from a programmatic point of view, we don't need to care about this arbiter. 2 "Memory-segment Management" of 2.1 segment selectors and segment registers on hardware

As mentioned earlier, the logical address consists of two parts: the segment and the offset within the segment. Where a segment can be represented by a 16-bit "segment selector" and an offset, it can be represented by a 32-bit variable.

The X86 processor provides segment registers for quick access to segment descriptors, which include:

• Cs

  Code Segment Register, which is used to store snippets, which is the program's instructions.

• Ss

  Stack Segment Register, which holds the stack for the current program.

• Ds

  Data Segment Register, which holds both global and static information.

• ES, FS, GS

  General purpose registers.

The CS register also has an important function: The CS register contains a 2-bit area: The CPL, which is the current Privilege level, represents the CPU's currently privileged class of 0 to the highest and 3 for the lowest. To verify that the current CPU can execute that code. In addition, Linux uses only levels 0 and 3, which correspond to the kernel state and user state respectively. 2.2 Segment Descriptor (Segment descritor)

Each "segment" is selected by the segment selector and is represented by a segment descriptor that represents the various properties of the segment, including the starting position of the segment, the length of the segment, and so on. The segment descriptor is 8 Byte in size and is stored in the GDT (Global descritor table) or the LDT (Local descritor table). Both the GDT and the LDT have corresponding registers for storage, named GDTR and LDTR respectively. As to which deposit the descritor is stored in, depending on the segment selector TI this bit.

The following table illustrates the composition of the segment descriptor:

                           Segment Descritor

byte/     3       |       2       |       1       |       0       |
   /              |               |               |               |
  |7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|
  +-------------------------------+-------------------------------+
7 |  BASE (24-31)  | G| d|.|.| LIMIT () | p| dpl| s| TYPE  |  BASE (16-23)  | 4
  +-------------------------------+-------------------------------+
3 |        BASE (0-15)             |       LIMIT (0-15)             | 0
  +-------------------------------+-------------------------------+

  Where the 4th bit of the 6th byte is the AVL

which

• Base

  The linear address of the first byte of the paragraph.

• G

  granularity, grain size. If the bit is 0, indicates the size of the segment in bytes, or 1, indicating the size in 4096 bytes (the size of a memory page).

• D
  • 1:address is 32-bit
  • 0:address-bit.
• Avl

  This bit is not used in Linux.

• Limit

  The offset of the last memory unit of the segment is saved in Limit, in other words, it indicates the size of the segment. Depending on the G, the size of the paragraph is also different:

  • G = 1: Segment size range is 1 Byte ~ 1 MB (2^20 * 1 B)
  • G = 0:4k ~ 4GB (2^20 * 4096 B)
• S

  Sytem flag, if the flag is set, indicates that the key data structure, such as the LDT, is stored in the paragraph, otherwise the segment is a normal data segment or code snippet.

• TYPE

  Indicates the type of the segment, as well as access rights.

• Dpl

  Descriptor Privilege Level: Used to restrict access to this segment, indicating the minimum permissions required to access the segment. For example, the dpl=0 segment is accessible only cpl=0 time, while the dpl=3 segment can be accessed by any PL CPU.

• P

  Segment-present flag: A value of 1 indicates that the segment exists in memory, otherwise the segment exists in SWAP.

Quick access to 2.3 segment descriptors

As mentioned earlier, the segment selector length is a bit, and its composition is as follows:

       Segment Selector

byte/     1       |       0       |
   /              |               |
  |7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|
  +-------------------------------+
  |          Index          |.|. |
  +-------------------------------+
                             ^  ^
                             |  |
    TI  ---------------------+  |   (0--GDTR, 1--LDTR)
   RPL  ------------------------+   (Required Privilege level)

which

• Index

  Index indicates the position of the segment descriptor that the segment selector points to in the GDT or the LDT.

• TI

  Table Indicator, indicates whether the segment selector is a GDT or a LDT.

  • TI = 0:gdt in.
  • TI = 1:ldt in.
• RPL

  Requestor Privilege level, indicates the CPL of the CPU when the segment selector is loaded.

By using TI and Index in the segment selector, you can find the appropriate segment descriptor. In addition, the 80X86 processor provides a non-programmable register for each segment selector register that holds the segment descriptor corresponding to the selector. Each time the segment selector is loaded in the segment selector register, the corresponding segment descriptor is automatically loaded into the corresponding non-programmable register for easy access.

Gdt/ldt Segment
+-------------+                    +---------------+
|       NULL | +----------->|. #.%#%---.+ +. |<----------+
+-------------+       |           |+.++* #m #-m*-+m | |
+----> |           Segment Descriptor 1 |-------+ |**#+#%--m#%++-.| |
| +-------------+                    |-..           #*+% #m%m-%+ | |
|         |                    2 | |.. +-- .+..           -++  | |
| +-------------+                    +---------------+           |
|         |                                                3 | |
| +-------------+                                                |
|         |   .                                                | |
| +-------------+                                                |
|         |   .                                                | |
| +-------------+                                                |
| |
| |
| ---------------            |
| Segment Register----/non-programmable registers \----|
| +-------------------+            /  +------------------+    \     |
+---|           Segment Selector |     (   |     Segment Descriptor | )----+
+-------------------+            \  +------------------+    /
----\               /----
---------------

2.4 Conversion of the logical address to the linear address

The conversion of the logical address to the linear address is done by the segmentation UNIT, which requires the following steps:

• Select GDT or LDT according to Selector->ti (the table below is selected as DT).
• Descriptor = DT + Selector->index * 8

  Because each segment descriptor is 8 byte in size, Index * 8 gets the segment descriptor.

• LINEARADDR = descriptor->baseaddr + selector->offset

Use a pseudo-code to represent:

addr_t logical_2_linear (Seg_selector selctor)
{
    descriptor_table dt = NULL;
    Seg_desc desc = NULL;

    /* Step 1 */
    Switch (selctor->ti) {
    Case 0: {
        DT = GDT;
        Break
    }
    Case 1: {
        DT = LDT;
        Break
    }

    Default
        DT = GDT;
        Break
    }

    /* Step 2 */

    desc = dt + Selector->index * 8;

    /* Step 3 */
    return desc->base + selector->offset;
}

80X86, the first two steps of the conversion process can be omitted due to the introduction of a non-programmable register for the segment descriptor corresponding to the bucket selector. 3 Memory-segment management in Linux

Linux prefers in-memory page management to memory-segment management. 80x86, you can use paragraph management when you need it. On-the-paragraph management I did not look carefully, only know:

• Linux memory-segment management, for the kernel space and user space, respectively, the memory is divided into code snippets and data segments, a total of four segments.
• The base of each segment is 0, so that Selector->offset is actually the linear address of the memory.

Other things, see when you meet. 4 page-managed on the hardware

The PAGING UNIT is responsible for translating the linear address into a physical address. Linear addresses are subdivided into pages by a fixed length.

Here are some basic concepts, as follows:

• PAGE Framge

  Refers to the space divided by the Page Size of RAM.

• PAGE

  Refers to the data on a page frame or disk.

• Page tables

  The page table, which is stored in main memory, is initialized by the kernel. A page table is used to map a linear address to a physical address.

4.1 General Paging

As mentioned earlier, the linear address is a length of three bits, it can be divided into three parts, as shown in:

byte/     3       |       2       |       1       |       0       |
   /              |               |               |               |
  |7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|
  +-------------------------------+-------------------------------+
  |    Directory      |      Table        |        Offset         |
  +-------------------------------+-------------------------------+

The conversion of a linear address to a physical address requires two steps, which requires two tables: page Directory and Page table. Each active process must have a page Directory assigned, but the corresponding page frame in page Table does not have to be allocated immediately, and the Pages frame is allocated when needed.

The physical address of the Page Directory of the process is stored in the control register CR3, so that the conversion of the linear address to the physical address can be divided into the following steps:

• Depending on the directory in the linear address, you can find the page Table from the page directory that CR3 points to
• Depending on the table in the linear address, you can find the corresponding page Frame from the page table found earlier
• Based on Offset in the linear address, locate the relative position in the pageframe from the Page Frame that was found earlier.

The entire process is as follows:

Linear Address
-------------+---------------+---------------+---------------+
BYTE/3 |       2 |       1 | 0 |
/--------------+---------------+---------------+---------------+
/7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|
+-------------------------------+-------------------------------+
|      Directory |        Table | Offset |
+--------+----------------------+----------------+--------------+
|                |       | Page Frame
|                |      | +---------+
|                |      |         | |
|                |      | +---------+
|       | Page +----->| ######  |
|      | Table +---------+
|    |         +---------+        | |
|    Page |         |        | +---------+
|    Directory |         +---------+        | |
| +------------+ +--->| ###### |------> +---------+
|            |       | +---------+
|         +------------+       | |
+----> | ########## |-----> +---------+
CR3 +------------+
+--------+                | |
| |--------------> +------------+
+--------+

When the page is aligned in 4 K, the page Frame size in memory is 4K, at which point the low 12 bits of each pageframe address are 0, and the Catalog Items and page table entries in page directory and page tables are 12 bits low For the control word, the shape is as follows:

BYTE/3 |       2 |       1 | 0 |
/              |               |               | |
|7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|
+---------------+---------------+-------+-----+-+-+-+-+-+-+-+-+-+
|     High bit | | | p| | | p| p| | | |
| Page Directory Entry Addr | avail| G| s| d| a| c| w| u| w| p|
|     Page Table Entry Addr | | | | | | d| t| | | |
+---------------------------------------+-----+++++++++++++++++++
| | | | | | | | |
| | | | | | | | +---> Present
| | | | | | | +-----> Writable
| | | | | | +-------> User/supervisor Flag
| | | | | +---------> Write-through
| | | | +-----------> Cache Disable
| | | +-------------> accessed
| | +---------------> Dirty
| +-----------------> Page Size Flag
+-------------------> Global Flag

which

• Present Flag

  The position indicates that the item (either a catalog item or a page table entry) is in main memory, and that zeroing is not in main memory. If the bit is found to be 0 when an address type conversion is made to a directory entry or page table entry, the PAGING UNIT will store the linear address on CR2 and then trigger page Fault.

• Writable

  Indicates whether the item is writable, 1-time writable.

• User/supervisor Flag

  Indicates the permissions required to access the page.

• PWT

  Buffer-related, indicating whether Write-through is involved later.

• PCT

  When set to 1, indicates that the buffer memory is turned off.

• Accessed

  Indicates that the item has been visited.

• Dirty

  Used only for page table entries, indicating that the page Frame was written.

• Page Size Flag

  For catalog items only, when the value is 0 o'clock, the frame size in the page table item that the catalog item points to (page table) is 4K. And when the value is 1 o'clock, the size of the page table entry becomes 4M.

• Global Flag

  Used only for Page table items (Entry), which is 1, indicating that the page is a global page.

4.2 Expand Paging

Starting with Pentium, Intel introduced the expanded paging (Extended pageing), expanded paging to change the page size to 4 M, in which case the first 10 bits of the linear address are still catalog items, but the middle page Table is removed, OFFSET from the original 12 bits into 22 bits. As shown in the following:

Linear Address
-------------+---------------+---------------+---------------+
BYTE/3 |       2 |       1 | 0 |
/--------------+---------------+---------------+---------------+
/7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|7 6 5 4 3 2 1 0|
+-------------------------------+-------------------------------+
|                Directory | OFFSET |
+--------+-------------------------+----------------------------+
| |
|             | 4MB Page
|        | +----------------+
|        |                | |
|        | +----------------+
|        |                | |
|        | +----------------+
| Page +------->| #############  |
| Directory +----------------+
|                +------------+              | |
|            |              | +----------------+
|                +------------+              | |
+----> | ########## |------------->+----------------+
CR3 +------------+
+--------+                | |
| |--------------> +------------+
+--------+

The difference between expanding the paging and the previous normal paging is:

• In the extended paging, the PS in the TOC item is 1, and the normal paging is 0.
• Expansion paging has no page table, and the following 22 bits are offset.

5 in-memory page management in Linux

In order to maintain the portability of Linux, the page management in Linux, is sealed one layer, one layer, one layer, one layer, another layer.
The virtual addresses used in the Linux kernel (Linear address) are divided into four tiers for portability reasons: page Global directory, page Upper directory, page M Iddle Directory, and Page Table. These layers, together with the last page offset, form the linear address of Linux, as shown in:

|<------------------------------Bits_per_long--------------------------------->|+---------------+------- --------+---------------+---------------+----------------+|    PGD        |      PUD      |     PMD       |      PTE      |    Offset      |+---------------+---------------+---------------+---------------+----------------+                |               | | |<-page_shift->|                |               |               | <-----------pmd_shift-------->|                |               | <----------------pud_shift------------------->|                | <--------------------------Pgdir_shift----------------------->|

The linear address space of a process is divided into both parts:

• 0x00000000 ~ 0xbfffffff:

  Both kernel space and user-space processes are accessible.

• 0xc0000000 ~ 0xFFFFFFFF:

  Limited to kernel-state process access only.

Author:yangyingchao, 2010-09-10

Reference: http://blog.163.com/vic_kk/blog/static/4947052420108104224228/

Linux Memory addressing