LINUX logical address, linear address, physical address, and virtual address

First, the concept
Physical addresses (physical address)
For memory chip-level cell addressing, corresponds to the address bus that the processor and CPU are connected to.
This concept should be one of the best understanding of these concepts, but it is worth mentioning that although the physical address can be understood directly into the machine on the memory itself, the memory as a large array of bytes from 0 bytes to the maximum empty byte, and then the array is called the physical address, but in fact, This is just a hardware-to-software image, and the memory is not addressed in this way. So, it is "relative to address bus", is more appropriate, but aside from the physical memory addressing mode of consideration, the physical address directly to the physical memory one by one corresponds, is also acceptable. Perhaps the wrong understanding is more favourable to the metaphysical image.

Virtual Memory (Vsan)
This is a description of the entire memory (not the one that is plugged into the machine). It is relative to the physical memory, can be directly understood as "not straight", "false" memory, for example, a 0x08000000 memory address, it is not on the physical address of the large array 0x08000000-1 that address element;
This is because modern operating systems provide a memory-managed image, virtual memory. The process uses the address in virtual memory, which is assisted by the operating system to "transform" it into a real physical address. This "conversion" is the key to all the issues discussed.
With such a pump, a program can use a much larger address space than the real physical address. (Rob Peter, pay Paul, banks do the same), and even multiple processes can use the same address. Not surprisingly, since the converted physical address is not the same.
--You can decompile the connected program and see that the connector has assigned an address to the program, for example, to invoke a function A, the code is not call a, but called 0x0811111111, that is, the address of function A has been fixed. There is no such thing as a "transform", there is no concept of virtual address, and this does not work at all.
Hold on, the question goes on again, and it won't hold up.

Logical addresses (logical address)
For compatibility, Intel retains the ancient memory management of the segment. A logical address is a machine language instruction used to specify an operand or an address of an instruction. In the example above, we say that the 0x08111111 of the connector is the logical address of the assigned address.
--but I'm sorry to say, it seems to go against the Intel Middle-management, the logical address requirements, "a logical address, is a segment identifier plus a specified segment of the relative address offset, expressed as [segment identifier: Intra-paragraph offset], that is, the 0x08111111 in the above example, should be represented as [a code snippet identifier: 0x08111111], so that it is complete "

Linear address (linear addresses) or also called virtual address (Vsan)
Similar to the logical address, it is also an unreal address, if the logical address is the corresponding hardware platform segment management conversion before the address, then the linear address corresponds to the hardware page-like memory of the pre-conversion address.

-------------------------------------------------------------
The CPU converts an address in a virtual memory space into a physical address, which takes two steps: First, given a logical address (in fact, the offset within the paragraph, this must be understood!!!). ), the CPU uses its segment memory management unit to convert a logical address into a thread address, and then use its page-based memory management unit to convert to the final physical address.

Doing this two conversions is really cumbersome and unnecessary because you can directly draw a linear address to the process. The reason for this redundancy is that Intel is completely compatible.

2. CPU segment memory management, how to convert logical address to linear address
A logical address consists of two parts, the segment identifier: The offset within the segment. A segment identifier is made up of a 16-bit long field called a segment selector. The first 13 bits are an index number. The following 3 bits contain some hardware details,

index number, or directly understand the array subscript-then it always corresponds to a set of the bar, it is what the index of what? This thing is "paragraph descriptor (segment descriptor)", hehe, the specific address of the segment descriptor describes a paragraph (for the "paragraph" of the word understanding, I think of it, took a knife, the virtual memory, cut into a number of sections--). In this way, a number of segment descriptors, a set of an array, called "segment Descriptor Table", so that you can go through the first 13 bits of the segment identifier, directly in the Segment descriptor list to find a specific segment descriptor, this descriptor describes a paragraph, I just the image of the paragraph is not very accurate, because look at what the descriptor inside That is how it is described, to understand what the paragraph is, each segment descriptor consists of 8 bytes, such as:

These things are very complex, although a data structure can be used to define it, but I only care about the same here, that is, the base field, which describes the starting position of a segment of the linear address.

Intel designed the idea that some global segment descriptors are placed in the Global Segment Descriptor List (GDT), and some parts, such as each process's own, are placed in the so-called "local segment Descriptor List (LDT)". When should the GDT be used, and when should the LDT be used? This is represented by the T1 field in the segment selector, = 0, which indicates the use of the LDT with Gdt,=1.

The address and size of the GDT in memory are stored in the GDTR control register of the CPU, while the LDT is in the LDTR register.

A lot of concepts, like tongue twisters. This picture looks more intuitive:

First, given a complete logical address [segment selector: offset address within paragraph],
1, see segment selector t1=0 or 1, know the current to convert is a GDT in the segment, or the LDT in the section, and then according to the corresponding register, get its address and size. We've got an array.
2, take out the segment selector in the first 13 bits, you can find the corresponding segment descriptor in this array, so that it is base, that is, the base address to know.
3, the base + offset, is to convert the linear address.

is quite simple, for the software, in principle, the need to convert the hardware required to prepare the information, you can let the hardware to complete the conversion. OK, let's see what Linux does.

3. Linux Segment Management
Intel requires two conversions, which is compatible, but it is very redundant, oh, no way, hardware requirements to do so, software can only comply, how to have the same formalism.
On the other hand, some other hardware platforms do not have the concept of two conversions, and Linux also needs to provide a high-level image to provide a unified interface. So, the Linux segment management, in fact, just "tricked" the hardware.

According to Intel's original intent, the global use of GDT, each process of its own ldt--but Linux for all processes using the same field for instruction and data addressing. That is, user data segment, user code snippet, corresponding, kernel data segment and kernel code snippet. There is nothing strange about this, it is formality, as we write the year-end summary.
Include/asm-i386/segment.h

1. #define GDT_ENTRY_DEFAULT_USER_CS 14
2. #define __USER_CS (GDT_ENTRY_DEFAULT_USER_CS * 8 + 3)
4. #define GDT_ENTRY_DEFAULT_USER_DS 15
5. #define __USER_DS (GDT_ENTRY_DEFAULT_USER_DS * 8 + 3)
7. #define Gdt_entry_kernel_base 12
9. #define GDT_ENTRY_KERNEL_CS (gdt_entry_kernel_base + 0)
10. #define __KERNEL_CS (GDT_ENTRY_KERNEL_CS * 8)
12. #define GDT_ENTRY_KERNEL_DS (gdt_entry_kernel_base + 1)
13. #define __KERNEL_DS (GDT_ENTRY_KERNEL_DS * 8)

Copy Code

To replace the macro with a value:

1. #define __USER_CS 115 [00000000 1110 0 11]
2. #define __USER_DS 123 [00000000 1111 0 11]
3. #define __KERNEL_CS 96 [00000000 1100 0 00]
4. #define __KERNEL_DS 104 [00000000 1101 0 00]

Copy Code

The square brackets are the 16-bit two representations of the four segment selectors, and their index numbers and T1 field values can also be calculated.

1. __user_cs index= t1=0
2. __user_ds index= t1=0
3. __kernel_cs index= t1=0
4. __kernel_ds index= t1=0

Copy Code

The T1 are all 0, which means that the GDT is used, and then the corresponding 12-15 items (arch/i386/head) in the content of the initialized GDT are seen. S):

1. . Quad 0X00CF9A000000FFFF/* 0x60 kernel 4GB Code at 0X00000000 */
2. . Quad 0X00CF92000000FFFF/* 0x68 kernel 4GB data at 0x00000000 */
3. . Quad 0X00CFFA000000FFFF/* 0x73 user 4GB Code at 0x00000000 */
4. . Quad 0X00CFF2000000FFFF/* 0x7b user 4GB data at 0x00000000 */

Copy Code

As described in the preceding paragraph descriptor table, they can be expanded to find that the 16-31 bits are all 0, that is, the base site of the four segments is all 0.

Thus, given a paragraph offset address, according to the previous conversion formula, 0 + paragraph offset, converted to linear address, you can come to an important conclusion, "under Linux, the logical address and linear address always consistent (is consistent, not some people say the same), That is, the value of the offset field for the logical address is always the same as the value of the linear address. ”

Too many details, such as permission checks for a segment, are ignored. Oh.

In Linux, most processes do not use the LDT, unless you are using wine to simulate Windows programs.

4.CPU of page-memory management

The page-type memory management unit of the CPU, which is responsible for translating a linear address into a physical address. From the management and efficiency point of view, the linear address is divided into fixed-length units, called pages (page), such as a 32-bit machine, the linear address can be a maximum of 4G, the 4KB is a page to divide, this page, the entire linear address is divided into a tatol_page[2^20] A large array of 2 of the 20 pages in a single page. This large array we call the page directory. Each directory entry in the directory is an address-the address of the corresponding page.

Another type of "page", which we call a physical page, or a page box, page frames. It is the paging unit that divides all physical memory into fixed-length management units whose length is typically one by one corresponding to the memory page.

Note here that this total_page array has 2^20 members, each member is an address (32-bit machine, one address is 4 bytes), then to represent such an array, it takes up 4MB of memory space. To save space, a two-level management-mode machine was introduced to organize paging units. The text description is too tired, look at the picture intuitively some:

Such as
1, paging Unit, the page directory is unique, its address in the CPU's CR3 register, is the starting point for address translation. The long march began to grow.
2, the process of each activity, because all have its own corresponding virtual memory (page directory is also unique), then it also corresponds to a separate page directory address. --run a process that needs to place its page directory address in the CR3 register and save it.
3. Each 32-bit linear address is divided into three parts, Surface directory Index (10-bit): Page table index (10-bit): offset (12-bit)
The conversion is performed according to the following steps:
1, remove the page directory address of the process from the CR3 (the operating system is responsible for the scheduling process, the address is loaded into the corresponding register);
2, according to the first ten linear address, in the array, find the corresponding index entry, because the introduction of the level two management mode, the page directory of the item, no longer the address of the page, but the address of a page table. (An array is introduced, and the address of the page is placed in the page table.)
3, according to the middle of the linear address 10 bits, in the page table (also array) to find the starting address of the page;
4, add the starting address of the page and the last 12 bits of the linear address, get the gourd we want eventually;

This conversion process should be said to be very simple. All by hardware, although a number of procedures, but save a lot of memory, or worthwhile. So, simply verify that:
1, such a level two mode is still able to represent the address of 4G;
Page Catalogs total: 2^10, which means there are so many page tables
Each eye table corresponds to: 2^10 page;
Addressable in each page: 2^12 bytes.
or 2^32 = 4GB

2, such a level two mode is really saving space;
That is to say, the page Catalog item and page table item occupy space (2^10 * 4 + 2 ^10) = 8KB. Hey...... How to say it!!!
Red error, mark it, after this discussion in the post ...
By < in-depth understanding of computer systems > Interpretation, two-level mode space savings are achieved from two aspects:
A, if a page table entry in a page table is empty, then the two-page table that refers to it does not exist at all. This shows a huge potential savings, because for a typical program, most of the 4GB virtual address space will be unassigned;
B, only a single-level page table needs to always be in main memory. The virtual memory system can be created when needed, and the page will be paged into or out of the Level two page table, which reduces the stress of main memory. Only the most frequently used level two page table needs to be cached in main memory. --but Linux does not fully enjoy this benefit, and its page table directories and page tables associated with the assigned pages are memory-resident.

It is worth mentioning that, although the page directory and the page table items are 4 bytes, 32 bits, but they are only high 20 bits, low 12-bit shielding for 0--to the page table of the low 12 shield to 0, it is very well understood, because it is just a page size corresponding to each other, everyone into an integer increase. It's a lot easier to calculate. However, why should you also screen the page directory low 12 bit off it? Because by the same token, just block its low 10 bit on it, but I think, because 12>10, so that can make the page directory and page table use the same data structure, convenient.

This post only introduces the principle of general conversion, extended paging, page protection mechanism, PAE mode paging these troublesome things will not be wordy ... You can refer to other professional books.

5.Linux of page-memory management
In principle, Linux only needs to allocate the required data structure for each process, put in memory, and then in the scheduling process, the switch register CR3, the rest of the hardware to complete (hehe, in fact, much more complex, but I only analyze the most basic process).

It says I386 's two-level page management architecture, but some CPUs, there are three levels, or even four-tier architectures, and Linux provides a unified interface for each CPU in order to provide a higher level of image extraction. Provides a four-layer page management architecture that is compatible with these two-, three-, and four-level management architecture CPUs. These four levels are:

Page Global directory PGD (corresponding to the page directory just now)
Page Parent Directory PUD (newly introduced)
Page Intermediate Catalog PMD (also on newly introduced)
Page Table pt (corresponds to the page table just now).

The whole conversion is based on the principle of hardware conversion, just two more times the index of the array, such as:

So, for hardware that uses level two management architecture 32 bits, and now the four-level conversion, how can they work together in a coordinated way? Well, look at this situation, how to divide the linear address it!
From the hardware point of view, the 32-bit address is divided into three parts-that is, how to do not manage the software, the final implementation of hardware, but also only know the three boss.
From the software point of view, due to the introduction of more than two parts, that is, a total of five parts. --to make the hardware of the two-tier architecture aware of the five parts is also very easy, in the Address Division, the page ancestor directory and the page intermediate directory length is set to 0.
In this way, the operating system to see is five parts, hardware or its rigid three parts division, will not be wrong, that is to say, we build a harmonious computer system.

In this way, although superfluous, but considering the 64-bit address, using the four-tier conversion architecture of the CPU, we will no longer set the median two to 0, so that the software and hardware again harmony--The smoke is strong AH!!!

For example, a logical address has been converted to a linear address, 0x08147258, into two, which is:
0000100000 0101000111 001001011000
The kernel divides this address
PGD = 0000100000
PUD = 0
PMD = 0
PT = 0101000111
offset = 001001011000

Now to understand Linux for hardware tricks, because the hardware can not see so-called PUD,PMD, so, essentially requires PGD index, directly corresponds to the PT address. Instead of going to pud and PMD to look up arrays (although they are both in linear addresses, with lengths of 0,2^0 = 1, which means that they all have an array of array elements), how does the kernel properly arrange the addresses?
From the software's point of view, because it has only one, 32-bit entries, it can store exactly the same length of the address pointer as in the PGD. So the so-called first to the PUD, to the PMD to do mapping conversion, it becomes to maintain the original value unchanged, a change of hands on it. In this way, the implementation of "logically points to a pud, and then point to a PDM, but is physically directly pointed to the corresponding PT of this image, because the hardware is not known to have pud, PMD this thing."

Then to the hardware, hardware to this address division, see is:
Page directory = 0000100000
PT = 0101000111
offset = 001001011000
Well, first, according to 0000100000 (32), index in the page directory array, find the address of its element, take its high 20 bits, find the address of the page table, the address of the page table is dynamically allocated by the kernel, and then add an offset, is the final physical address.

LINUX logical address, linear address, physical address, and virtual address