Jiurl
Date: October 30, 2015
Paging mechanism
Linux (X64CPU) uses virtual memory based on the paging mechanism. Each process has a 256TB (48-bit) virtual address space. Based on the paging mechanism, some parts of the 256TB address space are mapped to physical memory, and some parts do not have any mappings. The virtual address in the 256TB address space is used in the program. Access to physical memory requires the use of a physical address.
Physical Address Physical Address: addresses placed on the addressing bus. On the addressing bus, if it is read, the circuit transmits the data in the physical memory of the corresponding address to the bus in accordance with the value of each bit of the address. In the case of writing, the circuit places the physical memory of the corresponding address on the data bus, based on the value of each bit of the address. The physical memory is addressed and addressable in bytes (8 bits).
Virtual Address: The address in the 256TB virtual address space that is used in the program as a virtual address.
If the paging flag bit in the CPU-related register is set, the CPU automatically converts the virtual address into a physical address according to the information in the corresponding conversion structure (PML4,PDPT,PD,PT) when the machine instruction of the memory operation is executed, completing the instruction. For example "mov Eax,dword ptr[13ff3904ch]", is the address 13ff3904ch place value assigned to register assembly code, 13FF3904CH this address is the virtual address. When the CPU executes this line of code, it discovers that the paging flag bit in the register has been set up, automatically completes the virtual address to physical address conversion, uses the physical address to fetch the value, completes the instruction. For X86-64CPU, the 31st bit of the register CR0 is the paging flag bit, 1 for paging, and 0 for no paging. For Linux after initialization we observed CR0 through QEMU monitor and found that the value of 31st bit is 1. Indicates that Linux is using paging. The control bits in multiple registers are related to paging, which can be referred to in the Intel Software Development Manual for specific details.
The actual virtual address size of the X86-64 CPU is 48 bits (low 48 bits in 64 bits, high 16 bits are not used for address translation), addressing the 256TB address space. X86-64 the actual physical address size of the CPU, different CPUs are not the same, but the maximum is not more than 52 bits. The actual virtual address length and actual physical address length of the CPU can be queried through the machine instruction Cpuid. For example, I queried two computers, the size of the virtual address two computers are 0x30 (16 binary 30 is the decimal 48), the physical address of the size of a computer is 0x24 (16 24 is the decimal 36, 36-bit addressable 64GB), The other computer is 0x27 (16 binary 27 is decimal 39, 39-bit addressable 512GB). For the convenience of the following narrative, we call the actual physical address of the CPU the number of bits m, for example, for a physical address size of 36 bits cpu,m is 36.
Using the paging mechanism, the 256TB address space is divided into fixed-size pages (three sizes, 4KB,2MB,1GB), each page being mapped to physical memory, or nothing mapped. For the general program, only a small portion of the 256TB address space (bucket) maps the physical memory, and a large portion of it is not mapped to anything. Physical memory is also paged to map the address space. For x86-64, there are three sizes of pages, each of which is 4KB,2MB,1GB. The information that the CPU uses to convert virtual addresses into physical addresses is stored in the structure called "page map Level 4", "Page directory Pointer", "Pagedirectory" (pages directory), "pagetable" (page table). Each process has its own set of PML4,PDPT,PD,PT structures. A virtual address in a process that requires a maximum of four levels of conversion to obtain the corresponding physical address.
Physical memory paging, the size of a physical page is 4KB, the No. 0 physical page starts at the physical address 0x0000000000000000, the size is 4KB (0x1000b), and the 1th physical page starts at the physical address 0x0000000000001000. The 2nd page starts at the physical address 0x0000000000002000. Because the size of the page is 4KB, only the m-12bit in the 64-bit address (after the low 12bit) is required for addressing the physical page.
"Pagetable" (page table), a page table with a size of 4KB, placed in a physical page. Consists of 512 8-byte PTEs (page table entries). The size of the page table entry is 8 bytes (64 bits), so there are 512 page table entries in a page table. The contents of each item in the page table (8 bytes, 64 bits) low 12 bits after the M-12 bits are used to put the physical address of a physical page, the low 12bit puts some flags.
"Pagedirectory" (page directory), a page directory size of 4KB, placed in a physical page. Consists of 512 8-byte PDE (page catalog entries). The size of the page catalog entry is 8 bytes (64 bits), so there are 512 page catalog entries in a page catalog. The contents of each item in the page directory (8 bytes, 64 bits) low 12 bits after the M-12 bits are used to put a page table (page table in a physical page) of the physical address, low 12bit put some flags.
The "Pagedirectory pointer" table, a page directorypointer table size of 4KB, is placed in a physical page. consists of 512 x 8-byte pdpte entries. The size of the Pdpte item is 8 bytes (64 bits), so there are 512 pdpte in a pagedirectory pointer table. The contents of each item in the Page directorypointer table (8 bytes, 64 bits) low 12 bits after the M-12 bits are used to put a page directory (page directory placed in a physical page) of the physical address, low 12bit put some flags.
"Page maplevel 4" table, a page map level4 table size of 4KB, placed in a physical page. consists of 512 x 8-byte pml4e entries. The size of the pml4e item is 8 bytes (64 bits), so there are 512 pml4e in a PAGEMAP Level 4 table. The contents of each item in the page Map level4 table (8 bytes, 64 bits) low 12 bits after the M-12 bits are used to put a page directory pointer table (pagedirectory pointer table placed in a physical page) of the physical address, The low 12bit puts some signs.
For x86-64 systems, the physical address of the page Map level 4 table is placed in the CR3 register of the CPU.
The CPU translates the virtual address into a physical address:
A virtual address, a size of 8 bytes (64 bits, actually using only 48 bits), contains information to find the physical address, divided into 5 parts: the 39th to 47th bits of 9 bits (up to 9 bits) is the index in the "Pagemap Level 4" table, and the 30th-to-38th bit of this 9-bit is "page Directorypointer "table in the index, the 21st to the 29th bit is the index in the page directory, the 12th bit to the 20th bit of this 9 bits is the index in the page table, the No. 0 bit to the 11th bit 12 bits (low 12 bits) is the in-page offset. For a virtual address to be converted to a physical address, the CPU first finds the physical page where the "Pagemap level4" table is located, based on the value in CR3, and then finds the corresponding pml4e item based on the 9-bit (highest 9-bit) value of the virtual address from 39th to 47th bits. The PML4E item has the physical address of the "Pagedirectory pointer" table corresponding to this virtual address. With the physical address of the "page directorypointer" table, the corresponding PDPTE item in the "Page directorypointer" table is found, based on the 9-bit value of the 30th to 38th bit of the virtual address as the index. The PDPTE item has the physical address of the page directory that corresponds to this virtual address. With the physical address of the page directory, the 9-bit value of the virtual address as the 21st-to 29th-bit index, the corresponding page directory entry in the page directory, the page catalog entry has the physical address of the page table corresponding to the virtual address. With the physical address of the page table, the 9-bit value of the 12th to 20th bit of the virtual address is used as the index to find the corresponding page table entry in the page table and the physical address of the physical page corresponding to the virtual address in the page table entry. Finally, the minimum 12 bits of the virtual address, that is, the page offset, plus the physical address of the physical page, the virtual address of the corresponding physical address.
A "page Maplevel 4" table has 512 items, the virtual address from 48-bit to the low 9-bit can be indexed just 512 items (2 9 is equal to 512), a "pagedirectorypointer" table has 512 items, the virtual address next 9 bits just index 512. There are 512 items in a page directory, and the next 9 bits of the virtual address are just 512 entries in the index. A page table has 512 items, and the next 9 bits of the virtual address just index 512 items. The lowest 12 bits of the virtual address (2 of 12 equals 4096), as a page offset, can be indexed to 4KB, that is, each byte in a physical page.
The computational process of translating a virtual address into a physical address is that the processor finds the physical page of the current "page map Level4" table by CR3, takes the virtual address from 48 bits to the low 9 bits, and then shifts the 9 bits right 3 bits (because each pml4e item is 8 bytes long, Move right 3 bits equal to 8) get the address in the page, take out the address of the pml4e (8 bytes), found the virtual address corresponding to the "pagedirectorypointer" table where the physical page, and then the same method to find the corresponding page directory of the virtual page of the physical pages, The virtual address corresponds to the physical page of the page table that corresponds to the physical address of the physical page, and finally the physical address of the physical page corresponding to the virtual address plus a 12-bit offset in the page to get the physical address.
A 48-bit pointer that can address the range 0X000000000000-0XFFFFFFFFFFFF,256TB size. This means that a 48-bit pointer can address every byte of the entire 256TB addressing space.
A page table entry is responsible for 4KB of address space and physical memory mapping, a page table 512, which is responsible for the 512*4KB=2MB address space mapping.
A page catalog item that corresponds to a page table. A page directory has 512 items, corresponding to 512 page tables, each page table is responsible for 2MB address space Mapping, 512 page table is responsible for 512*2MB=1GB address space mapping.
A pdpte item that corresponds to a page directory. A "page directorypointer" table has 512 items, which corresponds to 512 page directories, and each page directory is responsible for mapping the 1GB address space. 512 page directories are responsible for 512*1GB=512GB address space mappings.
A "page map Level 4" table entry that corresponds to a "page directory pointer" table. A page Maplevel 4 table has 512 items and corresponds to 512 page directory pointer tables, and each page Directorypointer table is responsible for mapping the 512GB address space. 512 "page Directorypointer" tables are responsible for 512*512GB=256TB address space mappings.
A process has a "page map level4" table. So in page units, a set of PML4,PDPT,PD,PT structures can guarantee the mapping of each page and physical memory in the 256TB address space.
A virtual address during the conversion process, if the corresponding PDPTE item PS bit is found to continue after the conversion step, if found that the corresponding PDPTE item PS bit is 1, then the physical address in this pdpte item, is the virtual address corresponding to a size of 1GB physical page address, The 30 bits after the virtual address are offset within the page of the 1GB physical page, and the physical address corresponding to that virtual address can be obtained.
A virtual address during the conversion process, if the corresponding page catalog entry PS bit 0 will continue after the conversion step, if the corresponding page directory entry of the PS bit is 1, then the physical address in this page directory item is the address of the virtual address corresponding to a size of 2MB physical page, The 21 bits after the virtual address are offset within the page of the 2MB physical page, and the physical address corresponding to that virtual address can be obtained.
Virtual Address 48-bit, addressing 256TB. 48-bit address: 9bit+9bit+9bit+9bit+12bit.
Each level table has a size of 4KB. The table item size is 8B (64-bit), so each table has 512 items.
4-Level Conversion:cr3-> "page directorypointer" table, "page map," table, pages directory---Get physical address.
A PTE item corresponds to a 4KB address range. A PT table corresponds to the 512*4KB=2MB address range.
A PDE item corresponds to a 2MB address range. A PD table corresponds to the 512*2MB=1GB address range.
A pdpte item corresponds to a 1GB address range. A PDPT table corresponds to the 512*1GB=512GB address range.
A pml4e item corresponds to a 512GB address range. A PML4 table corresponds to the 512*512GB=256TB address range.
There are 3 kinds of page size, 4KB,2MB,1GB.
Each process has its own 256TB address space, "0000000000000000-00007fffffffffff" and "ffff800000000000-ffffffffffffffff". Because the CPU ignores the high 16 bits, it is equivalent to "000000000000-7fffffffffff" and "800000000000-ffffffffffff".
Because the CPU has special rules, each of the high 16 bits that are ignored must be equal to the value of the 47th bit (and will do the check, and the non-conformance will throw an exception). The 47th bit of "000000000000-7FFFFFFFFFFF" is 0, so the high 16 bits must also be 0. The 47th bit of "800000000000-FFFFFFFFFFFF" is 1, so the high 16 bits must also be 1.
Each process has its own 256TB address space, which is implemented by each process's own set of PML4,PDPT,PD,PT structures. Because each process has its own set of PML4,PDPT,PD,PT structures, the physical memory mapped for each process's address space is not the same. The values for the same virtual address of two processes (if all have physical memory mappings) are generally different because they tend to correspond to different physical pages.
The low 128tb,0x000000000000-0x7fffffffffff in the 256TB address space is the user address space, and the high 128tb,0x800000000000-0xffffffffffff in the 256TB address space is the system address space. Access to the system address space requires the program to have RING0 permissions.
not to be continued ...
Weibo: HTTP://WEIBO.COM/DDQQPPB
Email: [Email protected]
qq:6291898
Welcome to the Exchange.
Jiurl document-linux virtual memory and paging mechanism (x86-64 bit) (i)
