Deep understanding of Linux Kernel notes-chapter NO. 02: Memory addressing. MD

2.1 Memory Address

• Three different addresses for 80x86 microprocessors:
  
  • Logical address: The address used to specify an operand or an instruction in a machine language instruction
    
    • Each logical address has a segment and offset, which indicates the distance between the point where the segment starts and the actual address.
  • Linear address: Also becomes a virtual address, is a 32-bit unsigned integer that can be used to represent addresses up to 4G
    
    • Linear addresses are usually expressed in hexadecimal, with values ranging from 0x00000000 to OXFFFFFFFF
  • Physical Address: For memory chip-level memory unit addressing
    
    • They correspond to electrical signals sent from the address pin of the microprocessor to the memory bus
    • The physical address is represented by a 32-bit or 36-bit unsigned integer

• Memory Control Unit (MMU): Logical address = Physical Address

  • First, a logical address is converted to a linear address by a hardware circuit called a segmented unit.
  • The linear address is then converted to a physical address by the hardware circuit of the paging unit.

• Memory Arbiter:

  • Introduction Reason
    
    • In a multiprocessor system, all CPUs share the same memory, which means that the RAM chip can be accessed concurrently by a separate CPU
    • The read or write operation on the RAM chip must be executed serially
    • Even if the memory arbiter is used on a single processor, there are concurrent operations on the DMA and CPU in a single-processor system
  • Position: inserted between the bus and each RAM chip
  • Function: If a RAM chip is idle, grant a CPU access, if busy, delay the CPU access

2.2 Segmentation in Hardware

• Intel processors perform address translation in two ways:
  
  • In order to maintain the processor's compatibility with earlier models, and let the operating system bootstrap
  • Protection mode

2.3 Segment Selector and segment registers

• The composition of a logical address: The offset of a segment identifier and a relative address within a specified segment
  
  • The segment identifier is a 16-bit long field, called the segment selector
  • Offset is a 32-bit long field
• 6 Segment Registers:
  
  • CS: Code segment Register, point to segment containing program directives
    
    • Another important feature: contains a two-bit field to indicate the current privilege level (CPL) of the CPU: 0 for the highest priority, and 3 for the lowest priority
  • SS: Stack segment register, pointing to the segment containing the current program stack
  • DS: Data segment register, point to contain static data or global data segment
  • Its Karma 3 is a general segment register, can point to any data segment

2.4 Segment Descriptor

• Each segment has a 8-byte segment descriptor representation that describes the characteristics of the segment
  
  • The segment descriptor is stored in the Global Descriptor List (GDT) or in the local Descriptor descriptor (LDT)
• Typically only one GDT is defined, and each process needs to create an additional segment to have its own LDT
  
  • GDTR: Storing the address and size of the GDT in main memory
  • LDTR: Store the LDT address and size that is currently being used
• Segment Descriptor Field table (table 2-1,p43)

The

field name	description
Base	contains the first Linear address of Byte
G	granularity flag: 0, Segment size in bytes; otherwise in multiples of 4KB
Limit	holds the offset of the last memory cell in the segment, which determines the length of the segment
S	system flag: Cleared 0-This is a system segment; otherwise a normal code snippet or data segment
type	describes the type characteristics of a segment its access Permissions
DPL	descriptor Privilege Level field: The segment set to 0 can only be accessed if the CPL is 0 o'clock, and the segment of DPL 3 is accessible for any CPL value
P	segment-present flag: 0-segment is not currently in main memory; Linux always sets this flag (47 bits) to 1 because it never swaps the entire segment to disk
D or b	is called a B or D flag, depends on the code snippet or data segment, if the offset address is 32 bits, is basically set to 1, if 16 bits, then is cleared to 0
AVL flag	can be used by the operating system, but Ignored by Linux

-There are several different types of segments and their corresponding segment descriptors, types that are widely used in Linux:
-Code Snippet Descriptor: Indicates that this descriptor represents a piece of code that can be placed in a GDT or LDT with the S flag 1 (non-system segment)
-Data Segment Descriptor: Indicates that the segment descriptor represents a data segment that can be placed in a GDT or LDT, which is identified as 1, and the stack segment is implemented by a general data segment
-Task Status Segment Descriptor (TSSD): This descriptor represents a task status segment, which means that the segment is used to hold the contents of the processor register, which can only appear in the GDT
-The value of the Type field is 11 or 9, depending on whether the process is running on the CPU, and the S flag for that descriptor is set to 0
-Local Descriptor Descriptor: Indicates that the segment descriptor represents a segment containing the LDT, which appears only in the GDT, with the type value 2,s 0

2.5 Quick Access Segment descriptor

• The 80x86 processor provides an additional non-programmable register (cannot be set) and can be used by 6 programmable segment registers

  • Each non-programmable register contains a 8-byte segment descriptor that is specified by the selector in the segment selector in the corresponding segment register
  • Whenever a segment selector is loaded into a segment register, the corresponding segment descriptor is loaded into the corresponding non-programmed CPU register by memory
    
    • Then, for that segment of the logical address can not access the main memory of the GDT or the LDT, the processor only need to directly reference the storage Segment Descriptor CPU Register can be
    • It is only necessary to access the GDT or LDT when the contents of the segment register change

• Segment Selector Field

Field name	Description
Index	Specifies the corresponding segment descriptor placed in the GDT or LDT
TI	0: The segment descriptor is in the GDT; 1: In the LDT
RPL	Requester Privilege level: Indicates the current privilege level of the CPU when the segment selector is loaded into the CS register, and optionally weakens the processor's privileged level when accessing the data segment

-Since the segment selector is 8 bits long, its relative address in the GDT or LDT is the highest 13-bit value with the segment selector multiplied by 8.

For example, if the GDT is in 0x200020000, and a segment selector specifies an index number of 2, the corresponding segment descriptor address is 0x00020000 + (2 x 8), or 0x00020010
(The base site of the GDT is stored in the GDTR register)

• GDT Related:
  
  • The first entry is always set to 0, which ensures that the logical address of the empty segment selector is considered invalid and therefore causes a processor exception
  • The maximum number of segment descriptors that can be saved in the GDT is 8191, which is 2 of 13 parties minus 1

2.6 Segment Unit

• The following procedure is performed by the staging unit:
  
  • Check the Ti field of the segment selector first to determine which descriptor table the segment characters in
  • Calculates the address of the segment descriptor from the index field of the segment selector, the value of the index field multiplied by 8, and the result is added to the contents of the GDTR or LDTR register
  • Adding the offset of the logical address to the value of the base field of the segment descriptor yields a linear address

• The process of translating a logical address into a physical address

• With the non-programmable registers associated with the segment register, only the first two operations need to be performed if the contents of the segment register are changed to a string.

2.7 Segments in Linux

• Segments in the 80x86 microprocessor encourage programmers to divide their programs into logically related entities
  
  • Example: subroutine or global vs. local data area
• In fact, fragmentation and paging are somewhat redundant because they can divide the physical address space of a process
  
  • Segments can assign a different linear address space to each process
  • Paging can map the same linear address space to different physical spaces

• Linux is more inclined to use paging because:

  • Memory management becomes simpler when all processes use the same segment register value, i.e. they can share the same linear address
  • One of the goals of the Linux design is that it can be ported to most popular processors, and then RISC architectures have limited support for fragmentation
    -Version 2.6 Linux only needs to be segmented with 80x86 structure
  • All Linux processes running in the user state are addressing instructions and data in a pair of identical fields, that is, the user data segment and the user code segment
  • All Linux processes running in the kernel state are addressed with a pair of identical segments for instructions and data, i.e. kernel data segments and kernel code snippets
  • The value of the Segment Descriptor field for the four major Linux segments

• Segment Selection Fu Yuhong __user_cs,__user_ds,__kernel_cs and __kernel_ds are defined separately

  In order to address the kernel snippet, the kernel only needs to load the value generated by the __KERNEL_CS macro into the CS segment register

• The same logical address can be used by all processes under user and kernel state

  Because the linear address associated with the segment starts at 0, it reaches 2 of the 32-time-square minus 1 addressing limit.

• In Linux, the logical address is consistent with the linear address, that is, the value of the offset field of the logical address is always the same as the value of the corresponding linear address.

  Because all the fields start with 0x00000000

• As long as the current privilege level is changed, some segment registers must be updated accordingly, such as the DS segment Register (also the SS segment register):

  • When cpl=3 (user state), the DS register must contain segment selectors for user data segments
  • When Cpl=0, the DS segment register must contain the segment selector for the kernel data segment
• When you save a pointer to an instruction or data structure, the kernel does not need to set the segment selector for the logical address at all, because the CS register contains the current segment selector
  
  • For example:
    
    • When the kernel tune executes a call assembly instruction to invoke a function, the directive only specifies the offset portion of its logical address, because the segment selector is already hidden in the CS register.
    • Because the "kernel-state Execution" segment is only a snippet, defined by the macro __kernel_cs, it is sufficient to load the __kernel_cs into CS as soon as the CPU switches to the kernel state.
  • The case in the example is also suitable for pointers to kernel data structures and pointers to user data structures

2.8 Linux GDT

• Single-processor systems have only one GDT, one for each CPU in the processor's system
  
  • All GDT is stored in the cpu_gdt_table array.
  • The addresses of all GDT and their size are stored in the CPU_GDT_DESCR array

• GDT table:

• Description of the middle selector in the GDT table:

  • Code Snippets and data segments in user and kernel states total four Kernel_code, Kernel_data, Kernel_code, and User_data
  • Task status Segment (TSS), each processor has only one
    
    • each TSS corresponding linear address space is a small subset of the corresponding linear address space of the kernel data segment
    • All TSS are stored sequentially in the INIT_TSS array
      
      • Special Note: The base field of the TSS descriptor for the nth CPU points to the nth element of the INIT_TSS array
    The
    • G flag is clear 0, and the Limit field is set to 0xeb because the TSS segment is 236 bytes long
    The
    • Type field is set to 9 or 11 (32-bit TSS is available), and DPL is set to 0 because the process under the user state is not allowed to access the TSS segment
  • 1 segments that contain the default local descriptor, which is typically a segment shared by all processes
  • 3 Local thread bucket (TLS): This mechanism allows multithreaded programs to use a maximum of 3 thread-local data segments
    
    • system calls Set_thread_area () and Get_thread_area () Create and revoke a TLS segment for an executing process, respectively
  • 3 segments related to Advanced Power Management (AMP)
    
    • because the BIOS code uses a segment, when the Linux APM driver calls the BIOS letter Number to get or set the state of the APM device, you can use a custom code snippet and data segment
  • 5 segments related to the Plug and Play (PnP)-Capable BIOS service program
    
    • in the former case, like the previous As in the case of 3 segments related to AMP, you can customize code snippets and data segments
  • Special TSS segments used by the kernel to handle "double error" Exceptions
• Each processor in the system has a GDT copy, except in rare cases, where all of the GDT's replicas hold the same table entry
  
  • First, each processor has its own TSS segment, so its corresponding GDT is different
  • Second, only a few of the GDT may depend on the process the CPU is executing (LDT and TLS segment descriptors)
  • Finally, in some cases, the processor may temporarily modify an item in the GDT copy
    
    • This happens, for example, when you invoke the BIOS history of APM.

Linux LDT

• Most user-configured Linux programs do not use a local descriptor descriptor, so the kernel defines a default LDT for most processes to share
  
  • The local default LDT is stored in the Default_ldt array, which contains 5 items, and the kernel uses only two of them effectively:
    
    • Call gate for IBCs execution files
    • Call gate for solari/x86 executable file
      
      • A mechanism provided by the 80x86 microprocessor when invoking a gate is used to change the privileged level of the CPU when invoking a predefined function, as detailed in the Intel documentation
• In some cases, the process still needs to create its own local descriptor
  
  • Useful for some programs, such as the wine program, which perform segment-oriented Microsoft Windows applications
  • Modify_ldt () system call allows a process to create its own LDT
    
    • Any custom local descriptor that is created by Modify_ldt () still needs his own segment
    • The LDT table entries in the GDT copy of the CPU are modified when the processor starts executing the process of owning a custom LDT.
  • User-state programs can also use Modify_ldt () to allocate new segments, but the kernel never uses these segments, nor does it know the corresponding segment descriptors, because these segment descriptors are included in the process-specific LDT.

Paging through the paging mechanism hardware

• Paging Unit (paging unit) converts a linear address to a physical address
  
  • One of the key tasks is to compare the requested access type with the access to the linear address, and if this access is not valid, a fault is generated.
• A few important concepts:
  
  • Pages: Linear addresses are divided into groups of fixed-length units, called pages, and typically, pages are both a set of linear addresses and data contained in this set of addresses
    
    • The continuous linear address within the page is mapped to a contiguous physical address
    • The kernel can specify the physical address of a page and its access rights without having to specify access to all of the linear addresses contained in the page
  • Page box: Paging unit divides all ram into a fixed-length page box (physical page), and the length of a page box is consistent with the length of a page
    
    • A page box is a part of main memory and therefore a storage area, and a page is just a block of data that can be stored on any page box or disk
  • Page table: The data structure of linear address mapping to physical address becomes a page table, the page table is stored in main memory, and must be initialized by the kernel before paging unit is enabled
• All 80x86 processors support paging and are started by setting the PG flag of the CR0 register
  
  • The PG is 0, and the linear address is interpreted as a physical address.
  • PG is 1, start paging mechanism

General page Pagination

• Paging unit for Intel processors processes 4k pages
• A 32-bit linear address is divided into 3 domains:
  
  • Directory: Up to 10 bits, which determine the directory entry in the page directory, and the catalog entry points to the appropriate page table
  • Table (page table): Intermediate 10-bit, which determines the table items in the page table, the table item contains the physical address of the page box
  • Offset (offsets): Minimum 12 bits, it determines the relative position within the box, because there are 12 bits, so each page contains 4096 bytes of data
• Linear address Translation is done in two steps, each based on a conversion table:
  
  • The first conversion table is called a page directory conversion table (pages directory)
  • The second type of conversion table is called a page table

• Page Directory:

  • The physical address of the page directory being used is stored in the control register CR3
  • Each active process must have a page directory assigned to him, but it is more efficient to allocate RAM to the page table only if the process actually requires a page table

• Page catalog Items and page table entries have the same structure, and each item contains the following fields:

  • Present Logo:
    
    • 1: The specified page is in main memory
    • Clear 0: Not in main memory; If the flag is zeroed when performing an address translation, the paging unit stores the linear address in the CR2 and generates a page fault
  • Field containing the maximum 20-bit physical address of the page box
    
    • Because each page box size is 4KB, the physical address must be 4,096 times times, so the physical address minimum 12 bits is always 0
    • If this field points to a page directory, the corresponding page box contains a page table, and if you point to a page table, the corresponding page box contains one page of data
  • Accessed Logo:
    
    • This flag is set whenever the paging unit is addressed to the corresponding page box
    • When the selected page is swapped out, this flag can be used by the operating system
    • Paging unit never resets this flag, but must have an operating system to do
  • Dirty flag: Applies only to page table entries
    
    • This flag is set every time a page box is written
    • When the selected page is swapped out, this flag can be used by the operating system
    • Paging unit never resets this flag, but must have an operating system to do
  • Read/write flag: Access Permissions for page or page tables
  • User/supervisor flag: Privilege level with access page or page table
  • PCD and PWT Flags: How to control how a hardware cache handles a page or page table
  • Page size flag: Applies only to page directory entries, or 1 if the page directory points to a 2MB or 4MB page box
  • Global flag: Apply only to Page table entries
    
    • This flag is referenced in Pentium Pro to prevent frequently used pages from being flushed out of the TLB cache
    • This flag is enabled only if the page global enable (PGE) flag is set on the CR4 register

Extended Paging

• Page Catalog item's page size flag enables extended paging feature
• Extended paging allows page box size to be 4MB instead of 4KB

• Extended paging is used to convert a contiguous linear address of a large segment to a physical address

  • In this case, the kernel can save the kernel and retain the TLB without using the intermediate page table for address translation

• The paging unit divides a 32-bit linear address into two fields:

  • Directory: Up to 10-bit
  • Offset: The remaining 22 digits
• The same directories for extended paging and normal paging are basically the same, except:
  
  • The Page size flag must be set
  • A 20-bit physical address only has a high of 10 bits, because each physical address starts at a point where 4MB is the boundary, so the low 22 bits of this address are 0
• Extended paging common paging coexistence by setting the PSE flag of the CR4 register

Hardware Protection Solutions

• There are only two privileged levels associated with page tables because the privileges are controlled by the USER/SUPERVISOR flag mentioned in the "General paging" section
  
  • The flag is 0: only if the CPL is less than 3 (which means that the processor is in the kernel State for Linux) to address the page
  • The flag is 1: The page is always addressed
• There are only two access permissions for a page: Read and Write
  
  • The Read/write flag for a page directory or page table entry is 0: the corresponding page table or page is read-only
  • Otherwise, it is readable and writable.

Example of a regular page

• Assume that the linear address space of the memory to a running process is 0x20000000 to 0x2003ffff; the space is made up of just 64 pages
  
  • Analysis of the linear address space:
    
    • The linear address high 10 bits (directory fields) start with 2 followed by 0, so the high 10 bits have the same value, all 0x80, pointing to the 129th item in the page directory
      
      • The physical address of the page table in the page directory that must contain the process
      • If no other linear address is assigned to the process, the remaining 1023 items in the careers directory are filled with 0
    • The value in the Middle 10 (table field) is 0-0x03f, so only the first 64 bits of the page table are meaningful, and the remaining 960 are filled with 0
  • Assume that a thread reads bytes in 0x20021406
    
    • The 0x80 of the directory field is used to select the 0x80 directory entry for the page directory
    • The table field 0x21 is used to select the 0X21 table entry for the page table, which points to the page box that contains the page you want
      
      • If the present flag of the page table is 0, the page is not in main memory, and a fault is generated.
      • A fault is generated whenever a linear address is accessed from the linear space 0x20000000 to the 0X2003FFFF address.
        
        • Because these page tables are all filled in for 0, especially their present flags are cleared 0.
    • Offset field 0x406 The byte in the target page box to read offset to 0x406

Deep understanding of Linux Kernel notes-chapter NO. 02: Memory addressing. MD