Resolve high-end memory under Linux systems

Linux Kernel address space partitioning

Typically, the 32-bit Linux kernel virtual address space is divided into 0~3g for user space, 3~4g for kernel space (note that the kernel can use a linear address of only 1G). Note that this is a 32-bit kernel address space partition, and 64-bit kernel address space partitioning is different.

Typically, the 32-bit Linux kernel virtual address space is divided into 0~3g for user space, 3~4g for kernel space (note that the kernel can use a linear address of only 1G). Note that this is a 32-bit kernel address space partition, and 64-bit kernel address space partitioning is different.

The origin of high-end memory in Linux kernel

When the kernel module code or the thread accesses the memory, the memory address in the code is the logical address, but corresponds to the real physical memory address, needs the address one-to-one mapping, for example the logical address 0xc0000003 corresponding physical address is 0x3,0xc0000004 corresponding physical address is 0x4, ..., The relationship between a logical address and a physical address is

Physical Address = Logical Address –0xc0000000: This is the address translation relationship of the kernel address space, note that the kernel's virtual address is "high-end", but the physical memory address of TA mapping is at the low end.

In fact, "the Kernel direct mapping space" can not reach 1G, but also have to leave some linear space for "kernel dynamic mapping space."

As a result, Linux requires that the "kernel Direct mapping Space" map up to 896M of physical memory.

For high-end memory, the corresponding page can be obtained by alloc_page () or other functions, but in order to access the actual physical memory, the page must be converted to a linear address (why? Think about how MMU accesses physical memory), that is, we need a page that corresponds to the high-end memory Find a linear space, a process called a high-end memory map.

Assuming that the above simple address mapping relationship, then the kernel logical address space access to 0xc0000000 ~ 0xFFFFFFFF, then the corresponding physical memory range is 0x0 ~ 0x40000000, that is, only 1G physical memory access. If the machine installs 8G physical memory, then the kernel can only access the former 1G physical memory, the later 7G physical memory will not be able to access, because the kernel's address space has already mapped all to the physical memory address range 0x0 ~ 0x40000000. Even if 8G of physical memory is installed, then the physical address of the 0x40000001 memory, how the kernel to access it? The code must have a memory logical address, the 0xc0000000 ~ 0xFFFFFFFF address space has been used up, so unable to access the physical address 0x 40000000 of memory after.

It is obvious that the kernel address space 0xc0000000 ~ 0XFFFFFFF is not all used for simple address mapping. So the x86 architecture divides the kernel address space into three parts: ZONE_DMA, Zone_normal, and Zone_highmem. Zone_highmem is the high-end memory, which is the memory of the concept of high-end memory.

In the x86 structure, three types of zones (calculated starting from 3G) are as follows:

16MB of ZONE_DMA Memory start

Zone_normal 16MB~896MB

Zone_highmem 896MB ~ End (1G)

High-end memory refers to memory with a physical address greater than 896M. For such memory, it cannot be mapped in the kernel direct mapping space.

Why?

Because the "Kernel direct mapping Space" can only be from 3G to 4G, only 1G physical memory can be mapped directly, and there is nothing to do with the physical memory greater than 1G.

There are three ways to high end memory mappings:

1, map to "kernel Dynamic mapping Space"

This approach is simple because it is possible to get a page from a high-end memory (see the implementation of the Vmalloc) by Vmalloc (), while the "kernel Dynamic mapping Space" is requesting memory, so that high-end memory may be mapped to "kernel dynamic mapping Space".

2. Permanent Kernel mapping

If you get the high end memory page by Alloc_page (), how do you find a linear space for it?

The kernel specifically leaves a linear space, from Pkmap_base to Fixaddr_start, for mapping high-end memory. On the 2.4 kernel, this address range is 4g-8m to 4g-4m. This space is called "Kernel permanent Mapping Space" or "permanent kernel mapping Space"

This space and other space use the same page table of contents, for the kernel, is swapper_pg_dir, for ordinary processes, through the CR3 register point.

Typically, this space is 4M in size, so you just need a page table, and the kernel uses it to pkmap_page_table to find the page table.

by Kmap (), you can map a page to this space to

Because this space is 4M in size, you can map up to 1024 page at a time. Therefore, for the unused page, should be released in a timely manner from this space (also in addition to mapping off is the system), through the Kunmap (), can be a page corresponding to the linear address from this space released.

3. Temporary mapping

The kernel retains some linear space between the Fixaddr_start and the fixaddr_top for special needs. This space is called "Fixed map space"

In this space, there is a part of the temporary mapping for high-end memory.

This space has the following characteristics:

1, each CPU occupies a piece of space

2, in the space occupied by each CPU, and divided into several small space, each small space size is 1 page, each small space for a purpose, these purposes are defined in the kmap_types.h in the Km_type.

When you want to do a temporary mapping, you need to specify the purpose of the mapping, according to the mapping purposes, you can find the corresponding small space, and then the address of this space as a mapping address. This means that a temporary mapping will cause the previous mappings to be overwritten.

Temporary mappings can be implemented through Kmap_atomic ().

The following figure is a simple and simple representation of how to map high-end memory

The Linux memory linear address space size is 4GB, divided into 2 parts: the user space portion (usually 3G) and the kernel space portion (usually 1G). Here we focus mainly on the kernel address space section.

The kernel manages all the physical memory through the kernel page global catalog, and because the 3G space is used by the user, the first 768 items (just 3G) of the kernel page global catalog are 0, except 0 and 12, and the latter 256 (1G) is used to manage all the physical memory. The kernel page global catalog is statically defined as a swapper_pg_dir array at compile time, and the array is stored from the physical memory address 0x101000.

The

is visible from the diagram, where the kernel linear address space section begins with the Page_offset (usually defined as 3G), in order to load the kernel into memory, the 8M linear address from the Page_offset is used to map the physical memory address of the kernel (or the virtual address of the kernel is from Page_ offset); Next is the Mem_map array, where the starting linear address of the MEM_MAP is related to the architecture, for example, for the UMA structure, the 16M Physical address space corresponding to the 16M linear address space from the Page_offset is the DMA area, MEM_ The map array usually starts at the linear address of the page_offset+16m, and the linear address space from Page_offset to vmalloc_start–vmalloc_offset maps directly to the physical memory space (one by one corresponds to the mapping, physical address <== > Linear address-page_offset), the size of this area is related to the actual amount of physical memory that the machine has, and here Vmalloc_offset is 8M on the X86, which is used primarily to prevent cross-border errors; On systems with smaller memory, the remaining linear address space ( and subtract the blank area, or Vmalloc_offset, by the Vmalloc () function to map the discontinuous physical address space to a continuous linear address space, on a system with large memory, Vmalloc () using the Vmalloc_start to VMALLOC_ End (also known as Pkmap_base minus 2 pages of blank page size page_size (explanation vmalloc_end)) of the linear address space, at this time the remaining linear address space (and then subtract 2 pages of blank area that is Vmalloc_offset) Can be divided into 2 parts: the first part from Pkmap_base to Fixaddr_start used by the Kmap () function to establish a permanent mapping of high-end memory; the second part, from Fixaddr_start to Fixaddr_top, This is a fixed size temporary mapped linear address space (reference: Fixed virtual addresses are needed for subsystems that need to know the virtual addressing at compile Time such as the APIC), on the X86 architecture, Fixaddr_top is statically defined as 0xffffe000, at which point the fixed size space ends at the end of the entire linear address space, which is computed at compile time and stored in __ In the fixaddr_size variable.

The

is due to the presence of the Vmalloc () usage area, Kmap () usage area, and fixed large cell (kmap_atomic () use area) that limit the size of the Zone_normal area, because the kernel requires these functions at run time. So at least vmalloc_reserve the size of space in the linear address space. The size of the Vmalloc_reserve is related to the architecture, and the vmalloc_reserve is defined as 128M on X86, which is why zone_normal size is usually 16M to 896M.