Linux Kernel study note 9-kernel memory management

One page

The kernel uses the physical page as the basic unit of memory management; the memory management unit (MMU) converts the virtual address to the physical

Address, usually in the unit of page processing. MMU manages the tables in the system in the unit of page size.

32-bit system: page size 4 kb

64-bit system: 8 KB page size

The kernel uses the corresponding data structure to represent each physical page in the system:

<Linux/mm_types.h>

Struct page {}

The kernel uses such a data structure to manage all pages in the system. Therefore, the kernel determines whether a page is idle and who owns the page.

The owner may be: user space process, dynamically allocated kernel data, static KernelCodePage cache ......

Every physical page in the system needs to allocate such a struct for memory management.

Zone 2

Linux memory addressing problems:

Some hardware can only use some specific memory for DMA (Direct Memory Access)

Some architectures have a large physical addressing range for their memory. As a result, some memory cannot be permanently mapped to the kernel space.

Generally, 32-bit Linux kernel address space is divided into 0 ~ 3G is the user space, 3 ~ 4G is the kernel space. When the kernel module code or thread accesses the memory,

The memory addresses in the Code are logical addresses, which correspond to the real physical memory addresses. One-to-one address ing is required. Therefore, the kernel space address is 3 ~ 4G,

A maximum of 1 GB memory can be mapped. How can I ask if the memory size exceeds 1 GB!

Due to the limitations of the preceding conditions. Linux divides the kernel space address into three zones:

Zone_dma, zone_normal, and zone_highmem.

Zone_highmem is high-end memory, which is the concept of high-end memory.

 

In the x86 structure, the three types of regions are as follows:

16 MB of zone_dma memory

Zone_normal 16 MB ~ 896 MB

Zone_highmem 896 MB ~ End

Similarly, each zone contains multiple pages to form different memory pools for memory allocation by purpose.

Use the corresponding data structure to represent the partition:

<Linux/mmzone. h>

Struct zone {}

3 get page/memory

Static inline struct page *Alloc_pages(Gfp_t gfp_mask, unsigned int order)

This function is assigned 2 orders to the power of a consecutive physical page and returns the page struct pointer pointing to the first page.

 

Void *Page_address(Const struct page * Page)

Returns the logical address that points to the current location of the given physical page.

Extern unsigned long _ get_free_pages (gfp_t gfp_mask, unsigned int order );

Extern unsigned long get_zeroed_page (gfp_t gfp_mask );

Release:

Extern void _ free_pages (struct page * Page, unsigned int order );

Extern void free_pages (unsigned long ADDR, unsigned int order );

 

Memory Allocation may fail, and the memory release must be accurate!

 

1 kmalloc

The kmalloc () function is similar to a group of functions in the user space malloc to obtain a kernel memory in bytes.

Void * kmalloc (size_t size, gfp_t flags)

Void kfree (const void * objp)

 

The allocated memory is physically consecutive.

Gfp_t indicates the memory allocation method. For example:

Gfp_atomic: The allocation memory has a high priority and does not sleep.

Gfp_kernel: a common method that may be blocked.

 

2 vmalloc

 

Void * vmalloc (unsigned Long SIZE)

Void vfree (const void * ADDR)

Vmalloc () is similar to kmalloc. The virtual memory address allocated by vmalloc is continuous, and the physical address does not need to be consecutive. It is consistent with the user space allocation function.

Vmalloc allocates non-contiguous physical memory blocks. In the modified page table, the memory is mapped to the continuous area of the logical address space. The virtual address is continuous.

Whether consecutive physical addresses are required depends on the specific use case. In hardware devices that do not understand virtual addresses, the memory zone must be continuous.

By creating a page table and converting it to a virtual address space consecutively, there must be some consumption and performance impact.

Therefore, the kernel usually uses kmalloc to apply for memory, and vmalloc is used to allocate large memory.

 

Slab layer

Memory is often allocated and released in the kernel. To facilitate frequent data distribution and recovery, an empty

 

Idle linked list-memory pool. When you do not use the allocated memory, put it into the memory pool instead of releasing it directly.

The Linux Kernel provides the slab layer to manage memory allocation and release.

Frequent allocation and recovery will inevitably lead to memory fragmentation and cache them.

Design and Implementation of slab layer

The slab layer divides different objects into so-called high-speed cache groups. Each cache Group stores different types of objects. High-speed cache is divided into slab,

Slab consists of one or more consecutive physical pages. Each slab is in one of three states: full, partially full, and empty.

Relationship between high-speed cache, slab, and objects:

 

 

Compared with the traditional memory management mode, the slab cache distributor provides many advantages. First, the kernel usually depends on the allocation of small objects,

They are allocated countless times in the system lifecycle. The slab cache distributor provides this function by caching objects of similar sizes,

This avoids common fragmentation issues. The slab splitter also supports initialization of common objects, thus avoiding repeated objects for the same object.

. Finally, the slab splitter supports hardware cache alignment and coloring, which allows objects in different caches to occupy the same cache row,

This improves the cache utilization and improves performance.

 

Slab data structures and interfaces:

Each high-speed cache is represented by the kmem_cache structure:

Struct kmem_cache {

Struct kmem_list3 ** nodelists;

......

}

The cache contains three slab types: full, full, and idle.

Struct kmem_list3 {

Struct list_head slabs_partial;/* partial list first, better ASM Code */

Struct list_head slabs_full;

Struct list_head slabs_free;

......

};

Each slab contains multiple objects:

Struct slab {

Struct list_head list;

Unsigned long colouroff;

Void * s_mem;/* Including color offset */

Unsigned int inuse;/* num of objs active in slab */

Kmem_bufctl_t free;

Unsigned short nodeid;

};

 

Related interfaces: mm/slab. c

The kernel function kmem_cache_create is used to create a new cache. This is usually executed during kernel initialization or when the kernel module is loaded for the first time.

Struct kmem_cache *Kmem_cache_create(

Const char * Name,

Size_t size,

Size_t align,

Unsigned long flags,

Void (* ctor) (void *))

NameThe parameter defines the cache name, which is used by the proc file system (in/proc/slabinfo) to identify the cache.

SizeThe parameter specifies the size of the object created for this cache,

AlignThe parameter defines the alignment required for each object.

FlagsThe parameter specifies the option to enable the cache:

Some options of kmem_cache_create (specified in the flags parameter)

The slab_red_zone inserts a flag in the object header and tail to support checks on Buffer Overflow.

Slab_poison fills slab with a known mode, allowing monitoring of cached objects (the objects belong to all objects, but can be modified externally ).

Slab_hwcache_align specifies that the cache object must be aligned with the hardware cache line.

Ctor and dtorThe parameter defines an optional object constructor and destructor. Constructor and destructor are user-provided callback functions. When a new object is allocated from the cache, it can be initialized through the constructor.

To allocate an object from a named cache, you can use the kmem_cache_alloc function.

 

VoidKmem_cache_alloc(Struct kmem_cache * cache, gfp_t flags );

This function returns an object from the cache. Note that if the cache is currently empty, this function will call cache_alloc_refill to add memory to the cache.

The flags option of kmem_cache_alloc and

Cache: The created cache Zone

FlagsParameters:

Gfp_user allocates memory for the user (This call may cause sleep ).

Gfp_kernel allocates memory from the kernel RAM (This call may cause sleep ).

Gfp_atomic forces the call to be in a non-sleep state (for interrupt handlingProgramVery useful ).

Gfp_highuser allocates memory from high-end memory.

 

5. High-end memory ing

Permanent ing: may block

Map a given page structure to the kernel address space:

Void * kmap (struct page * Page)

Un ing:

Void kunmap (struct page * Page)

 

Temporary ing: no blocking

Void * kmap_atomic (struct page * Page)

 

Six assignment function Selection

L consecutive physical pages: kmalloc or low-level page distributor

L high-end memory allocation: alloc_pages points to the page structure pointer instead of the Logical Address pointer. Then, the high-end address memory is mapped to the logic address space of the kernel through kmap.

L no need for consecutive physical addresses: vmalloc virtual addresses may be discontinuous physical addresses, resulting in performance loss.

L frequent creation and destruction of many large data structures: the establishment of an slab cache area improves object allocation and recovery performance.

 

Linux high-end memory:
Http://ilinuxkernel.com /? P = 1013
Linux slab distributor analysis:
Https://www.ibm.com/developerworks/cn/linux/l-linux-slab-allocator/