Linux kernel Learning notes-kernel memory management method

One page

The kernel takes the physical page as the basic unit of memory management; the Memory Management Unit (MMU) transforms the virtual address into a physical

Addresses, which are typically processed in pages. The MMU manages the tables in the system in the page size unit.

32-bit system: Page size 4KB

64-bit system: Page size 8KB

The kernel represents each physical page in the system with the corresponding data structure:

<linux/mm_types.h>

struct Page {}

The kernel manages all the pages in the system through such a data structure, so the kernel determines whether a page is free and who owns the page

, the owner may be: User space process, dynamically allocated kernel data, static kernel code, page cache ...

Each physical page in the system is allocated such a structure for memory management.

Second District

There is a problem with Linux memory addressing:

Some hardware can only perform DMA (direct memory access) with some specific memory

Some architectures have a much larger range of physical addressing for their memory than you can address. This results in some memory not being permanently mapped to the kernel space.

Typically, the 32-bit Linux kernel address space is partitioned 0~3g to user space and 3~4g to kernel space. When the kernel module code or thread accesses memory,

The memory addresses in the code are logical addresses, which correspond to the actual physical memory addresses and require a one-to-one mapping of the addresses. So the kernel space address is 3~4g,

You can only map up to 1G of memory, and memory beyond 1G will ask!

Due to the limitations of the above conditions. Linux divides the kernel space address into three zones:

ZONE_DMA, Zone_normal and Zone_highmem.

Zone_highmem is the high-end memory, which is the origin of memory high-end memory concept.

In the x86 structure, three types of zones are as follows:

ZONE_DMA memory starts at 16MB

Zone_normal 16MB~896MB

Zone_highmem 896MB ~ End

Also, each zone contains many pages, forming different memory pools and allocating memory according to their purpose.

Use the corresponding data structure to represent the zone:

<linux/mmzone.h>

struct Zone {}

Three fetch pages/memory

Static inline struct page *alloc_pages(gfp_t gfp_mask, unsigned int order)

The function allocates 2 of the order's contiguous physical pages, and returns a pointer to the page struct body of the first one.

void *page_address(const struct page *page)

Returns the logical address pointing to the current location of the given physical page

extern unsigned long __get_free_pages (gfp_t gfp_mask, unsigned int order);

extern unsigned long get_zeroed_page (gfp_t gfp_mask);

Release:

extern void __free_pages (struct page *page, unsigned int order);

extern void Free_pages (unsigned long addr, unsigned int order);

Memory allocation may fail, memory release to be accurate!

1 Kmalloc

The Kmalloc () function is similar to the user space malloc set of functions, obtaining a chunk of kernel memory in bytes.

void *kmalloc (size_t size, gfp_t flags)

void Kfree (const void *OBJP)

The allocated memory is physically contiguous.

Gfp_t flag: Indicates how memory is allocated. Such as:

Gfp_atomic: High allocation memory priority, no sleep

Gfp_kernel: Common way, may be blocked.

2 Vmalloc

void *vmalloc (unsigned long size)

void Vfree (const void *ADDR)

Vmalloc () is similar to Kmalloc, where the memory virtual address assigned by Vmalloc is contiguous and the physical address does not need to be contiguous, consistent with the user space allocation function.

Vmalloc the virtual address is contiguous by allocating non-contiguous blocks of physical memory, correcting the page table, and mapping the memory to contiguous regions of the logical address space.

Whether there must be a continuous physical address and a specific usage scenario. In hardware devices that do not understand virtual addresses, the memory area must be contiguous.

By creating a page table to convert to virtual address space on the continuous, there must be some consumption, resulting in a performance impact.

So normally the kernel uses Kmalloc to apply for memory and allocates it using vmalloc when large chunks of memory are needed.

Four slab layers

Memory allocations and releases are often performed in the kernel. In order to facilitate the frequent allocation and recycling of data, it is common to create an empty

Busy list--memory pool. When the allocated memory is not used, it is placed in the memory pool instead of being released directly.

The Linux kernel provides a slab layer to manage the allocation and release of memory.

Frequent allocations and recoveries inevitably result in memory fragmentation, which caches them.

Design and realization of slab layer

The slab layer divides different objects into so-called cache groups. Each cache group holds different types of objects. The cache is divided into slab,

Slab consists of one or more physically contiguous pages. Each slab is in one of three states: full, partially full, empty.

Cache, slab, relationships between objects:

The slab cache allocator offers many advantages over the traditional memory management model. First, the kernel usually relies on the allocation of small objects,

They are assigned countless times during the system life cycle. The slab cache allocator provides this functionality by caching objects of similar size,

This avoids common fragmentation problems. The slab allocator also supports initialization of common objects, thus avoiding the duplication of one object for the same purpose

To initialize. Finally, the slab allocator can also support hardware cache alignment and shading, which allows objects in different caches to occupy the same cache line.

This increases the utilization of the cache and allows for better performance.

Slab data Structures and interfaces:

Each cache is represented by a Kmem_cache structure:

struct Kmem_cache {

struct KMEM_LIST3 **nodelists;

......

}

Buffer contains three kinds of slab: full, not full, idle

struct KMEM_LIST3 {

struct List_head slabs_partial; /* Partial list First, better ASM code */

struct List_head slabs_full;

struct List_head slabs_free;

......

};

Each of the slab contains multiple objects:

struct Slab {

struct List_head list;

unsigned long colouroff;

void *s_mem; /* including colour offset */

unsigned int inuse; /* num of OBJS active in slab */

kmem_bufctl_t free;

unsigned short Nodeid;

};

Related interfaces: MM/SLAB.C

The kernel function kmem_cache_create is used to create a new cache. This is usually done when the kernel is initialized, or when the kernel module is first loaded.

struct Kmem_cache *kmem_cache_create (

const Char *name,

size_t size,

size_t Align,

unsigned long flags,

void (*ctor) (void *))

The name parameter defines the cache names, which are used by the proc file system (in/proc/slabinfo) to identify the cache.

The size parameter specifies the sizes of the objects created for this cache.

The align parameter defines the required alignment for each object.

The flags parameter specifies the options that are enabled for the cache:

Partial options for Kmem_cache_create (specified in the flags parameter)

Slab_red_zone is used to support the check of buffer overflow in the object header and tail insertion flags.

Slab_poison fills the SLAB with a self-aware pattern that allows for monitoring of objects in the cache (objects are owned by the object, but can be modified externally).

SLAB_HWCACHE_ALIGN specifies that the cache object must be aligned with the hardware cache line.

The ctor and Dtor parameters define an optional object builder and destructor. Constructors and destructors are user-supplied callback functions. When a new object is allocated from the cache, it can be initialized through the constructor.

To assign an object from a named cache, you can use the Kmem_cache_alloc function.

void Kmem_cache_alloc (struct kmem_cache *cachep, gfp_t flags);

This function returns an object from the cache. Note If the cache is currently empty, then this function calls Cache_alloc_refill to add memory to the cache.

The flags option of Kmem_cache_alloc and the Kmalloc

Cachep: The built-in buffers

Flags parameter:

Gfp_user allocates memory to the user (this call may sleep).

Gfp_kernel allocates memory from kernel RAM (this call may sleep).

Gfp_atomic makes the call mandatory in a non-sleep state (useful for interrupt handlers).

Gfp_highuser allocates memory from high-end memory.

Five high-end memory mappings

Persistent mappings: May block

Maps a given page structure to the kernel address space:

void *kmap (struct page *page)

To de-map:

void Kunmap (struct page *page)

Temporary mapping: does not block

void *kmap_atomic (struct page *page)

Selection of six allocation functions

L Continuous Physical page: Kmalloc or low-level page splitter

L High-end memory allocation: alloc_pages points to the page structure pointer, not a logical address pointer. The high-end address memory is then mapped to the logical address space of the kernel through Kmap ().

L No continuous Physical address required: vmalloc virtual address continuous Physical address may be discontinuous, relative performance loss

L frequently create and destroy many large data structures: establish slab buffers to improve object allocation and recovery performance.

Linux high-end memory:
http://ilinuxkernel.com/?p=1013
Analysis of Linux Slab allocator:
https://www.ibm.com/developerworks/cn/linux/l-linux-slab-allocator/

Linux kernel Learning notes-kernel memory management method