Linux partner system (I)-Overview of partner system

An important task of Linux kernel memory management is how to avoid fragments when memory is frequently applied for release. Linux uses the partner system to solve external fragments and slab to solve internal fragments. here we will first discuss the problem of external fragments... an important task of Linux kernel memory management is how to avoid fragments when memory is frequently applied for and released. Linux uses the partner system to solve external fragments and slab to solve internal fragments. here we will first discuss the problem of external fragments. There are two ways to avoid external fragments: one is to use the allocation of non-continuous memory as described earlier; the other is to use an effective method to monitor the memory, ensure that the kernel does not intercept a segment from the continuous idle memory when it applies for a small block of memory, thus ensuring the continuity and integrity of the large block of memory. Obviously, the former cannot be a common solution to the problem. First, the linear address space for ing non-contiguous memory is limited. second, each ING must rewrite the page table of the kernel and then refresh the TLB, this greatly reduces the allocation speed, which is obviously intolerable for the kernel that requires frequent memory application. Therefore, Linux uses the latter to solve the problem of external fragmentation, that is, the famous partner system. Www.2cto.com what is a partner system? The goal of the partner system is to use the smallest memory block to satisfy the kernel's memory requests. At first, there was only one block, that is, the entire memory. if it is 1 MB and the minimum allowed block is 64 KB, when we applied for a memory of KB, first, we need to split the 1 m block into two equal points, each of which is 512 K. The relationship between the two points is called a partner, and then split the first K memory block into two equal points, you can allocate the first 256 K memory block to the memory, which is a process of allocation. The following describes how the partner system allocates and recycles memory blocks. 1. during initialization, the system has 1 MB of continuous memory, and the minimum memory block allowed is 64 kB. the white part in the figure is the idle memory block. the colored part indicates that the memory block is allocated. 2. program A applies for A memory with A size of 34 KB, and the corresponding order is 0, that is, 2 ^ 0 = 1 min memory block 2.1 the system does not have order 0 (64 K) therefore, the memory block of order 4 (1 M) is split into two memory blocks of order 3 (512 K) 2.2 and there is still no memory block of order 0, therefore, the memory block of order 3 is split into two memory blocks of order 2 (256 K) 2.3 and there is still no memory block of order 0, therefore, the memory block of order 2 is split into two memory blocks of order 1 (128 K) 2.4 and there is still no memory block of order 0, therefore, the memory block of order 1 is split into two memory blocks of order 0 (64 K) 2.5, and the memory block of order 0 is found, and one of them is allocated to program, the memory of the partner system is an order 0 memory block and an order 1 Memory block, one memory block of order 2 and one memory block of order 3 Program B applies for a memory block with a size of 66K and the corresponding order is 1, that is, 2 ^ 1 = 2 minimum memory blocks. because there is a memory block of order 1 in the system, it is directly used to allocate 4 Program C to apply for a memory block of 35 KB, the corresponding order is 0. because there is a memory block of order 0 in the system, it is directly used to allocate 5 program D to apply for a memory block with a size of 67K, the corresponding order is 1 5.1. in the system, there is no memory block of order 1. Therefore, the memory block of order 2 is split into two memory blocks of order 1 5.2 and the memory block of order 1 is found, assigned program B www.2cto.com 6 to release the memory it applied, that is, the memory block of an order 1 7 program D releases the memory it applied for 7.1 the memory block of an order 1 is recycled to the memory 7.2 because the partner of this memory block is also idle, therefore, two memory blocks of order 1 are merged into one memory block of order 2. Program 8 A releases the applied memory, that is, a memory block of order 0 9 Program C releases the memory it applied for 9.1 a memory block of order 0 is released 9.2 both order 0 partner blocks are idle and merged, generate an order 1 Memory Block m 9.3 both order 1 partner blocks are idle, merge, generate an order 2 memory block 9.4 both order 2 partner blocks are idle, merge to generate an order 3 memory block. 9.5 both order 3 partner blocks are idle and merged, generate a data structure related to the memory block of order 4. in the previous article, we have briefly introduced the structure of struct zone. each management zone has its own struct zone, the struct free_area in struct zone is used to describe the idle memory block of the partner system in the management zone. [Cpp] struct zone {... www.2cto.com... struct free_area [MAX_ORDER]; ......} [Cpp] struct free_area {struct list_head free_list [MIGRATE_TYPES]; unsigned long nr_free ;}; free_area contains MAX_ORDER elements, where the order element records the free block of the 2 ^ order, these idle blocks are organized in the form of a two-way linked list in free_list. for idle blocks of the same size, their types are different and organized in different free_list, nr_free records the total number of idle memory blocks in the free_area. The default value of MAX_ORDER is 11, which means the size of the maximum memory block is 2 ^ 10 = 1024 page boxes. For memory blocks of the same size, the start page of each memory block is used to connect nodes in the linked list, these nodes correspond to the lru domain [cpp] struct page {www.2cto.com ...... struct list_head lru;/* Pageout list, eg. active_list * protected by zone-> lru_lock! */...} The connection is as follows: in kernel versions earlier than 2.6.24, the free_area structure has only one free_list array. starting from 2.6.24, the free_area structure contains MIGRATE_TYPES free_list, these arrays are divided based on the mobility of page frames. why is this division necessary? In fact, it was also proposed to reduce fragments. we considered the following situation: the figure contains 32 pages and only four page boxes are allocated, however, the maximum continuous memory that can be allocated is only eight page boxes (because the memory allocated by the partner system must be an integer power-factor page box of 2 ), the kernel solves this problem by grouping different types of pages. Allocated pages can be divided into three types: Non-movable pages, which have fixed locations in the memory and cannot be moved. Most of the core allocated memory of the kernel belongs to this type of www.2cto.com Reclaimable pages: such pages cannot be moved directly, but can be deleted, and their content pages can be regenerated from other places, for example, the data mapped from a file belongs to this type. for such pages, the kernel has a dedicated page recycling processing movable page: such pages can be moved at will, the page used by the user space application belongs to this category. They are mapped through the page table. if they are copied to a new location, the page table items are updated accordingly, and the application will not notice any changes. If the majority of pages are mobile pages, and the four pages assigned out are non-mobile pages, because the non-mobile pages are inserted in the middle of other types of pages, as a result, a large memory block cannot be allocated from the idle continuous memory area. Consider: separate the recyclable page from the unmovable page, so that although the block of continuous memory cannot be allocated in the area of the unmovable page, however, the area of the recycle page is not affected, and large blocks of continuous memory can be allocated. The kernel defines the migration type as follows: [Cpp] # define MIGRATE_UNMOVABLE 0 # define limit 1 # define MIGRATE_MOVABLE 2 # define limit 3/* the number of types on the pcp lists */# define MIGRATE_RESERVE 3 # define MIGRATE_ISOLATE 4/ * can't allocate from here */# define MIGRATE_TYPES 5 The first three types of MIGRATE_PCPTYPES are per_cpu_pageset, this indicates the number of migration types in the linked list in the cache data structure on each CPU page. when MIGRATE_RESERVE is useless in the first three lists, you can allocate MIGRATE_ISOLATE from MIGRATE_RESERVE to move the physical memory page across NUMA nodes on large systems, it is beneficial to move the physical memory page to the nearest point, so the most frequent CPU www.2cto.com MIGRATE_TYPES on this page indicates the number of migration types. when there is no idle block in the linked list corresponding to a specified migration type, [cpp] static int fallbacks [MIGRATE_TYPES] [MIGRATE_TYPES-1] = {[MIGRATE_UNMOVABLE] = {MIGRATE_RECLAIMABLE, MIGRATE_MOVABLE, region}, [region] = {region, MIGRATE_MOVABLE, region}, [MIGRATE_MOVABLE] = {region, region, region}, [MIGRATE_RESERVE] = {MIGRATE_RESERVE, MIGRATE_RESERVE, MIGRATE_RESERVE }, /* Never used */}; by vanbreaker