System Analysis of Linux memory management partners-Xudong's blog-System Analysis of Linux system memory management partners

System Analysis of Linux memory management partners-Xudong's blog-blog Park

System Analysis of Linux memory management partners

I went to the interview today and an interviewer mentionedPartner System for Memory ManagementAt that time, I was disconnected, because I have never heard of it. Back in the evening, I checked some materials in the lab and summarized them as follows:

1. Partner System Concept

The partner system is a classic memory management method. The introduction of the Linux partner system provides a kernelAssign a group of consecutive pagesAnd an efficient allocation policy, which effectively solves the problem of external fragmentation.

2. Organization Structure of the partner System

Linux The memory management page size in is 4 kb . Group all idle pages 1 One chain table. Each chain table contains 1 , 2 , 4 , 8 , 16 , 32 , 64 , 128 , 256 , 512 And 1024 consecutive page boxes. Up 1024 Consecutive Page, corresponding 4 MB Memory size. Each The physical address of the first page of a page is an integer multiple of the size of the page.

Structure: In the I block linked list, num indicates the number of page blocks whose size is (2 ^ I), and address indicates the first address of the page block whose size is (2 ^ I.

 

3. Memory Allocation and release of partner systems

When directed to the kernelRequest allocation(2 ^ i-1),2 ^ I]When the number of page blocks is 2 ^ IPage block request processing.If no idle page blocks exist in the corresponding block linked list, find them in the larger page block linked list. When there are redundant pages in the allocated page blocks, the partner system inserts them into the corresponding idle page block linked list based on the size of the redundant page boxes.

when releases the memory of a single page , the kernel places it in CPU in high-speed cache, a pair may appear in cache page, is placed in the" quick table "list. In this process, the kernel first determines CPU whether the number of pages in the cache exceeds a certain" threshold ". If yes, return a batch of memory pages to the partner system, then add the page to CPU high-speed cache.

WhenRelease multiple pagesThe kernel first calculates the address of the partner of the memory block.The kernel calls the three parts that meet the following conditions as partners.:(1)The two blocks have the same size.B.(2)Their physical addresses areContinuous.(3)The first page of the First PartPhysical addressYes2* (2 ^ B. If the partner of the memory block is found, make sure that all pages of the partner are idle for merging. The memory continues to check the "partners" of the merged page blocks and checks whether they can be merged, and so on.

4. Anti-fragmentation mechanism of partner systems

The kernel divides the allocated page into three types:

(1)Unmovable page: These pages are fixed in memory and cannot be moved.

(2)Recycling page: These pages cannot be moved, but can be deleted. The kernel recycles pages when the recycle page occupies too much memory or when there is a shortage of memory.

(3)Movable page: These pages can be moved as needed.ProgramAll pages used belong to this category. They are mapped by page tables. When they are moved to a new location, the page table items are also updated accordingly.

During memory subsystem initialization, all pages are marked as movable. During startup, the memory allocated by the core kernel cannot be moved. In this case, the kernel policy is to assignContinuous memory blocks as large as possible, convert them from removable list to non-removable list. Instead of selecting smaller memory blocks that meet the requirements, the reason is as follows:

for example, the existing one can be 16 consecutive idle memory blocks on the page. When the kernel needs to be allocated 1 when the page cannot move the Memory Page. 8 page ( not one page ) move to the unmovable list, and then select 1 Page used for allocation . If the kernel needs to be allocated 1 If the page cannot be moved, select a page from the list that is not moved for allocation. 4 memory distribution after secondary allocation.

However, if you only select1Page is used to add a page to the unmovable list. When you need to allocate a page next time, select a page to add it to the unmovable list,Shown in this way4The memory distribution after the next allocation.

 

Therefore, when the unmovable idle block that can be used for allocation is not met, the distributor will migrate a contiguous memory block as large as possible in the movable list to the unmovable list. This wayAvoidDuring startup, the memory allocated by the kernel is hashed to all parts of the physical memory, so that other types of memory are allocated from fragments.

 

I went to the interview today and an interviewer mentionedPartner System for Memory ManagementAt that time, I was disconnected, because I have never heard of it. Back in the evening, I checked some materials in the lab and summarized them as follows:

1. Partner System Concept

The partner system is a classic memory management method. The introduction of the Linux partner system provides a kernelAssign a group of consecutive pagesAnd an efficient allocation policy, which effectively solves the problem of external fragmentation.

2. Organization Structure of the partner System

Linux The memory management page size in is 4 kb . Group all idle pages 1 One chain table. Each chain table contains 1 , 2 , 4 , 8 , 16 , 32 , 64 , 128 , 256 , 512 And 1024 consecutive page boxes. Up 1024 Consecutive Page, corresponding 4 MB Memory size. Each The physical address of the first page of a page is an integer multiple of the size of the page.

Structure: In the I block linked list, num indicates the number of page blocks whose size is (2 ^ I), and address indicates the first address of the page block whose size is (2 ^ I.

 

3. Memory Allocation and release of partner systems

When directed to the kernelRequest allocation(2 ^ i-1),2 ^ I]When the number of page blocks is 2 ^ IPage block request processing.If no idle page blocks exist in the corresponding block linked list, find them in the larger page block linked list. When there are redundant pages in the allocated page blocks, the partner system inserts them into the corresponding idle page block linked list based on the size of the redundant page boxes.

when releases the memory of a single page , the kernel places it in CPU in high-speed cache, a pair may appear in cache page, is placed in the" quick table "list. In this process, the kernel first determines CPU whether the number of pages in the cache exceeds a certain" threshold ". If yes, return a batch of memory pages to the partner system, then add the page to CPU high-speed cache.

WhenRelease multiple pagesThe kernel first calculates the address of the partner of the memory block.The kernel calls the three parts that meet the following conditions as partners.:(1)The two blocks have the same size.B.(2)Their physical addresses areContinuous.(3)The first page of the First PartPhysical addressYes2* (2 ^ B. If the partner of the memory block is found, make sure that all pages of the partner are idle for merging. The memory continues to check the "partners" of the merged page blocks and checks whether they can be merged, and so on.

4. Anti-fragmentation mechanism of partner systems

The kernel divides the allocated page into three types:

(1)Unmovable page: These pages are fixed in memory and cannot be moved.

(2)Recycling page: These pages cannot be moved, but can be deleted. The kernel recycles pages when the recycle page occupies too much memory or when there is a shortage of memory.

(3)Movable page: These pages can be moved at will. All pages used by user space applications belong to this category. They are mapped by page tables. When they are moved to a new location, the page table items are also updated accordingly.

During memory subsystem initialization, all pages are marked as movable. During startup, the memory allocated by the core kernel cannot be moved. In this case, the kernel policy is to assignContinuous memory blocks as large as possible, convert them from removable list to non-removable list. Instead of selecting smaller memory blocks that meet the requirements, the reason is as follows:

for example, the existing one can be 16 consecutive idle memory blocks on the page. When the kernel needs to be allocated 1 when the page cannot move the Memory Page. 8 page ( not one page ) move to the unmovable list, and then select 1 Page used for allocation . If the kernel needs to be allocated 1 If the page cannot be moved, select a page from the list that is not moved for allocation. 4 memory distribution after secondary allocation.

However, if you only select1Page is used to add a page to the unmovable list. When you need to allocate a page next time, select a page to add it to the unmovable list,Shown in this way4The memory distribution after the next allocation.

 

Therefore, when the unmovable idle block that can be used for allocation is not met, the distributor will migrate a contiguous memory block as large as possible in the movable list to the unmovable list. This wayAvoidDuring startup, the memory allocated by the kernel is hashed to all parts of the physical memory, so that other types of memory are allocated from fragments.