How to implement memory retention in the Linux kernel

The reserved memory (reservedmemory) in Linux refers to keeping a portion of the memory in the system, and the kernel does not set up a page table for it, and the general application cannot access the memory. In the process of board debugging, memory testing and equipment dam debugging, this method can be used to verify that the system can start successfully in the case of only low-end memory, in addition, the server and storage system environment, can also be used in this way from a large number of system memory to retain a part, For special purposes use or simulate devices such as NVDIMM. Therefore, it is necessary to have a certain understanding of the use of reservedmemory.

1. Implement reserved memory

The Linux kernel boot parameter cmdline provides a number of options for setting kernel-initiated parameters and configurations. The "mem" option is used to limit the size of the system memory that the kernel can see, so by setting

The MEM parameter allows the memory to be preserved. When the size specified by the mem parameter is less than the actual physical memory available to the system, the rest of the mem specified in the actual system memory is reservedmemory. For example, if a system has 64G of physical memory, by adding mem=48g to Kernelcommand line, the last (64-48) =16g high-end memory can be preserved.

Observing and comparing the output of/proc/iomem and dmesg with or without mem parameter kernel, you can see that the 64G physical memory mapped to the CPU address is 0X100000000~0XFFFFFFFFF, The CPU address space address of the 48G high-end physical memory map is: 0X100000000~0XBFFFFFFFF, so after adding mem=48g, the CPU physical address 0xc00000000~ The 0xFFFFFFFFF corresponding memory from 48G to 64G is reserved.

2. Using reserved memory

Because the kernel does not start with a page table for reserved memory, neither the kernel-supplied kmalloc () or the dma_malloc_coherence () function, or the application's malloc (), realloc () function can request space from the reserved memory. So in order to access reserved memory, you need to use some method to establish a mapping of the userspace address to the reserved memory physical addresses. There are two ways to implement this mapping:

One, based on the implementation of the interface provided by some drivers

Many device drivers can map their own physical address space to the kernel virtual address space, and some SDK packages provide a device-driven API interface for mapping user-state addresses and device-specific physical address spaces. For this reason, you can refer to this part of the code to implement a mapping of reserved memory.

The advantage of this approach is that the mapping method itself enables different levels of cache consistency, such as Cache/uncache/writecombing/writthourgh, which are mapped in the same way as the properties specified in the referenced device driver. Of course, the disadvantage is obvious, it makes the original and other devices do not have any relationship and reservememory, when used to become dependent on a device driver and the corresponding SDK library files.

Second, based on the implementation of/DEV/MEM

It is well known that the implementation of Linux kernel and driver takes into account the principle of separation of mechanism and rule. /dev/mem is the mechanism that the kernel provides to the user to map a physical memory address to the user state space, which relies primarily on the mmap () system invocation. The system call establishes a page table for a specified length of memory starting from the specified start address, and maps to the idle user-State address space of the current program.

Specific implementations can refer to the following sample code:

char *vmem = NULL;

FILE * fp= fopen ("/dev/mem", "w+");

int fp = open ("/dev/mem", O_RDWR | O_sync);

if (FP < 0) {

printf ("Open/dev/mem error!\n");

return-1;

}

Vmem = Mmap (NULL, LEN, Prot_read | Prot_write, map_shared, FP,

Reserverd_mem_start);

if (Vmem = = NULL) {

printf ("Mmap reserver mem on/dev/mem error!\n");

return-1;

}

The advantage of this method is less reliance, only in the userspace development, debugging convenience, insufficient points out that the above method relies on the implementation of the/dev/mem driver, can not guarantee that the mapped memory is uncached, and the call level is deep.

Through the above analysis and comparison is not difficult to see, the actual development of the final use of which needs to be based on the actual project requirements and resources to choose.

How to implement memory retention in the Linux kernel