Linux memory Management "Go"

transferred from: http://blog.chinaunix.net/uid-25909619-id-4491368.htmlLinux Memory Management

absrtact : This chapter first examines the Linux process memory management from the application developer's point of view, and on this basis, discusses the system physical memory management and the use of kernel memory in the kernel. Strive to from outside to inside, naturally guide the user analysis of Linux memory management and use. At the end of this chapter, we give an example of memory mapping to help users understand the relationship between kernel memory management and user memory management, hoping that we can finally harness Linux memory management.

Objective

Memory management has always been the focus of all operating system books focused on the content, whether on the market or on the internet is filled with a large number of memory management materials and materials. Therefore, we have to write the Linux memory management to take COP strategy, from the theoretical level will not go to swim, laughable. What we want to do most and possibly do is to talk about the understanding of memory management from the developer's point of view, and the ultimate goal is to share with you our experience of using memory in kernel development and the knowledge of Linux memory management.

Of course, this one also involves some basic theories of memory management such as section pages, but our aim is not to emphasize the theory, but to guide the understanding of the practice in development, so just donuts, do not delve into it.

Following the doctrine of "theory comes from practice", we do not have to drill into the kernel to see how the system memory is managed, which often makes you into a indefinitely dilemma (I made this mistake!). ）。 So the best way is to first look at how the process uses memory from the outside (User programming category), and then go deep into the kernel to learn more about how memory is managed, when you have a more intuitive understanding of how memory is used. Finally, through an instance programming, the content of the speech is assimilated.

How does the process and memory process use memory?

There is no doubt that all processes (executing programs) must occupy a certain amount of memory, either to store program code loaded from disk, or to store data from user input, and so on. However, the way the process manages these memory is different from memory usage, some of which are statically allocated and collected uniformly, while others are dynamically allocated and reclaimed as needed.

For any ordinary process, it involves 5 different pieces of data. A friend with a little programming knowledge can imagine that these data segments contain "program code Snippets," "Program Data Segments," "Program stack segments," and so on. Yes, these data segments are in it, but in addition to the above data segments, the process also contains two different data segments. Let's briefly summarize the 5 different data areas contained in the memory space of the process.

Code Snippet: A code snippet is an action instruction used to hold an executable file, that is, it is a mirror in which the executables are stored. Code snippets need to be prevented from being illegally modified at run time, so only read operations are allowed, not write (modify) operations--it is not writable.

Data Segment : A data segment is used to hold a global variable initialized in an executable file, in other words, a variable and a global variable that holds the program statically assigned [1].

BSS segment [2]: The BSS segment contains uninitialized global variables in the program, all zeros in the in-memory BSS segment.

Heap (Heap ): The heap is used to store dynamically allocated memory segments in the process running, which are not fixed and can be dynamically expanded or scaled down. When a process calls a function such as malloc to allocate memory, the newly allocated memory is dynamically added to the heap (heap is expanded), and freed memory is removed from the heap when memory is freed (heap is scaled down).

stack : The stack is a local variable that is temporarily created by the user's program, that is, the variables defined in the parentheses "{}" (but not the static declared variables, static means that the variables are stored in the data segment). In addition, when a function is called, its arguments are also pressed into the process stack that initiates the call, and the return value of the function is stored back to the stack when the call ends. Due to the advanced first-out features of the stack, the stack is particularly handy for saving/recovering call sites. In this sense, we can think of the stack as a memory area where temporary data is stored and exchanged.

How does the process organize these areas?

The data segments, BSS, and heap in the above-mentioned memory areas are often stored continuously-memory locations are contiguous, and code snippets and stacks are often stored independently. Interestingly, the heap and stack of two regional relationships are "ambiguous", they are one down "long" (I386 architecture stack down, heap up), one upward "long", relative. But you don't have to worry that they will meet, because they are spaced very large (how much, you can calculate from the following example program), there are few opportunities to meet together.

A brief description of the distribution of process memory areas:

"The facts speak louder than words," and we use a small example (from "User-level Memory Management") to show the differences and locations of the various memory areas described above.

#include <stdio.h>

#include <malloc.h>

#include <unistd.h>

int Bss_var;

int data_var0=1;

int Main (int argc,char **argv)

{

printf ("Below is addresses of types of process ' s mem\n");

printf ("Text location:\n");

printf ("\taddress of Main (Code Segment):%p\n", main);

printf ("____________________________\n");

int stack_var0=2;

printf ("Stack location:\n");

printf ("\tinitial End of Stack:%p\n", &stack_var0);

int stack_var1=3;

printf ("\tnew End of Stack:%p\n", &stack_var1);

printf ("____________________________\n");

printf ("Data location:\n");

printf ("\taddress of Data_var (data Segment):%p\n", &data_var0);

static int data_var1=4;

printf ("\tnew End of Data_var (data Segment):%p\n", &data_var1);

printf ("____________________________\n");

printf ("BSS location:\n");

printf ("\taddress of Bss_var:%p\n", &bss_var);

printf ("____________________________\n");

Char *b = SBRK ((ptrdiff_t) 0);

printf ("Heap location:\n");

printf ("\tinitial End of Heap:%p\n", b);

BRK (B+4);

B=SBRK ((ptrdiff_t) 0);

printf ("\tnew End of Heap:%p\n", b);

return 0;

}

It results in the following

Below is addresses of types of process ' s MEM

Text Location:

Address of Main (Code Segment): 0x8048388

____________________________

Stack Location:

Initial End of STACK:0XBFFFFAB4

New end of Stack:0xbffffab0

____________________________

Data Location:

Address of Data_var (data Segment): 0x8049758

New End of Data_var (data Segment): 0x804975c

____________________________

BSS Location:

Address of bss_var:0x8049864

____________________________

Heap Location:

Initial End of heap:0x8049868

New End of heap:0x804986c

You can also see the size of the program by using the size command, such as executing size example will get

Text data BSS Dec hex filename

1654 280 8 1942 796 Example

But these data are static statistics compiled by the program, which shows the dynamic values of the process runtime, but the two correspond.

In the previous example, we have taken a peek at the logical memory distribution used by the process. This part we continue to enter the operating system kernel to see how the process of memory specific how to allocate and manage.

From the user to the kernel, the memory representation used in the form will go through the "logical address"-"linear address"-"Physical Address" several forms (about the interpretation of several addresses are described above). The logical address is transformed into a linear address by the segment mechanism, and the linear address is transformed into a physical address by the page mechanism. (but we want to know that although the Linux system retains the segment mechanism, but all the program's segment address is dead to 0-4g, so although the logical address and the linear address are two different address space, but in Linux the logical address is equal to the linear address, their values are the same). Along this trail, the main problems we are studying are focused on the next few questions.

1. How is the process space address managed?

2. How does the process address map to physical memory?

3. How is physical memory managed?

And some sub-problems caused by the above problems. such as system virtual address distribution, memory allocation interface, continuous memory allocation and non-contiguous memory allocation, etc.

Process Memory Space

The Linux operating system uses virtual memory management technology so that each process has its own process address space that is not interfering with each other. This space is a block size of 4G linear virtual space, the user sees and touches the virtual address, cannot see the actual physical memory address. Using this virtual address not only protects the operating system (the user does not have direct access to physical memory), but more importantly, the user program can use a larger address space than the actual physical memory (for specific reasons see the Hardware Base section).

Before discussing the details of the process space, here are some questions to clarify:

The first, 4G process address space is artificially divided into two parts-user space and kernel space. The user space is from 0 to 3G (0xC0000000) and the kernel space occupies 3G to 4G. Typically, a user process can access only the virtual address of the user space and cannot access the virtual address of the kernel space. The kernel space can be accessed only when the user process makes a system call (on behalf of the user process in kernel state execution).

Second, the user space corresponding process, so each time the process switch, the user space will change, and kernel space is the kernel responsible for mapping, it does not follow the process changes, is fixed. The kernel space address has its own page table (INIT_MM.PGD), and the user processes each have different page tables.

L Thirdly, the user space of each process is completely independent and irrelevant to each other. If you do not believe it, you can run the above program 10 times simultaneously (of course, in order to run at the same time, let them sleep together before returning 100 seconds), you will see 10 processes occupy the same linear address.

Process Memory Management

The object of process memory management is the memory image on the process linear address space, which is actually the virtual memory region used by the process. A process virtual space is a 32-or 64-bit "flat" (separate contiguous interval) address space (the exact size of the space depends on the architecture). It is not easy to manage such a large flat space uniformly, and for ease of management, virtual space is divided into memory areas that are variable in size (but must be multiples of 4096), which are arranged in a process-linear address in an orderly manner like a parking space. The partitioning principle for these zones is to "keep the address spaces consistent with the access attributes", where the so-called access attributes are nothing more than "readable, writable, executable, etc."

If you want to see the area of memory occupied by a process, you can get it using command Cat/proc/<pid>/maps (PID is the process number, you can run the example we gave above ——./example &;p ID will print to the screen), You can find a lot of digital information similar to the following.

Because the program example uses a dynamic library, the memory areas used by the dynamic libraries are included in addition to the memory area used by the example itself (area order: Code snippet, data segment, BSS segment).

We only extract information about example, except for the code snippet and data segment that the first two lines represent, and the last line is the stack space used by the process.

-------------------------------------------------------------------------------

08048000-08049000 R-xp 00000000 03:03 439029/home/mm/src/example

08049000-0804a000 Rw-p 00000000 03:03 439029/home/mm/src/example

...............

bfffe000-c0000000 Rwxp ffff000 00:00 0

--------------------------------------------------------------------------------------------------------------- -------

The data format for each row is as follows:

(memory area) Start-end access offset main device number: Secondary device number I node file.

Note that you will certainly find that the process space contains only three memory areas, does not seem to be mentioned above the heap, BSS, etc., in fact, the memory area of the program memory segment and the process address space is a fuzzy correspondence, that is, the heap, BSS, data segments (initialized) are represented in the process space by the data segment memory region.

The data structure of the corresponding process memory area in the Linux kernel is: vm_area_struct, the kernel manages each memory area as a separate memory object, and the corresponding operation is consistent. The object-oriented approach enables the VMA structure to represent many types of memory regions-such as memory-mapped files or process user-space stacks-and the operation of these areas is different.

The structure of the vm_area_strcut is more complex and the detailed structure of it is described in the relevant information. We'll just add a little more to its organizational approach here. Vm_area_struct is the basic management unit that describes the address space of a process, which often requires multiple memory areas to describe its virtual space, and how to correlate these different areas of memory. You might all think of using a linked list, and indeed the vm_area_struct structure is actually linked in a linked list, but to make it easier to find, the kernel organizes memory areas in the form of red-black trees (the previous kernel uses the balance tree) to reduce the time-to-search. The coexistence of two forms of organization is not redundant: The linked list is used when it is necessary to traverse all nodes, while the red-black tree is suitable for locating specific memory areas in the address space. The kernel has high performance for various operations on the memory area, so both of these data structures are used at the same time.

A management model that reflects the process address space:

The description structure for the address space of the process is the memory descriptor structure, which represents the full address space of the process, which contains all the information about the process address space, which of course contains the memory area of the process.

Allocation and recycling of process memory

Process-related operations such as creating a process fork (), program loading EXECVE (), Mapping file Mmap (), Dynamic memory allocation malloc ()/brk (), and so on, need to allocate memory to the process. However, the process requested and obtained is not the actual memory, but virtual memory, accurate is "memory area". The allocation of the memory area by the process eventually boils down to the Do_mmap () function (the BRK call is implemented separately as a system call, without Do_mmap ()),

The kernel uses the Do_mmap () function to create a new linear address range. However, it is not very accurate to say that the function creates a new VMA, because if the address interval created is adjacent to an existing address range and they have the same access rights, then the two intervals will be merged into one. If you can't merge, then you really need to create a new VMA. In either case, the Do_mmap () function adds an address range to the address space of the process-whether it is expanding an existing area of memory or creating a new zone.

Similarly, releasing a memory area should use the function Do_ummap (), which destroys the corresponding area of memory.

How to become real by virtual!

The address that the process can directly manipulate from above is the virtual address. When the process requires memory, the kernel obtains only the virtual memory area, not the actual physical address, the process does not get physical memory (physical page-page concept please refer to the Hardware Foundation chapter), only to obtain a new linear address range of the right to use. The actual physical memory will only be generated by the request page mechanism when the process actually accesses the newly acquired virtual address, thus entering a routine that allocates the actual page.

This exception is the basic guarantee of the virtual memory mechanism-it tells the kernel to actually allocate the physical page to the process and create the corresponding page table, after which the virtual address is actually mapped to the physical memory of the system. (Of course, if the page is swapped out to disk, there will also be a page fault exception, but then there is no need to create a table)

This request-page mechanism defers the allocation of pages until it can no longer be postponed, and is not in a hurry to get everything done at once (this idea is a bit like the proxy mode in design mode). The reason to do this is to take advantage of the "local principle" of memory access, the benefit of the request page is to save the free memory, improve the system throughput rate. To get a clearer picture of the request page mechanism, you can look at the deep understanding of the Linux kernel.

Here we need to explain the nopage operation on the memory area structure. When the accessed process virtual memory is not actually assigned to the page, the operation is called to allocate the actual physical page, and the page table entry is established for the page. In the last example we will show how to use this method.

System Physical Memory Management

Although an application operates on virtual memory that is mapped to physical memory, the processor directly operates on physical memory. So when an application accesses a virtual address, the virtual address must first be converted to a physical address before the processor can resolve the address access request. The conversion of the address needs to be done by querying the page table, which, in summary, requires that the address translation fragment the virtual address so that each virtual address points to the page table as an index, and the page table entry points to the next level of page table or to the final physical page.

Each process has its own page table. The PGD domain of the process descriptor points to the page global catalog of the process. Let's take a look at a picture from the Linux device driver to see the transition between the process address space and the physical page.

The above process is easy to say and difficult to do. Because a physical page must be allocated before the virtual address is mapped to the page-that is, you must first obtain the free page from the kernel and create a page table. Let's look at the kernel's mechanism for managing physical memory.

Physical Memory Management (page management)

The Linux kernel manages physical memory through a paging mechanism that divides the entire memory into countless 4k (in the i386 architecture)-sized pages, so that the basic unit of allocating and reclaiming memory is the memory page. Paging management helps to flexibly allocate memory addresses because allocations do not have to require large chunks of contiguous memory [3], and the system can make up the required memory for the process to be used by the east and West pages. Despite this, the fact that the system uses memory tends to allocate contiguous blocks of memory, because the page table does not need to be changed when allocating contiguous memory, thus reducing the flush rate of the TLB (frequent refreshes can greatly reduce access speed).

In view of the above requirements, the kernel allocates physical pages in order to minimize discontinuities and uses a "partner" relationship to manage the idle pages. Partnership allocation algorithm Everyone should be familiar-almost all the operating system aspects of the book will be mentioned, we do not go into detail about it, if not clear can see the relevant information. It is only necessary to understand that the organization and management of the free pages in Linux takes advantage of the partnership, so the free page allocation also needs to follow the partnership, the smallest unit can only be a power page size of 2. The basic function of allocating idle pages in the kernel is get_free_page/get_free_pages, which either assigns a single page or assigns a specified page (2, 4, 8 ...). 512 pages).

Note: Get_free_page is allocating memory in the kernel, unlike malloc's allocation in user space, malloc leverages heap dynamic allocation, which is actually called the BRK () system call, which is the role of enlarging or shrinking the process heap space (it modifies the process's BRK domain). If the existing memory area is not enough to hold the heap space, the corresponding memory area is expanded or shrunk in multiples of the page size, but the BRK value is not modified in multiples of the page size, but instead is modified as the actual request. As a result, malloc allocates memory in user space in bytes, but the kernel is still internally allocated on a page-by-unit.

In addition, it should be mentioned that the physical page is described in the system by the page structure struct-page, where all the pages in the system are stored in the array mem_map[], which can be used to find each page in the system (idle or non-idle). The free pages can be indexed by the Partnership's free Page list (Free_area[max_order]) mentioned above.

Kernel Memory usage

Slab

The so-called ruler has a long, inch is short. allocating memory in the smallest amount of pages is really convenient for physical memory in the kernel management system, but the kernel itself most often uses memory blocks that are small (much less than a page)--such as storing file descriptors, process descriptors, virtual memory area descriptors, and so on, with less than a page of memory. The memory used to store the descriptor is like bread crumbs and bread compared to the page. Multiple of these small chunks of memory can be clustered on a full page, and these chunks of memory are generated/destroyed as frequently as breadcrumbs.

To satisfy the need of the kernel for this small memory block, the Linux system employs a technique called the slab allocator. The implementation of the slab allocator is quite complex, but the principle is not difficult, the core idea is "storage pool [4]" application. Memory fragments (small chunks of memory) are considered objects, and when they are used, they are not released directly but are cached in the "Storage Pool" for the next use, which undoubtedly avoids the extra load of creating and destroying objects frequently.

Slab technology not only avoids the inconvenience of internal memory shards (explained below) (the main purpose of the introduction of the slab allocator is to reduce the number of calls to the partner system allocation algorithm-frequent allocations and recoveries will inevitably lead to memory fragmentation-difficult to find chunks of contiguous free memory), It also makes good use of the hardware cache to improve access speed.

Slab is not a form of memory allocation that is independent of the partnership, Slab is still based on the page, in other words, slab the page (the list of free pages from the partnership management) into many small chunks of memory for allocation, object allocation and destruction in slab kmem_ Cache_alloc and Kmem_cache_free.

Kmalloc

The slab allocator is not only used to store kernel-specific structures, it is also used to handle kernel requests for small chunks of memory. Of course, given the characteristics of the slab allocator, it is generally true that requests for small chunks of memory smaller than one page in the kernel program are done through the interface Kmalloc provided by the slab allocator (although it allocates 32 to 131072 bytes of memory). From the point of view of kernel memory allocation, Kmalloc can be seen as an effective complement to get_free_page (s), with more flexibility in the granularity of memory allocations.

If you are interested, you can find the various slab information that the kernel performs on-site using in/proc/slabinfo, where you will see all the slab usage information in the system. You can see from the information that in addition to the slab used in the dedicated structure, there are a large number of slab prepared for kmalloc (some of which are DMA-prepared).

kernel non-contiguous memory allocation ( Vmalloc )

Partnership or slab technology, from the memory management theory point of view is basically consistent, they are to prevent "shard", but fragmentation is divided into external shards and internal shards said, the so-called internal Shard is said that the system in order to meet the needs of a small memory area (continuous), Had to allocate a large area of contiguous memory to it, resulting in wasted space; external sharding refers to a system that has enough memory but is fragmented and unable to meet the need for chunks of "contiguous memory". Any shard is an obstacle to the efficient use of memory by the system. The slab allocator allows a large number of small chunks of memory contained within a page to be allocated independently, avoiding internal shards and saving free memory. The partnership, which manages the block of memory by size, reduces the damage to the external shards to some extent, because the page box assignment is not blind, but is ordered in order of size, but the partnership only mitigates the external shards, but it is not completely eliminated. You do the rest of the free memory after you've allocated the page multiple times.

So the final idea of avoiding an external shard is to make use of discontinuous blocks of memory to combine into "seemingly large chunks of memory"-a situation that is similar to a user-space allocation of virtual memory, in which memory is logically contiguous and is actually mapped to not necessarily contiguous physical memory. The Linux kernel borrows this technology, allowing kernel programs to allocate virtual addresses in the core address space, as well as mapping virtual addresses to distributed Memory pages using page tables (kernel page tables). This perfectly solves the problem of external shards in kernel memory usage. The kernel provides the VMALLOC function to allocate kernel virtual memory, which is different from Kmalloc, which can allocate much larger memory space than Kmalloc (which can be larger than 128K, but must be a multiple of the page size), but compared to Kmalloc, Vmalloc need to remap the kernel virtual address, must update the kernel page table, so the allocation efficiency is lower (with space to change time)

Like the user process, the kernel also has a mm_strcut structure called INIT_MM that describes the kernel address space, where the page table entry Pdg=swapper_pg_dir contains the mapping of the system kernel space (3G-4G). So Vmalloc allocates the kernel virtual address must update the kernel page table, and Kmalloc or get_free_page because of the allocated contiguous memory, so do not need to update the kernel page table.

The kernel virtual memory allocated by Vmalloc is in a different interval from the kernel virtual memory allocated by Kmalloc/get_free_page and does not overlap. Because the kernel virtual space is managed by the partition, it performs its duties. The process space address is distributed from 0 to 3G (actually to Page_offset, which is equal to 0xc0000000 in 0x86), from 3G to vmalloc_start this is the physical memory map area (which contains the kernel image, the physical page table Mem_ Map, etc.) for example, I use the system memory is 64M (can be seen with free), then (3g--3g+64m) This memory should be mapped to physical memory, and Vmalloc_start location should be near 3g+64m (say "near" Since there will also be a 8M gap between the physical memory map area and the Vmalloc_start to prevent the leap, the vmalloc_end position is close to 4G (said "Close" because the last position system retains a 128k-sized area for dedicated page mappings. There may also be high-end memory map areas, these are details, here we do not entanglement).

is a fuzzy outline of the memory distribution

Contiguous memory allocated by the Get_free_page or Kmalloc functions is trapped in the physical map area, so the kernel virtual address and the actual physical address that they return are just an offset (page_offset), and you can easily convert it to a physical memory address. The kernel also provides the Virt_to_phys () function to convert the physical map area addresses in the kernel virtual space into physical addresses. Be aware that the address in the physical memory map area is in order with the kernel page table, and each physical page in the system can find its corresponding kernel virtual address (in the physical memory map area).

The address assigned by Vmalloc is limited to Vmalloc_start and Vmalloc_end. Each piece of Vmalloc allocated kernel virtual memory corresponds to a vm_struct struct (do not mix with vm_area_struct, which is the structure of the process virtual memory area), the different kernel virtual address is 4k the size of the idle zone interval, to prevent cross-border-see). Like the properties of a process virtual address, these virtual addresses do not have a simple displacement relationship to physical memory and must be converted to physical addresses or physical pages through a kernel page table. It is possible that they have not yet been mapped to actually allocate physical pages when a page fault occurs.

Here is a small program to help you understand the above several allocation functions corresponding to the region.

#include <linux/module.h>

#include <linux/slab.h>

#include <linux/vmalloc.h>

unsigned char *pagemem;

unsigned char *kmallocmem;

unsigned char *vmallocmem;

int init_module (void)

{

Pagemem = get_free_page (0);

PRINTK ("<1>pagemem=%s", Pagemem);

Kmallocmem = Kmalloc (100,0);

PRINTK ("<1>kmallocmem=%s", Kmallocmem);

Vmallocmem = Vmalloc (1000000);

PRINTK ("<1>vmallocmem=%s", Vmallocmem);

}

void Cleanup_module (void)

{

Free_page (PAGEMEM);

Kfree (KMALLOCMEM);

Vfree (VMALLOCMEM);

}

Instance

Memory Mapping (MMAP) is a great feature of the Linux operating system, which maps system memory to a file (device) so that access to memory can be achieved by accessing the file contents. The greatest benefit of this is increased memory access speed and the ability to access memory using the FileSystem's interface programming (the device is handled as a special file in Linux), reducing the difficulty of development. Many device drivers use the memory-mapping feature to correlate a segment of a user's space to device memory, which is actually access to device memory whenever memory is read and written within the allocated address range. Access to the device files is also equivalent to accessing the memory area, which means that the memory can be accessed through the file manipulation interface. The x server in Linux is an example of using memory maps to achieve direct, high-speed access to video card memory.

A friend who is familiar with file operations will know that there is a mmap method in the file_operations structure, which is called when the user executes the MMAP system call--but before the file system Mmap method is called, the kernel also needs to process the allocated memory area (Vma_ struct), create a page table, and so on. For specific mapping details are not introduced, it should be emphasized that the establishment of the page table can be used Remap_page_range method to establish all the mapping area of the page table, or the use of Vma_struct Nopage method in the page when the page pages to create pages table. The first method is simple and convenient compared to the second one, but the flexibility is not high. All of the page tables are trained at once, not for those that need to create a page table on site-for example, the mapping area needs to be extended or the situation in our example below.

Our example here wants to use a memory map to map some of the virtual memory in the system kernel to the user space for the application to read-you can use it for large-scale information transfer from kernel space to user space. So we're going to try to write a virtual character device driver that maps the system kernel space to the user space --mapping the kernel virtual memory to the user virtual address. As seen from the previous section, there are two virtual addresses in the Linux kernel space: One is physical and logical contiguous physical memory mapped virtual addresses, and the other is logical contiguous but not physically contiguous Vmalloc allocated memory virtual addresses. Our example program will demonstrate the entire process of mapping the Vmalloc allocated kernel virtual address to the user address space.

There are two main issues to be addressed in the program:

The first is how to correctly convert the kernel virtual memory allocated by Vmalloc to a physical address?

Because the memory map first obtains the mapped physical address, it can be mapped to the required user virtual address. We have seen that the address in the memory map area of the internal nuclear physics can be converted to the actual physical memory address by the kernel function Virt_to_phys, but the kernel virtual address assigned by Vmalloc cannot be converted directly to physical address, so we must take care of this part of virtual memory especially It is converted into an address in the internal nuclear physics Memory map area and then changed to a physical address using Virt_to_phys.

The conversion effort requires the following steps:

A) Locate the page table that corresponds to the vmalloc virtual memory and find the corresponding page table entry.

b) Gets the page pointer corresponding to the page table item

c) The corresponding kernel physical memory mapped area address is obtained through the page .

As shown in the following:

Second, when accessing the Vmalloc allocation area, if you find that virtual memory has not yet been mapped to a physical page, you need to handle a "page fault". Therefore, we need to implement the Nopaga operation in the memory area in order to return the mapped physical page pointer, in our instance, return the address in the kernel physical memory map area of the above procedure . because the Vmalloc assigned virtual address and the physical address of the corresponding relationship is not assigned to be determined, the page must be created in the pages table, so here can not use the Remap_page_range method, can only be used VMA Nopage method page by page of the establishment.

Program composition

MAP_DRIVER.C, which is a virtual character driver that is loaded as a module. The driver is responsible for mapping a long kernel virtual address (Vmalloc assigned) to a device file. The main function has--vaddress_to_kaddress () is responsible for the Vmalloc assigned Address Page table parsing, to find the corresponding kernel physical mapping address (kmalloc assigned address); Map_nopage () When the process accesses a VMA page that does not currently exist, it looks for the physical page that corresponds to the address and returns a pointer to the page.

TEST.c it uses the corresponding device files of the above driver module to read the kernel memory in the user space. As a result, you can see the contents of the kernel virtual address (ok!), which is displayed on the screen.

Perform steps

Compile map_driver.c for MAP_DRIVER.O module, see makefile for specific parameters

Load module: Insmod MAP_DRIVER.O

Generate the corresponding device files

1 find map_driver corresponding device and device number under/proc/devices: grep mapdrv/proc/devices

2 Setting up the device file Mknod mapfile C 254 0 (the device number is 254 in my system)

Use Maptest to read the Mapfile file and print the information from the kernel to the screen.

All Programs Download Mmap.tar (thanks to Martin Frey, the main body of the program inspired by him)

[1] allocating memory statically is when the compiler allocates memory according to the source program when compiling the program. Allocating memory dynamically is when the runtime calls the run-time library function to allocate memory after the program compiles. Static allocations are fast, efficient, but limited in size before the program runs. Dynamic allocation is performed when the program is running, so it is slow but highly flexible.

[2] The term "BSS" has been for some years, and it is the abbreviation for block started by symbol . Because uninitialized variables do not have a corresponding value, they do not need to be stored in an executable object. But because the C standard enforces that uninitialized global variables are given special default values (basically 0 values), the kernel is going to load variables (unassigned) from executable code into memory, then map 0 pages to that slice of memory, and these uninitialized variables are given a value of 0. This avoids the explicit initialization of the target file and reduces wasted space (from Linux kernel development)

[3] There are also cases where memory continuity must be required, such as memory used in DMA transmissions, which must be allocated continuously because the page mechanism is not involved.

[4] The idea of this storage pool is widely used in computer science, such as database connection pools, memory access pools, and so on.

Linux memory Management "Go"