Linux memory management and its basic concepts __linux

1. Basic Concepts

1.1 Address

(1) Logical address: the part of the offset address that is generated by the program in relation to the segment. In the C language pointer, read the value of the pointer variable itself (& operation), which is actually the logical address, which is relative to the address of your current process data segment.

(2) Linear address: The offset address (the logical address) in the paragraph, plus the base address of the corresponding segment, generates a linear address.

(3) Physical Address: The address placed on the addressing bus.

(4) Virtual Address: protection mode in the next paragraph and paragraph offset the composition of the address, and the logical address is within the Code section offset, or the process of the logical address.

1.2 Memory

(1) Virtual memory: A technique for memory management of computer systems. It makes the application think that it has contiguous available memory (a contiguous complete address space), which is usually separated into multiple physical memory fragments, and partly temporarily stored on external disk storage for data exchange when needed. Systems that use this technology make it easier to write large programs and use real physical memory (such as RAM) more efficiently than systems that do not use virtual memory technology.

(2) Physical memory: The actual memory. The physical address is divided into discrete units that become pages (page). Most systems currently have a page size of 4k.

1.3 Address Translation

Linux uses a Segment-page management mechanism with two parts for address translation: segmented parts and paging parts.

(1) Segmented part: Converts the logical address to a linear address. Fragmentation provides a mechanism for isolating individual code, data, and stack areas, so multiple programs (tasks) can run on the same processor without interfering with each other.

(2) Pagination part: Converts the linear address to the Physical Address (page table and page catalog), if the paging mechanism is not enabled, then the linear address is the physical address directly.

2. Memory allocation Kmalloc and Vmalloc is allocated the kernel of memory, malloc allocated is the user's memory kmalloc ensure that the distribution within the existence of physical is continuous, Vmalloc guaranteed in the virtual address space on the continuous, malloc does not guarantee anything (this is their own guess, not necessarily correct) Kmalloc can be allocated a limited size (generally less than 128 K. It is based on slab (memory pool) to speed up the small memory request efficiency), Vmalloc and malloc can allocate relatively large size memory only need to be DMA access when the physical continuous vmalloc is slower than Kmalloc

Comments:

1 What is the difference between kernel space and user space

Learning C language should be learned, from the user space, each process is silly to think that they have 4G of memory space, which is located in the high address (3G-4G) 1G space for the kernel, and the other 3G (0-3G) is it a person exclusive. So the user space is very generous to 3G of space divided into several areas, such as heaps, stacks, code snippets. where the malloc () allocates the space in the heap, and the automatic variable in the program, as you define in the function "int i", is placed on the stack at the same time. The stack of user space is variable stack, that is, with the increase of data, the stack space of corresponding function will increase.

With each user space process is not the same, the kernel only 1G of space, at the same time, in addition to their own processes running, the kernel will allow users to call the space process system calls into the kernel space to execute. So the kernel is pretty stingy about this, which stipulates that each process in the kernel has only a fixed-length stack of 4KB or 8KB (32-bit). For this reason, large data structures can not be allocated on the stack, only request the kernel to allocate new space to store data, such as function Kmalloc ().

2 Why the process in the kernel space has a stack, what is the role?

When a process executes a system call or is interrupted, it enters the kernel space, and the program in the kernel needs the stack for execution such as local variables, function calls, and so on. The function is similar to the stack effect in the user state.

3. Frequently Asked Questions

(1) When the malloc function is invoked, the OS allocates the actual memory space immediately.

A: No, will only return a virtual address, when users want to use memory, the OS will issue a page break, at this time, the memory management module will allocate real memory for the program.

(2) The advantages and disadvantages of segment management and page-style management.

In segment storage management, the address space of the program is divided into several segments (segment), so that each process has a two-dimensional address space, independent of each other and non-interference. The program is divided into several modules, such as code snippets, data segments, shared segments. The advantage of this is that you can write and compile one file for the source program separately, and you can take different protections for different types of segments, or you can share them by segment. The advantage of segment storage management is that there is no internal fragmentation, the outer fragments can be eliminated by the memory tightening, and easy to realize memory sharing.

In page-type storage management, the logical address space of a program is divided into a fixed-size page, and the physical memory is divided into page frames of the same size (pageframe). When the program loads, you can put any page in the memory of any page box, these page boxes do not have to continuous, thus achieving a discrete allocation. The advantage of this management approach is that there is no external fragmentation and that a program does not have to be stored continuously. This makes it easy to change the size of the program footprint.

There are many similarities between the page-and-segment systems. For example, both are distributed in discrete ways, and address mapping is implemented through address map mechanisms. But the concept of the two also has a lot of differences, mainly in: [1] The page is the physical unit of information, paging is to achieve the discrete allocation method to reduce the amount of memory, improve memory utilization. Or, paging is only due to the need for system management, not the needs of users. A segment is a logical unit of information that contains a set of information that is relatively complete in meaning. The purpose of segmentation is to better meet the needs of users.

[2] The size of the page is fixed and determined by the system, the logical address divided into page number and page address two parts, is realized by the machine hardware. The length of the segment is not fixed, and it is decided by the user to write the program, usually by the compilation system in the source program compiled according to the nature of the information.

[3] The page system address space is one-dimensional, that is, a single linear address space, the programmer can only use an identifier to represent an address. Segmentation of the job address space is two-dimensional, the programmer in the identification of an address, both the need to give the paragraph name, but also to give a paragraph within the address.

(3) malloc under what circumstances to invoke mmap.

From the operating system point of view, there are two ways in which the process allocates memory, which is done by two system calls: BRK and mmap (regardless of shared memory). BRK is to push the top address pointer of the data segment (. data) _edata to the high address, mmap to find an idle one in the virtual address space of the process (typically the heap and the stack). Both of these methods are allocated virtual memory, and no physical memory is allocated. When the first access to the allocated virtual address space, there is a page break, the operating system is responsible for allocating physical memory, and then establish a mapping between virtual memory and physical memory.

In the standard C library, the Malloc/free function allocation frees up memory, which is implemented by brk,mmap,munmap these system calls.

By default, the malloc function allocates memory, and if the request memory is greater than 128K (which can be adjusted by the M_mmap_threshold option), it is not to push the _edata pointer, but to use the MMAP system call to allocate a piece of virtual memory from the middle of the heap and the stack. This is mainly because the memory allocated by BRK will not be released until high address memory is released (for example, A is impossible to release before B is released), and Mmap allocated memory can be freed separately.

(4) 32-bit system, usually, the maximum virtual address and physical address space for how much.

Without PAE (Physical address Extension), the maximum virtual and physical addresses are 4G, and if PAE is used, the maximum virtual address is still 4G and the physical address space can become 64G (x86, 32 to 36 bits).

(5) How to achieve malloc and free.

malloc implementation can be considered using the buddy algorithm +slob algorithm, free similar.

Comments:

1)Buddy algorithm

Group all the free page boxes into 10 block lists, each block element of each block list contains 1,2,4,8,16,32,64,128,256,512 consecutive page boxes, and the physical address of the first page box of each block is an integer multiple of the block size. For example, a block with a size of 16 page boxes whose starting address is a multiple of the 16*2^12 (the size of a page box is the size of the 4k,16 page box is 16*4k,1k=1024=2 10, 4k=2 12).

For example, suppose you want to request a block of 128 page frames, the algorithm first checks whether the list of 128 page boxes have free blocks, and if not, check the list of 256 page boxes, then divide 256 pages into two pieces, one for use, and one for the list of 128 page boxes. If not, check the list of 512 page boxes, some words will be divided into 128,128,256, a 128 use, the remaining two insert corresponding linked list. If not found in 512, the error signal is returned.

4. Reference materials

Linux Memory Management Notes:

Http://vmlinz.is-programmer.com/posts/26540.html

Memory-Segment Page management notes:

Http://chenxu.yo2.cn/articles/linux_memory.html

Conversion of virtual address, linear address and physical address:

http://dogking.chinaunix.com/space.php?uid=23208702&do=blog&id=163527

The difference between Kmalloc, Vmalloc and malloc:

http://blog.csdn.net/macrossdzh/article/details/5958368

Physical and virtual storage space layouts in Linux:

http://book.chinaunix.net/showart.php?id=3266

[Baidu share] The analysis of the performance problems caused by the frequent allocation of free memory

Http://topic.csdn.net/u/20100325/16/0b86c0ed-5b8d-4eec-a757-c782ae9a3a35.html

The difference between Kmalloc, Vmalloc and malloc

http://blog.csdn.net/macrossdzh/article/details/5958368 buddy algorithm: Http://baike.baidu.com/view/1040266.htm?tp=2_11

5. PostScript

Recent interviews are often asked about some of the basics of memory management, especially to organize this article, as a small summary of this stage interview.

original articles, reproduced please specify: reprinted from Dong's Blog

This article link address: http://dongxicheng.org/os/linux-memory-management-basic/