Basic concepts of Linux memory management

1. Basic Concept 1.1 address

(1) Logical address: Refers to the part of the offset address generated by the program related to the segment. In the C language pointer, read the pointer variable itself value (& operation), which is actually the logical address, which is relative to your current process data segment address.
(2) Linear address: The offset address (logical address) in the segment, plus the base address of the corresponding segment, generates a linear address.
(3) Physical Address: The address placed on the addressing bus.
(4) Virtual address: The address of the offset in the next segment and paragraph in protected mode, and the logical address is the offset within the code segment, or the logical address of the process.

1.2 Memory

(1) Virtual Memory: a technology of computer system memory management. It allows the application to assume that it has contiguous available memory (a contiguous, complete address space), and in fact, it is usually separated into multiple physical memory fragments, and some are temporarily stored on external disk storage and exchanged for data when needed. Systems that use this technology make it easier to write large programs than systems that do not use virtual memory technology, and are more efficient at using real physical memory, such as RAM.
(2) Physical memory: actual memory. The physical address is divided into discrete units that become pages (page). Most systems currently have a page size of 4k.

1.3 Address Translation

Linux uses a section-page management mechanism with two parts for address translation: segmented parts and pagination parts.
(1) Segmented part: Converts the logical address to a linear address. Fragmentation provides a mechanism for isolating individual code, data, and stack areas, so that multiple programs (tasks) can run on the same processor without interfering with each other.
(2) Paging parts: The linear address is converted to Physical Address (page table and page directory), if the paging mechanism is not enabled, then the linear address is the physical address directly.

2. Memory allocation

The difference between Malloc,kmalloc and vmalloc?
(1) Kmalloc and Vmalloc are allocated kernel memory, and malloc allocates the user's memory.
(2) Kmalloc guarantee that the distribution of the existence of the physical is continuous, vmalloc is guaranteed to be in the virtual address space continuous.
(3) Kmalloc application of memory is relatively small, generally less than K. It is based on slab (memory pool) to speed up the efficiency of small memory applications.

3. Frequently Asked Questions

(1) After invoking the malloc function, will the OS allocate the actual memory space immediately?
A: No, only a virtual address is returned, and when the user wants to use memory, the OS issues a missing fault, and the memory management module allocates real memory for the program.

(2) The pros and cons of segment management and page management?
In the segment storage management, the address space of the program is divided into several segments (segment), so that each process has a two-dimensional address space, independent of each other, non-interference. A program is divided into several modules, such as code snippets, data segments, and shared segments. The advantage of this is that you can write and compile one file for the source program separately, and you can take different protections for different types of segments, or you can share them by segment. The advantage of segment storage management is that there is no inner fragment, outer fragments can be eliminated by memory crunch, and memory sharing is easy to implement.
In page storage management, the logical address space for a program is divided into fixed-size pages (page), and physical memory is divided into page boxes of the same size (pageframe). When the program loads, you can put any page in the memory of any one of the page boxes, the page boxes do not have to be contiguous, thus achieving a discrete allocation. The advantage of this management approach is that there is no external fragmentation and a program does not have to be stored continuously. This makes it easy to change the size of the program footprint.
There are many similarities between page-and segment-style systems. For example, both use discrete allocation, and address mapping mechanism to achieve address transformation. But there are many differences between the two concepts, mainly in:

• A page is a physical unit of information that is paged out in order to achieve a discrete allocation method to reduce the amount of memory and improve memory utilization. Or, paging is simply due to the need for system management, not the needs of the user. A segment is a logical unit of information that contains a set of information that is relatively complete in its meaning. The purpose of segmentation is to better meet the needs of users.
• The size of the page is fixed and determined by the system, the logical address is divided into page number and in-page address two parts, is implemented by the machine hardware. The length of the segment is not fixed and is determined by the user to write the program, usually by the compilation system when the source program compiled according to the nature of the information to be divided.
• The page system address space is one-dimensional, that is, a single linear address space, where programmers can represent an address simply by using an identifier. The job address space of the segment is two-dimensional, when the programmer identifies an address, it needs to give both the segment name and the address in the paragraph.

(3) Under what circumstances does malloc invoke mmap?
From the operating system perspective, the process allocates memory in two ways, with two system invocations: BRK and mmap (regardless of shared memory). BRK is to push the highest address pointer of the data segment (. data) _edata toward the high address, mmap is to find an idle piece in the virtual address space of the process (usually the middle of the heap and stack). Both of these methods allocate virtual memory and no physical memory is allocated. In the first access to the allocated virtual address space, a page break occurs, the operating system is responsible for allocating physical memory, and then establish a mapping between virtual memory and physical memory.
In the standard C library, Malloc/free function allocations are provided to release memory, which is implemented by brk,mmap,munmap these system calls.
By default, the malloc function allocates memory, and if the request memory is greater than 128K (which can be adjusted by the M_mmap_threshold option), it is not to push the _edata pointer, but instead use the MMAP system call to allocate a piece of virtual memory from the middle of the heap and stack. This is done primarily because the memory allocated by the BRK needs to wait until the high address memory is freed (for example, A is impossible to release before B is released), and the memory allocated by MMAP can be freed separately.

On Linux systems, when a program is loaded into memory, the kernel establishes code snippets, data segments, and stack segments for the user process address space, which is used for dynamic memory allocation between the data segment and the stack segment.

The member variables Start_code and End_code in the kernel data structure mm_struct are the starting and ending addresses of the process snippet, Start_data and End_data are the start and end addresses of the process data segment, and Start_stack is the process stack segment start address , START_BRK is the process dynamic memory allocation start address (the start address of the heap), and a BRK (the current last address of the heap), which is the current terminating address of the dynamic memory allocation.

The basic function of C language dynamic memory allocation is malloc (), and the basic implementation on Linux is called by the kernel's BRK system. BRK () is a very simple system call, simply changing the value of the member variable BRK of the MM_STRUCT structure.

The MMAP system call implements a more useful dynamic memory allocation function, which can map all or part of a disk file to a user space, and the process of reading and writing files becomes a read-write operation. The Do_mmap_pgoff () function in the Linux/mm/mmap.c file is the core of the MMAP system call implementation. The code for Do_mmap_pgoff () simply creates a new vm_area_struct structure and assigns the parameter of the file structure to its member variable m_file, and does not actually load the contents into memory. (Excerpt from http://blog.csdn.net/ugg/article/details/4344522)

(4) 32-bit systems, typically, what is the maximum virtual address and physical address space?
Without PAE, the maximum virtual address and physical address space are 4G, and if PAE is used, the maximum virtual address is still 4G, while the physical address space can become 64G (x86, 32 to 36 bits).

(5) How to achieve malloc and free?
MALLOC implementations may consider using the buddy algorithm +slob algorithm, which is similar to free.

Transferred from: http://dongxicheng.org/os/linux-memory-management-basic/

Basic concepts of Linux memory management