How to familiarize yourself with Linux memory management

Memory is one of the most important resources managed by the Linux kernel, and the memory management system is the most important part of the operating system. For Linux beginners, it is very important to be familiar with Linux memory management.

A process is a program running in a virtual address space. It can be said that any program running in Linux is a process. Linux systems include interactive processes and batch processing processes. Interactive processes are controlled and run by shell, which can be run either on the foreground or in the background. The batch processing process does not belong to a terminal and is submitted to a queue for sequential execution. Most processes require virtual memory.
Memory size required

For a typical Linux application system, MB memory is a reasonable choice. If you do not run the X-window system, you can only use 8 MB of memory on a special-purpose machine (for example, the "crash and burn" system used to debug the device driver.

I have done some experiments. It takes almost the same time to compile the kernel at 9.1 MB and MB, with less than 3 minutes and a half (the author's Linux release version is Mandrake Linux and kernel 2.4.21 ). In a system with 8 MB memory, compilation takes a longer time. Multimedia Applications such as Web browsers run smoothly when the memory is sufficient, especially when the program is compiled and browsed online. Therefore, if you only have MB of memory, the expected performance will be reduced. Similarly, if you want to develop applications that consume a large amount of memory, more memory may be required. Therefore, the amount of memory required is determined by the work requirements.

Monitor memory usage in real time

1. Use the "free" command on the command line to monitor memory usage

# Free
Total used free shared buffers cached
Mem: 256024 192284 63740 0 10676 101004
-/+ Buffers/cache: 80604 75420
Swap: 522072 0 522072

The above Code provides a mb ram and a MB swap space system. The third row of output (MEM :) displays the physical memory. The total column does not display the core physical memory (usually about 1 MB ). The used column shows the total memory used (the second row does not count as a buffer ). The free column shows all unused memory. The shared column displays the total memory shared by multiple processes. The buffers column displays the current disk cache size. The fifth line (SWAP :) is used to swap the space. The displayed information is similar to the preceding one. If the behavior is 0, no swap space is used. By default, the free command displays memory usage in kilobytes (that is, 1024 bytes. Use the-H parameter to Display memory usage in bytes, or use the-M parameter to Display memory usage in MB. You can also use the-S parameter to continuously monitor memory usage:
# Free-B-S5
This command continuously reports memory usage in the terminal window and updates every 5 seconds.

2. Run the vmstat command to monitor virtual memory usage.

# Vmstat
Procs ----------- memory ---------- --- swap -- ----- Io ---- System -- ---- CPU ----
R B SWPD free buff cache Si so Bi Bo in CS us Sy ID wa
1 0 0 63692 10704 101008 0 239 42 126 48 45 7 0

The vmstat () command is a common monitoring program. It is short for virtual MEOMORY statistics (Virtual Memory statistics. If you do not use any command line parameters when using the vmstat command, a one-time report will be generated. The vmstat Command reports the main activity types: Process (procs), memory (in kilobytes), swap partition (in kilobytes), from Block devices (hard drive) the input output, system interruption (the number of times each second occurs), and the proportion allocated by the central processing unit (CPU) to the user, the system, and the idle time.

Virtual Memory Implementation Mechanism

The storage management subsystem is one of the most important components of the operating system. In the early computing era, because people needed much more memory than physical memory, a variety of strategies were designed to solve this problem. The most successful one was the virtual memory technology, it satisfies the memory space required by the system's limited physical memory competition process. By sharing memory among processes, the virtual memory seems to have more memory than the actual memory. Linux supports Virtual Memory, that is, the disk is used as the Ram Extension, so that the available memory can be effectively expanded accordingly. The core stores unused memory blocks on the hard disk to free up memory for other purposes. When the original content is used again, it is read back to the memory. Programs running on Linux only view a large amount of available memory, regardless of which part is on the disk. Of course, the read/write hard disk is slower than the real memory (about times), so the program runs slowly. This part of the hard disk used for virtual memory is called swap space.

The virtual memory technology not only allows us to use more memory, but also provides the following features:

1. Huge addressing space
The operating system makes the system look much larger than the actual memory. The virtual memory can be many times the actual physical space in the system. Each process runs in its own virtual address space. These virtual spaces are completely isolated from each other, so they do not affect each other. At the same time, the hardware Virtual Memory organization can set some areas of the memory as non-writable, which can protect code and data from malicious programs.

2. Fair physical memory allocation
The memory management subsystem allows every running process in the system to share the physical memory fairly.

3. shared virtual memory
Although the virtual memory allows the process to have its own virtual address space, it sometimes needs to share the memory between processes. For example, there may be several processes running the bash command shell program at the same time. To avoid the existence of bash program copies in the virtual memory space of each process, a good solution is that only one Bash copy exists in the physical memory of the system, and shared among multiple processes. The dynamic library is another way to share and execute code between processes. Shared memory can be used as a means of inter-process communication (IPC). Multiple processes exchange information through shared memory. Linux supports the shared memory IPC Mechanism of System V.

4. Process Protection
Each process in the system has its own virtual address space. These virtual address spaces are completely separated. Such a process does not affect other processes, and the virtual memory mechanism on the hardware is protected and the memory cannot be written. This prevents lost applications from overwriting code data.

5. Linux virtual memory Implementation Mechanism
Linux virtual memory requires support of six mechanisms: Address ing, memory allocation and recycling, cache and refresh, request page, swap, and Memory Sharing.

The Memory Manager maps the Logical Address of the user program to the physical address through the ing mechanism. When the user runs the program, if the virtual address used in the program does not have the corresponding physical memory, the request page is required. If there is idle memory available for allocation, it is requested to allocate memory (so the memory is allocated and recycled ), and record the physical pages in use in the cache (the cache mechanism is used ). If there is not enough memory for allocation, call the swap mechanism to free up some memory. In addition, in address ing, you need to use TLB to find physical pages. In the switching mechanism, you also need to use the swap cache and switch the physical page content to the swap file, you also need to modify the page table to map the file address. The implementation principle of Linux virtual memory is shown in Figure 1.

Figure 1 Linux virtual memory implementation principle

6. virtual memory capacity setting
Some people may say that the virtual memory capacity should be allocated 2 times the physical memory, but this is just a rule. If the physical memory is small, you can set it like this. If there is MB physical memory or more, the virtual memory can be reduced. Linux uses a large amount of memory for cache, but it will be reclaimed when resources are insufficient. As long as you see that swap is 0, or this number is very small, you can rest assured that the memory is not used is the biggest waste.

Memory leakage and memory recovery methods

1. Definition of Memory leakage
Generally, memory leakage refers to heap memory leakage. Heap memory refers to the memory allocated by the program from the heap, which is of any size (the size of the memory block can be determined during the running period), and the released memory must be displayed after use. In general, applications use functions such as malloc, realloc, and new to allocate a block of memory from the heap. after use, the program must call free or delete to release the block. Otherwise, the memory won't be used again, so we can say that the memory is leaked.

2. hazards of Memory leakage
From the user's perspective, memory leakage does not cause any harm. As a general user, there is no memory leakage. The real danger is the accumulation of Memory leakage, which will eventually consume all the memory of the system. From this perspective, one-time memory leakage does not cause any harm because it does not accumulate. The implicit memory leakage is very harmful, because it is more difficult to detect than the frequent and occasional Memory leakage. In addition to occupying more memory, programs with Memory leakage problems will also cause a sharp decline in program performance. If this happens to the server, even if the system does not crash, it will seriously affect the use.

3. Memory leakage detection and recovery
For memory overflow and other problems, you may encounter it when writing complex programs with many pointers. In Linux or UNIX, C and C ++ are the most commonly used tools. However, the C ++ program lacks proper means to detect memory information. It can only use the top command to observe the total amount of dynamic memory of the process. When the program exits, we cannot know any memory leakage information.

(1) use Linux commands to recycle memory. You can use the PS and kill commands to check memory usage and recycle memory. When you use superuser permission, run the "Ps" command to list the names of all running programs and the corresponding process ID (PID ). The working principle of the kill command is to send a system operation signal and a program process number (PID) to the Linux operating system kernel ).

The following example illustrates how to use the PS parameter V to efficiently recycle memory:

# Ps v
PID tty stat time majfl trs drs rss % mem command
2530 VC/1 s 104 6 1325 408 0.1/sbin/mingetty tty1
2531 VC/2 s 104 6 1325 408 0.1/sbin/mingetty tty2
2532 VC/3 s 104 6 1325 408 0.1/sbin/mingetty tty3
2533 VC/4 s 104 6 1325 408 0.1/sbin/mingetty tty4
2534 VC/5 s 104 6 1325 408 0.1/sbin/mingetty tty5
2535 VC/6 s 104 6 1325 408 0.1/sbin/mingetty tty6
2639 pts/1 s 545 16 2643 968 0.3 [Su]
2684 pts/1 s 361 586 2501 1592 bash
2711 pts/0 s 545 16 2643 968 0.3 [Su]
2714 pts/0 s 0: 00 361 586 2501 1592 bash
2754 pts/2 s 545 16 2643 968 0.3 [Su]
2757 pts/2 S 0: 00 361 586 2501 1592 bash
2818 pts/1 s 120 29 1478 480 0.1 Ping 192.168.1.7
2939 pts/2 R 0: 00 156 58 2469 644 0.2 PS-V

To recycle the memory of the ping command, run the following command:
# Kill-9 2818

(2) Use tool software
Memprof is an attractive and easy-to-use software created by Red Hat's Owen talyor. This tool is used by the gnome front-end Boehm-Demers-Weiser garbage collector. This tool can be executed directly without any modifications to the source code. During program execution, this tool displays the memory usage in a graphical manner. The working interface is shown in figure 2.

Figure 2 memprof

However, this tool can only run on x86 and PPC architecture Linux systems, and requires a complete GNOME environment. In this way, it cannot be used flexibly in all places. In addition, the development of this tool is also relatively slow, and the current version is 0.5.1.

The above describes the concept of Linux memory, the amount of memory required, the real-time monitoring of memory usage, the virtual memory implementation mechanism, and the memory leakage and recovery methods, it is expected to be helpful for Linux beginners to use the memory system efficiently.