Basic memory management knowledge of Linux system explained

memory is one of the most important resources managed by the Linux kernel. The memory management system is the most important part of the operating system, because the system's physical memory is always less than the amount of memory required by the system. Virtual memory is the strategy used to overcome this contradiction. The system's virtual memory makes the system appear to have more memory capacity than the actual memory by sharing memory between processes. Linux supports virtual memory, which is the use of disk as an extension of RAM, so that available memory is effectively expanded accordingly. The core saves the currently unused memory blocks to the hard disk, freeing up memory for other purposes. When the original content is used again, read back to memory.

First, Memory usage monitoring

(1) Real-time monitoring of memory usage

Use the "free" command at the command line to monitor memory usage

#free

Total used free shared buffers Cached

mem:256024 192284 63740 0 10676 101004

-/+ buffers/cache:80604 175420

swap:522072 0 522072

The above gives a system of 256 megabytes of RAM and 512 megabytes of swap space. The third line of output (Mem:) Displays physical memory. The total column does not show the physical memory used by the core (usually about 1MB). The Used column shows the total amount of memory used (the second row is not buffered). The free column shows all memory that is not in use. The shared column displays the total amount of memory shared by multiple processes. The Buffers column displays the current size of the disk cache. The five-line (Swap:) shows the information similar to the above in the swap space. If this behavior is all 0, then swap space is not used. In the default state, the free command displays memory usage in kilobytes (that is, 1024 bytes). You can use the-H parameter to display memory usage in bytes, or you can use the-m parameter to display memory usage in megabytes. You can also use the-s parameter to monitor memory usage without interruption by using a command:

#free–b–s2

This command will continually report memory usage in the terminal window, updated every 2 seconds.

(2) combined watch and free command for real-time monitoring of memory usage:

#watch-N 2-d Free

Every 2.0s:free Fri Jul 6 06:06:12 2007

Total used free shared buffers Cached

mem:233356 218616 14740 0 5560 64784

-/+ buffers/cache:148272 85084

swap:       622584       6656      615928
 
 
Watch command executes free once every two seconds, clears the screen before execution, and displays the data in the same location. Because the watch command does not scroll the screen, it is suitable for long-time monitoring of memory utilization. You can use the-N option to control the frequency of execution, or you can use the-D option to have the command appear each time it is different. The watch command executes until you press [Ctrl]-[c].
 
Second, the concept of virtual memory
 
(1) Linux virtual Memory Implementation mechanism
 
The implementation of Linux virtual memory requires six kinds of mechanism support: address mapping mechanism, memory allocation recycle mechanism, cache and refresh mechanism, request page mechanism, switching mechanism, memory sharing mechanism.
 
The memory manager first maps the logical address of the user program to the physical address through the mapping mechanism, and when the user program runs, if it discovers that the virtual address in the program does not have corresponding physical memory, the request page request is made, and if there is free memory available for allocation, the request allocates memory ( The memory allocation and recycling is used, and the physical pages being used are recorded in the cache (using a caching mechanism). If there is not enough memory to allocate, then the swap mechanism is called to free up some of the memory. In addition, in the address mapping through the TLB (translation backup memory) to find the physical page, the exchange mechanism will also use the Exchange cache, and the physical page content Exchange to the swap file, but also to modify the page table to map the file address.
 
(2) virtual memory capacity setting
 
Maybe someone told you that you should allocate twice times the physical memory of virtual memory, but this is an irregular rule. If you have a small physical save, you can set this up. If you have 1G of physical memory or more, you can reduce the amount of virtual memory. Linux makes a lot of memory for the cache, but recycles it back when resources are tight. As long as you see swap is 0 or very small can be relieved, because the storage is not the biggest waste.

Third, make the Vmstat command to monitor virtual memory usage

Vmstat is an abbreviation for virtual meomory Statistics (dummy memory statistics) that monitors the operating system's virtual memory, processes, and CPU activity. It is the overall situation of the system statistics, the disadvantage is that a process can not be in-depth analysis. The command test is typically used with Vmstat 5 5, which represents 5 samples in 5 seconds. Will get a summary of the data it can reflect the real system situation.

#vmstat 5 5

procs-----------Memory-------------Swap-------io------System------CPU----

R b swpd free buff cache si so bi bo in CS US sy ID WA

1 0 62792 3460 9116 88092 6 30 189 89 1061 569 17 28 54 2

0 0 62792 3400 9124 88092 0 0 0 14 884 434 4 14 81 0

0 0 62792 3400 9132 88092 0 0 0 14 877 424 4 15 81 0

1 0 62792 3400 9140 88092 0 0 0 14 868 418 6 20 74 0

1 0 62792 3400 9148 88092 0 0 0 15 847 400 9 25 67 0

The output of the Vmstat command is divided into six parts:

(1) Process procs:
R: The number of processes waiting in the run queue.
B: The number of processes waiting for IO.

(2) Memory Memoy:

SWPD: Current swap memory available (in kilobytes).

Free: Idle memory (in kilobytes).

Buff: The amount of memory to buffer (in kilobytes).

Cache: The amount of memory that is used as a cache (in kilobytes).

(3) Swap swap page

Si: The number of swap pages, in kb/seconds, from disk swap to memory.

So: the number of swap pages from memory swap to disk, in kb/seconds.

(4) IO block device:

BI: The number of blocks sent to the block device, in blocks per second.

Bo: The number of blocks received from the block device, in blocks per second.

(5) System:

In: Number of interrupts per second, including clock interrupts.

CS: The number of environment (context) switches per second.

(6) CPU central Processor:
CS: The time that the user process used. expressed as a percentage.
SY: The time that the system process was used. expressed as a percentage.
ID: The idle time of the central processing Unit. expressed as a percentage.

If R is often greater than 4 and the ID is often less than 40, it means that the CPU is heavily loaded. If the Bi,bo is not equal to 0 for a long time, it indicates that the physical memory capacity is too small.

Iv. memory leaks and methods of recovering memory for Linux servers

1. Definition of memory leak:

In general, the memory leaks that we often say refer to the leaks in heap memory. Heap memory means that the program is allocated from the heap, arbitrarily sized (the size of the memory block can be determined during the program's run time), and the freed memory must be displayed after use. Applications typically use functions such as malloc,realloc,new to allocate a piece of memory from the heap, and after use, the program must be responsible for the corresponding call to free or delete to release the memory block, otherwise the memory cannot be reused, we say this memory leak.

2. Damage to memory leaks

From the user's point of view of using the program, the memory leak itself does not have any harm, as a general user, there is no sense of memory leaks. What is really harmful is the accumulation of memory leaks, which eventually consumes all the memory of the system. From this point of view, a one-time memory leak is harmless, because it does not accumulate, and the implicit memory leak is very harmful because it is more difficult to detect than the usual and sporadic memory leaks. Programs that have a memory leak problem can significantly reduce the performance of the program in addition to taking up more memory. If this is the case for the server, it can be severely impacted even if the system does not crash.

3. Memory Leak Detection and recovery

For problems such as memory overflow, you may encounter when you are writing more complex programs with pointers. Under Linux or UNIX, the C, C + + language is the most used tool. However, our C + + program lacks the means to detect memory information, and can only use the top command to observe the total dynamic memory of the process. And when the program exits, we are not aware of any memory leak information.

using the KILL command

Using the Linux command to reclaim memory, we can use PS, kill two commands to detect memory usage and recycle. Use the command "Ps" when using superuser privileges, which lists all running program names, and the corresponding process number (PID). The KILL command works by sending a system operation signal and a program's process number (PID) to the kernel of the Linux operating system.

Application Examples:

In order to efficiently reclaim memory you can use the command PS parameter V:

[[Email protected] ~]# PS V

PID TTY STAT time MAJFL TRS DRS RSS%MEM COMMAND

2542 tty1 ss+ 0:00 0 8 1627 428 0.1/sbin/mingetty Tty1

2543 Tty2 ss+ 0:00 0 8 1631 428 0.1/sbin/mingetty Tty2

2547 tty3 ss+ 0:00 0 8 1631 432 0.1/sbin/mingetty tty3

2548 Tty4 ss+ 0:00 0 8 1627 428 0.1/sbin/mingetty Tty4

2574 tty5 ss+ 0:00 0 8 1631 432 0.1/sbin/mingetty tty5

2587 tty6 ss+ 0:00 0 8 1627 424 0.1/sbin/mingetty Tty6

2657 tty7 ss+ 1:18 1710 29981 7040 3.0/usr/bin/xorg:0-br-a

2670 PTS/2 Ss 0:01 2 682 6213 1496 0.6-bash

3008 PTS/4 Ss 0:00 2 682 6221 1472 0.6/bin/bash

3029 PTS/4 s+ 0:00 2 1783 548 0.2 ping 192.168.1.12

3030 PTS/2 r+ 0:00 2 5134 768 0.3 PS V

Then, if you want to reclaim the memory of the ping command, use the command:

# Kill-9 3029

Using the tool software

Memprof is a very attractive and easy-to-use software that was founded by Red Hat's Owen Talyor. This tool is the Boehm-demers-weiser garbage collector for the gnome front end. This tool can be executed directly, and it works without any modification to the source code. When the program executes, the tool will graphically display the usage of memory.

Basic memory management knowledge of Linux system explained