Linux memory Bottleneck Analysis

In order to improve disk access efficiency, Linux has made some careful designs, in addition to caching dentry (for VFS, accelerating the conversion of file path names to inode ), two major cache methods are also adopted: buffer cache and page cache. the former is used to read and write disk blocks, and the latter is used to read and write inode files. these caches effectively shorten the time for I/O system calls (such as read, write, getdents.

Memory activity can basically be quantified using three numbers: total active virtual memory, swapping rate and paging rate. the first digit indicates the total memory demand, and the last two digits indicate the proportion of memory in use. the goal is to reduce memory activity or increase the amount of memory until the page rate remains at an acceptable level.

Total active virtual memory (VM) = actual memory size (size of real memory) + used swap space (amount of swap space used)

When the program requires more memory than the physical memory, the UNIX system uses the page adjustment mechanism, that is, the system copies some pages in the memory to the disk, and releases the space for the process to use.
Most systems can tolerate occasional page adjustments, but frequent page adjustments will cause a sharp drop in system performance.

UNIX memoryManagement: Unix systems use two methods for memory management: "Page adjustment algorithm" and "exchange technology ".
The paging algorithm is used to change pages that are not frequently used in the memory to disks and keep frequently used pages (Active pages) in the memory for the process to use.
Switching technology means that the system switches the entire process, not some pages, to the disk. Under normal circumstances, the system will have some exchange process.
When the memory is seriously insufficient, the system will frequently use paging and swap, which increases the disk I/O load. This further reduces the execution speed of the system on the job, that is, the system I/O resource issue will affect the allocation of memory resources.

UNIX virtual memory

UNIX virtual memory is a very complex subsystem that implements inter-processCodeThe transparency of the Data Sharing Mechanism and the ability to allocate more memory to the existing physical memory of the system. The virtual memory of some operating systems can even affect the performance of the file system by providing the cache function, there are many differences in the implementation methods of various types of UNIX virtual memory, but they are inseparable from the following four concepts.

1: actual memory
The actual memory refers to the physical memory actually exists in a system, which is called ram. The actual memory is the fastest and most effective way to store temporary data. Therefore, you must allocate it to applications as much as possible. There are multiple ram forms: Simm, dimm, Rambus, DDR, etc, many Ram users can use the error correction mechanism (ECC ).

2: swap space
Swap space is a disk space dedicated for temporary storage of memory. It is usually used for page scheduling and process data exchange. It is recommended that swap space be two to four times the size of physical memory.

3: Page Scheduling
Page scheduling refers to the process of transferring data from the disk to the memory, and the opposite. This process is called page scheduling because the Unix memory is evenly divided into large and small equal pages; the page size is usually 4 kb and 8 KB (you can use the pagesize command in Solaris to view the page size ). When an executable program starts to run, its image is converted from the disk page by page. Similarly, when some programs are idle for a period of time, they can be swapped out to the swap space, so that free RAM can be handed over to other programs that need it.

4: Exchange
Page scheduling is often confused with the concept of switching. Page scheduling refers to transferring the idle part of the memory occupied by a process to the disk, switching means that when the actual memory in the system is insufficient to meet the new allocation requirements, the entire process is transmitted to the disk. Switching activities usually mean that the memory is insufficient.

[Root @ localhost ~] # Vmstat-N 3 (refresh every 3 seconds)
Procs ----------- memory -------------------- swap ------ Io ---- -- System ---- ------ CPU --------
R B SWPD free buff cache Si so Bi Bo in CS us Sy ID wa
1 0 144 186164 105252 2386848 0 0 18 166 83 2 48 21 31 0
2 0 144 189620 105252 2386848 0 0 177 1039 34 10 56 0
0 0 144 214324 105252 0 0 0 10 2386848 32 5 63 0
0 0 144 202212 105252 0 0 0 2386848 189 20 3 77 0
2 0 144 158772 105252 2386848 0 0 203 1065 70 14 15 0

Memory
-Swap: Switch to swap memory (in KB by default)
If the value of swap is not 0, or it is still relatively large, for example, more than m, but the value of Si and so is 0 for a long time, we do not have to worry about this situation, it will not affect the system performance.

-Free: idle physical memory-Buff: Used as the buffer cache memory to buffer reading and writing of Block devices
-Cache: memory used as page cache, cache used by the File System
If the cache value is large, it indicates that the number of files in the cache is large. If files frequently accessed can be cached, the disk read Io BI will be very small. Swap
-Si: swap memory usage, transferred from disk to memory
-So: swap memory usage, transferred from memory to disk
When the memory is sufficient, these two values are both 0. If these two values are greater than 0 for a long time, the system performance will be affected, and disk I/O and CPU resources will be consumed. I found that some of my friends thought that the memory was not enough when they saw little or nearly zero free memory. In fact, I cannot simply look at this point. Linux is a memory-occupied OS, it also needs to be combined with Si and so. If there is little free, but Si and so are also few (mostly 0), you don't have to worry, and the system performance will not be affected at this time. From: http://space.itpub.net/8554499/viewspace-580301