Linux in Buffer/cache,swap, virtual memory and page + +

1. Buffer and Cache

The free command provides a more concise view of system memory usage relative to top:

[[email protected] ~]$ free-m--in megabytes
Total used free shared buffers Cached
mem:32109 30133 1975 0 472 16881
-/+ buffers/cache:12780 19328
swap:16063 10 16053

Mem: represents physical memory statistics.
-/+ buffers/cached: Cache statistics representing physical memory
Swap: Indicates the usage of the swap partition on the hard disk. Only Mem is actually occupied by the current process, that is, swap is not used without the buffers and cache.

See First line: Mem

Total Memory: 32109MB
Used number of memory used: 30133 MB
Free Memory Number: 1975
GKFX is currently obsolete. No, always 0.
Buffers Buffer Cache Memory: 472mb--indicates that the system has been allocated, but the buffer size is not used.
Cached Page Cache Memory: The 16881mb--system has been allocated, but the cache size has not been used.

Relationship: total = used + Free

Line 2nd:
The meaning of-/+ Buffers/cache is equal to:
-buffers/cache Memory: 12780 MB (equals 1th row of used-buffers-cached)
+buffers/cache Memory: 19328MB (equals 1th line of Free + buffers + cached)
The visible-buffers/cache reflects the memory that is actually eaten by the program, and +buffers/cache reflects the total amount of memory that can be appropriated.

Size of memory Allocated:

Used (MEM) = Used (-/+ buffers/cache) + buffers (MEM) + Cached (MEM)

30133MB = 12780 MB + 472MB + 16881MB

The third line is separate for the swap partition, so you don't have to say it again.

In order to improve disk access efficiency, Linux has done some careful design, in addition to the Dentry cache (for VFS, speed up the file path name to Inode conversion), but also adopted two main cache mode: Buffer cache and Page cache. The former is for the disk block read and write, the latter for the file inode read and write. These caches effectively shorten the time for I/O system calls (such as read,write,getdents).

If interested can further reference the file/proc/meminfo,free command is generated based on its information. The source of the free command can be obtained from procps-xxx-.src.rpm, XXX is the version number, such as procps-3.2.3-5.3.src.rpm.

In the Linux memory allocation mechanism, the first use of physical memory, when the physical memory is idle (enough), does not release its memory consumption, even if the memory-hogging program has been shut down, the program occupied by the memory used for cache use, for open programs, or read just accessed data will be faster.

2. Swap, virtual memory, and page

To understand this, first you need to know what is protected mode and real mode.

The previous operating system was a real mode, such as DOS. Each time only one process is running, this process uses all the physical memory. Later developed to the protection model, time-sharing multi-process. A CPU runs multiple processes, but the process does not know exactly how much memory is available and it can access the maximum memory address. For example, a 16-bit system can access the 2^16BYTE,32 bit is the 2^32 bit. But actually not that much memory? What to do? The protection model was born.

Assuming that the process is a Liu Xiang, the referee (System) starts running as soon as it is started. But the referee said to give you 1 seconds, you can run 100 meters. So Liu Xiang started running (memory address), a second after the Liu Xiang only ran 10 meters, the referee whistle said: Liu Xiang you first, I want to go to Wang Junxia whistle, now remember where you go (protection), and so will start from here. Referee to Wang Junxia whistle, one run a second, back to Liu Xiang whistle again run 1 seconds, so reciprocating. Fewer people start running, Liu Xiang can also stand on the runway to rest. The runners were running more and more, and the runways were full of people, so they could only move some people to the lawn next to the runway to rest (in exchange). Later found that some people disorder lying, occupy a lot of space, so that everyone can only stand (page), so that can accommodate a lot of people.

Swap-the meadow, which is the hard disk space to store page.
Virtual memory-Assuming that Liu Xiang run n-circle is already the distance to Beijing, but they are still in situ. There is no such thing as virtual.
Page-a lattice on the grass that accommodates only one person at a time.

In order to improve disk access efficiency, Linux has done some careful design, in addition to the Dentry cache (for VFS, speed up the file path name to Inode conversion), but also adopted two main cache methods:
Buffer Cache and Page cache. The former is read-write to the disk block, the latter for the file inode read and write. These caches effectively shorten the I/O system calls (e.g. Read,write,getdents

) of the time.

Memory activity can basically be quantified with 3 numbers: Total active virtual memory, exchange (swapping) rate and paging (paging) rate. The first number indicates the total memory demand, and the last two numbers indicate how much of the memory is in use. The goal is to reduce memory activity or increase the amount Until the paging rate remains at an acceptable level.

Total Active virtual memory (VM) = actual memory size (size of real memories) (physical memory) + Swap space Used (amount of swap spaces used)

When the program needs more memory than the physical memory, the UNIX system uses paging mechanism, that is, the system copy some in-memory pages to disk, freeing up space for the process to use. Most systems can tolerate occasional paging, but frequent paging can drastically degrade system performance.

UNIX Memory Management:The UNIX system uses 2 methods for memory management, "paging algorithm", "Switching technology".
Paging algorithm is to swap the most recently used pages in memory to disk, and keep the Frequently used pages (active pages) in memory for the process to use.
Switching technology is the system that swaps the entire process, not part of the page, to the disk. Under normal circumstances, some exchange processes occur.

When memory is critically low, paging and swapping are used frequently, which increases the load on disk I/O. It further reduces the execution speed of the system to the job, that is, the system I/O resource problem will affect the allocation of memory resources.

virtual Memory for UNIX:Unix virtual memory is a very complex subsystem, it realizes the transparency of inter-process code and data sharing mechanism, and can allocate more memory than the system's existing physical memory, some operating system virtual storage can even affect the performance of the file system by providing caching function. UNIX implementations of various styles differ greatly in the way they are implemented, but are inseparable from the following 4 concepts.

1: Actual memory
Actual memory refers to the physical memory that is actually present in a system, called RAM. Real memory is the fastest and most efficient way to store temporary data, so it must be distributed to the application as much as possible, and there are now several forms of RAM: SIMM, DIMM, Rambus, DDR, and many more RAM can use error correction mechanism (ECC).

2: Swap space (Swap)
Swap space is a piece of disk space dedicated to temporarily storing memory, typically used when paging and exchanging process data, and it is generally recommended that the swap space should be two to four times times the size of physical memory.

3: Page Scheduling
Page scheduling refers to the transfer of data from disk to memory, and the reverse process, which is called page scheduling, because UNIX memory is divided evenly into pages of equal size, usually with a page size of 4KB and 8KB (which can be viewed in Solaris with the PageSize command). When an executable program starts running, its image is swapped out of the disk on a page-by-page, similar to when some memory is idle for a period of time, it can be swapped out into the swap space, so that the free RAM can be given to other programs that need it to use.

4: Exchange
Page scheduling is usually easily confused with the concept of swapping,paging is the transfer of an idle portion of a process's memory to disk, whereas swapping refers to transferring the entire process to disk when the actual memory in the system is not sufficient to meet the new allocation requirements, which usually means that there is not enough memory.

Vmstat Monitoring memory Performance: This command is used to check the statistics of virtual memory and to display information about process status, idle and swap space, paging, disk space, CPU load and exchange, cache refreshes, and interrupts.
Test:

[Email protected] ~]# Vmstat
procs-----------Memory-------------Swap-------io------System-------CPU------
R b swpd free buff cache si so bi bo in CS us sy ID WA St
0 0 0 1592800 41124 319788 0 0 1084 95 791 375 6 28 57 10 0
[Email protected] ~]#

Procs
R:the number of processes waiting for run time.
The number of processes running and waiting (CPU time slices) to run, this value can also determine whether the CPU needs to be increased (longer than 1)

B:the number of processes in uninterruptable sleep.
The number of processes in a non-interruptible state, which is usually caused by IO

W:the number of processes swapped out but otherwise runnable.

Memory

Swpd:the amount of virtual memory used (KB).

Free:the amount of idle memory (KB).
Free physical Memory

Buff:the amount of memory used as buffers (KB).
Buffers the memory of the buffer cache, buffering the read and write of the block device

Cache:the amount of memory used as cache.

Swap

Si:amount of memory swapped in from disk (KB/S). Page import of virtual memory (import RAM from swap disk)

So:amount of memory swapped to disk (KB/S). Page export of virtual memory.

(from RAM to swap DISK)

1, page In/out: During the system operation, it generally does not lead to serious performance problems, even if your physical memory is large, it will inevitably appear more or less page in/out, which is the internal mechanism of the multitasking operating system decided;

2, swap in/out: In general, the system physical memory is relatively tense performance, it is the page in/out behavior of the further upgrade, page in/out is to indicate that the system memory is not enough, but only page or block and disk exchange, but page in/out this " Insufficient memory "is generally difficult to meet, but also dynamic.
3, page in/out and swap in/out The difference is: page in/out is the page or block between memory and disk exchange, but swap in/out is the entire process space exchange, which can produce serious performance problems.
4, in fact, PAGE, swap and SGA, PGA is not comparable ah, they should be linked, but can not be compared to discuss, the former is operating system memory management of a behavior, and the latter is the Oracle database of two and memory-related parameters, make an inappropriate analogy: like cooking and appetite;
4, in general, the SGA and PGA is based on the operating system memory management mechanism, but the specific DBMS buffer internal management, I am not particularly clear, but I personally think that the DBMS memory management and general software memory management is different, the difference is that, DBMS memory management is not entirely dependent on the OS, that is, the DBMS in memory management, autonomy is stronger, the various DBMS buffer management mechanism is not exactly the same, personal view, we discussed together.

Page in/out and swap in/out are operating system behaviors, which are designed to empty memory for other processes when memory is tight, which is not the same as when the application reads and writes memory from disk, that is, the database reads data from disk and writes out data to disk behavior, not a page In/out and Swap in/out, in books about operating system principles, should have this part.

Linux in Buffer/cache,swap, virtual memory and page + +