"Essays" Memory performance monitoring related

The "Memory" here includes physical memory and virtual memory, which extends the computer's memory space to the hard disk, the physical memory (RAM) and a portion of the hard disk space (SWAP) together as virtual memory provides a consistent virtual memory space for the computer. The advantage is that we have "more" memory, you can run more and larger programs, the disadvantage is that some hard disk when the overall performance of memory, hard disk read and write slower than memory several orders of magnitude, and the exchange between RAM and swap increased the burden of the system.

In the computer, all programs are running in memory, and when running the program requires insufficient memory, memory management will transfer some inactive pages to virtual memory.

Virtual memory is a technology of computer system memory management. It makes the application think it has contiguous memory available. In fact, it is usually separated into multiple physical memory fragments, and some are temporarily stored on the external disk storage and exchanged for data when needed.

In different systems, virtual memory technology is applied. In Windows, called "virtual Memory", in Linux, it's called "Swap space."

The Linux kernel reads and writes virtual memory as a "page", and in the x86 system, each page size is 4KB. The memory is transferred to the hard disk swap space (swap) and read from the swap space to the memory by the page to read and write. This exchange process for memory and swap is called paging.

In a program run, because the kernel reads data from the cache much faster than the data on the hard disk, it will save some critical data to the cache, which is called a "dirty page" when the process changes the page-based data in the cache. The kernel will write the dirty page data to disk at the appropriate time to keep the data in the cache consistent with the data in the disk.

Here are two key processes for kernel management memory KSWAPD and Pdflush:

　　Kswapd

function is used to manage memory

The KSWAPD process is used to check the pages_hight and Pages_low two thresholds, and when the system is free of memory below Pages_low, the KSWAPD daemon is awakened, KSWAPD will start scanning and attempt to release the page. and repeat the process until the available memory is greater than pages_hight.

3 things are checked when scanning KSWAPD:

1, if the page has not been modified, this will put the page in the available memory list;

2, if the page is modified by the file system, the page content will be written to disk;

3, if the page has been modified, but not by the file system modification, this will write the page to swap space.

　　Pdflush

The function is used to synchronize the file-related memory pages, and synchronize the memory pages to the hard disk in time.

For example, open a file, the file is imported into memory, the file has been modified and saved, the kernel does not immediately save the file to the hard disk, by Pdflush decide when to write the corresponding page to the hard disk, which is controlled by a kernel parameter vm.dirty_background_ratio.

The Pdflush process is typically awakened by the following scenarios to write back the dirty page:

1. Timing mechanism:

The timer mechanism can wake up the Pdflush kernel process periodically and go back and forth some dirty pages. Available by command:

Cat/proc/sys/vm/dirty_writeback_centisecs

To see the timing period, in units of "1/100" seconds.

2. When the memory is low:

All dirty pages are not written to disk at this time, and when you write about 1024 pages each time, you know that the free memory meets your needs.

3, when the write operation found that dirty pages more than a certain proportion

When the amount of system memory in a dirty page exceeds "/proc/sys/vm/dirty_background_ratio", the Pdflush process will wake up to write dirty pages, knowing that the proportions are below the set value.

4, the user calls the sync system call

The system wakes up the pdflush process until all dirty pages are written to disk.

Sometimes, when you view a process through the top command, you find that the KSWAPD process consumes a lot of CPU resources, which can cause system downtime, which is caused by the system discovering that memory resources are not sufficient to wake up the KSWAPD process to free up memory resources, just to optimize the system. Releasing a process that takes up too much memory is a good way to go.

"Essays" Memory performance monitoring related