Linux memory missing page and replacement

1) page missingWhen the CPU requests a memory page that is not in Ram, a page is missing. For example, if we read/write data from memory but the data is not in memory, a page is missing. we use the following program to test the Memory Page shortage. The program allocates a large block of memory for the program to use. The program no longer uses the memory after only one access, it allocates memory through malloc, modifies 1 byte on each page, and then enters sleep state. note: Linux is very sensitive and does not provide any physical storage for pages that have not been modified. Therefore, we must read or write at least 1 byte in each page of an allocated area, to consume pages in memory. testing Program hog. c: # include <stdio. h> # include <string. h> # include <stdlib. h> # include <unistd. h> intmain (INT argc, char * argv []) {If (argc! = 2) Exit (0); size_t MB = strtoul (argv [1], null, 0); size_t nbytes = Mb x 0x100000; char * PTR = (char *) malloc (nbytes); If (PTR = NULL) {perror ("malloc"); exit (exit_failure);} size_t I; const size_t stride = sysconf (_ SC _page_size ); for (I = 0; I <nbytes; I + = STRIDE) {PTR [I] = 0;} printf ("allocated % d MB \ n", MB ); pause (); Return 0;} compile GCC hog. c-o hog view current memory free-m total used free shared buffers cachedmem: 503 206 296 0 30 140-/+ buffers/cache: 36 467 swap: 1027 0 1027 we use the GNU time command to view the number of page missing \ time. /hog 100 allocated 100 mbcommand terminated by signal 20.00 user 3.12 system. 52 elapsed 69% CPU (0 avgtext + 0 avgdata 0 maxresident) k0inputs + 0 outputs (0 major + 25719 minor) pagefaults 0swaps Note: 25719minor indicates 25719 page missing Times, 4 kb each time, exactly MB memory allocation. major indicates a primary missing page. A primary missing page indicates a missing page that must be input/output to the disk. minor indicates the next missing page. The next missing page is any other missing page. 2) ReplacementReplacement refers to the memory requested by the program, and when the physical memory is insufficient, the kernel will be forced to store the missing pages to the swap partition, that is, it replaces the least recently used pages (LRU) to swap. if swap is insufficient or swap is not available, memory allocation will fail. swapoff-. /hog 500 & [1] 2901 malloc: cannot allocate memory [1] + Exit 1. /hog 500 let's see the occurrence of replacement. First, check the current memory. swapon-afree-m total used free shared buffers cachedmem: 503 43 459 0 0 9-/+ buffers/cache: 33 469 swap: 1027 0 1027 now there is 469mb free space, in this case, we use the hog program to request MB of memory space, which will be replaced, as shown in \ time. /hog 500 allocated 500 mb (here, the program pause is interrupted with Ctrl + C to see the following information. In this case, we can view the memory space on another Terminal) command terminated by signal 20.02 user 3.53 system. 19 elapsed 49% CPU (0 avgtext + 0 avgdata 0 maxresident) k0inputs + 0 outputs (0 major + 128142 minor) pagefaults 0swaps on another terminal to view the memory as follows: free-m total used free shared buffers cachedmem: 503 497 6 0 0 3-/+ buffers/cache: 493 9 swap: 1027 53 974 here we see a MB memory replacement. return to the first terminal and we can see that the missing page information is: 0 major + 128142minor. This 0 master missing page refers to the primary missing page only when the process requests a page that resides on the disk, in this case, the pages do not exist, so they do not reside on the disk, so they are not counted as the number of Primary missing pages. although the hog process causes the system to write pages to the disk, but in fact it does not write those pages to the disk, the actual write is completed by kswapd. the kswapd kernel thread is responsible for migration of data from the memory to the disk. Only when the processes that have those pages call them again will a primary missing page be generated. 3) Top CommandThe top command is a periodic update. the following describes how to use top commands. 1) switch the display command name and complete command input C2) sort by resident memory size input m3) sort by CPU usage percentage input P4) sort by time/accumulative time input t5) display thread input H6) divide the top information into four screens, respectively def, job, mem, USR input SHIFT + A1: def-12:50:16 up, 2 users, load average: 0.00, 0.01, 0.00 tasks: 72 total, 2 running, 70 sleeping, 0 stopped, 0 zombiecpu (s): 0.0% us, 5.6% Sy, 0.0% Ni, 94.4% ID, 0.0% wa, 0.0% hi, 0.0% Si, 0.0% STMEM: 515600 k total, 72684 K used, 442916 K free, 33 72 K buffersswap: 1052248 k total, 23140 K used, 1029108 K free, 48128 K cached1 PID user PR Ni virt res shr s % CPU % mem time + command 138 root 13-5 0 0 0 s 0.0 0.0. 23 [kswapd0] 2569 root 15 0 8216 664 436 R 5.6 0.1. 69 sshd: root @ pts/0, pts/1 2463 root 18 0 1924 160 140 s 0.0 0.0. 32 Hald-addon-storage: polling/dev/HDC 2405 root 15 0 25188 6180 1792 s 0.0 1.2. 23/usr/bin/Python/usr /Sbin/Yum-updatesd 2439 haldaemo 18 0 5348 640 392 s 0.0 0.1. 81 Hald 1 root 15 0 2032 160 s 140 0.0. 18 init [3] 3234 root 15 0 2164 1000 r 796 0.0 0: 00. 95 top 2422 avahi 15 0 3696 352 260 s 0.0 0: 00. 94 avahi-daemon: Running [test1.local] 323 root 10-5 0 0 0 s 0.0 0.0. 75 [kjournald] 2 PID ppid time + % CPU % mem PR Ni s virt swap res UID Command 3234 2721. 95 0.0 0.2 15 0 r 2164 1164 1000 0 top 2721 2569. 29 0.0 0.2 15 0 s 4620 3812 0 bash 808 2571. 45 0.0 0.1 16 0 s 4616 4020 0 bash 596 2569. 69 5.6 0.1 15 0 r 8216 7552 664 0 sshd 2539 1. 01 0.0 0.0 24 0 s 1624 1524 100 0 mingetty 2516 1. 01 0.0 0.0 24 0 s 1624 1524 100 0 mingetty 2515 1. 01 0.0 0.0 24 0 s 1628 1528 100 0 mingetty 2514 1. 00 0.0 0.0 18 0 s 1624 1524 100 0 m Ingetty 2511 1. 00 0.0 0.0 18 0 s 1624 1524 100 0 mingetty 3 PID % mem virt swap res code data SHR nflt mcm t s pr Ni % CPU command 2405 1.2 25188 18 m 6180 4 8688 1792 0 s 15 0 0.0 Yum-updatesd 3234 0.2 2164 1164 1000 344 796 0 0 r 15 0 0.0 top 1866 0.2 9612 8672 4 940 1664 72 0 s 12-3 512 Python 0.0 2721 4620 3812 808 684 400 620 77 0 s 15 0 0.0 bash 2235 0.1 12716 11 m 672 4 3656 376 75 0 s 15 0 0.0 Python 2569 0.1 8216 7552 664 368 736 90 0 r 15 0 436 sshd 5.6 2439 0.1 5348 4708 640 260 1908 66 0 s 18 0 392 Hald 0.0 2571 0.1 4020 596 684 396 452 84 0 s 16 0 0.0 bash 1549 0.1 2280 1768 512 432 31 0 s 15 0 420 dhclient 4 PID ppid uid user ruser tty time + % CPU % mem s command 2371 1 43 XFS?. 07 0.0 0.0 s XFS 1941 1 32 RPC?. 01 0.0 0.0 s Portmap 2405 1 0 root?. 23 0.0 1.2 s Yum-updatesd 3234 2721 0.0 0 root pts/1. 95 0.2 1866 R top 1864 0 Root?. 30 0.0 0.2 s Python 2721 2569 0 Root pts/1. 29 0.0 0.2 s bash 2235 1 0 Root?. 24 0.0 0.1 s Python 2569 2271 0 root? 0: 02. 69 5.6 0.1 R sshd 2571 2569 0 root pts/0. 45 0.0 0.1 s bash allows you to customize the display field of the screen to be defined. For example, if you want to add the time field to the mem screen, perform the following operations. enter W to switch the window, set the current screen to Mem, and then enter F to see the following information. current fields: anopqrstuvbcdefgjlmyzwhikx for window 3: memtoggle fields via field letter, type any other key to return * A: pid = process ID * n: % mem = memory usage (RES) * O: virt = virtual image (Kb) * P: swap = swapped size (Kb) * Q: res = resident size (Kb) * R: code = code size (KB) * s: Data = Data + stack size (Kb) * T: SHR = shared mem size (Kb) * u: nflt = page fault count * V: ndrc t = dirty pages Count B: ppid = parent process PID C: ruser = real user name D: uid = user ID e: User = user name f: Group = group name G: tty = controlling tty J: P = last used CPU (SMP) L: time = CPU time M: Time + = CPU time, hundredths Y: wchan = sleeping in Function Z: flags = task flags <sched. h> * W: S = Process status * H: Pr = Priority * I: ni = nice value * K: % CPU = CPU usage * X: command = command name/line enter L, indicating to select L: time = CPU time. After selection, the * number will appear before the option, for example, * l: time = CPU time, enter return to view the time information on the mem screen, as shown below: 3 PID % mem virt swap res code data SHR nflt MCM t time s PR Ni % CPU command 2405 1.2 25188 6180 18 m 8688 4 1792 264 0.0 0 0 002 s 15 0 3234 Yum-updatesd 0.2 2164 1164 1000 52 344 796 0 0 0 0 01 R 15 0 0.7 top 1866 0.2 9612 8672 4 940 1664 72 0 0 0 0 s 12-3 512 Python 0.0 2721 0.2 4620 3812 808 684 400 77 0 0: 00 s 15 0 0.0 bash 2235 0.1 11 m 12716 4 672 3656 75 0 0 0 0 s 15 0 376 Python 0.0 2569 0.1 8216 7552 664 90 0 0 0 0 02 s 15 0 368 sshd 2439 0.1 5348 4708 640 260 1908 66 0 001 01 S 18 0 392 Hald 0.0 2571 0.1 4616 4020 596 684 396 84 0 0 0 0 s 16 0 452 bash 0.0 1549 0.1 2280 1768 512 432 420 392 31 0 s 19 0 0.0 dhclient same, we can also sort by specified columns. For example, we want to sort the mem screen by virt (Virtual Memory) and perform the following operations: Input O (uppercase), indicating sorting. only row N: % mem = memory usage (RES) has a number *, indicating the percentage of memory selected. we input O (uppercase), indicating that the selected O: virt = virtual image (Kb) is as follows: Current sort field: N for window 3: memselect sort field via field letter, type any other key to return a: pid = process id B: ppid = parent process PID C: ruser = real user name D: uid = user ID e: User = user name F: group = group name G: tty = controlling tty H: Pr = Priority I: ni = nice value J: P = last used CPU (SMP) K: % CPU = CPU usage L: time = CPU time M: Time + = CPU time, hundredths N: % mem = memory usage (RES) * O: virt = virtual image (Kb) P: swap = swapped size (Kb) Q: res = resident size (Kb) R: code = code size (Kb) S: Data = Data + stack size (Kb) T: SHR = shared mem size (Kb) U: nflt = page fault count V: mcm t = dirty pages count W: S = Process status X: command = command name/line Y: wchan = sleeping in Function Z: Flags = task flags <sched. h> after you press enter to return to the home screen, virt is sorted as follows: 3 PID % mem virt swap res code data SHR nflt MCM t time s PR Ni % CPU command 2405 1.2 25188 6180 18 m 8688 4 1792 264 0.0 0 0 002 s 15 0 2235 Yum-updatesd 0.1 12716 11 m 672 4 3656 376 75 0 0 0 s 15 0 0.0 Python 2141 0.0 12 m 12692 84 10 m 244 1 0 0 0 s 25 0 188 pcscd 0.0 1864 0.1 11 m 12048 92 10 m 220 14 0 s 16-3 0.0 auditd 2252 0.0 9644 9484 160 364 644 156 5 0 0 0 0 s 18 0 0.0 cupsd 1866 0.2 9612 8672 940 4 1664 512 72 0 0 0 0 0 s 12-3 0.0 Python 2190 0.1 9332 8868 464 196 7168 37 0 0 0 0 s 25 0 360 automount 0.0 2569 0.1 8216 7552 664 368 90 0 0 0 0 0 02 s 15 0 736 sshd 436 0.0 2439 4708 640 260 1908 392 66 0 s 18 0 0.0 Hald 4) Comprehensive instances of memory missing pages and replacementTo complete the last test, we changed the previous program and added signal processing and time output, as shown below: # include <stdio. h> # include <string. h> # include <stdlib. h> # include <signal. h> # include <time. h> # include <unistd. h> # include <sys/time. h> void handler (INT sig) {}# define timespec2float (TV) (double) (TV ). TV _sec + (double) (TV ). TV _nsec * 1e-9) intmain (INT argc, char * argv []) {If (argc! = 2) Exit (0); signal (SIGUSR1, Handler); size_t MB = strtoul (argv [1], null, 0); size_t nbytes = Mb x 0x100000; char * PTR = (char *) malloc (nbytes); If (PTR = NULL) {perror ("malloc"); exit (exit_failure);} int val = 0; const size_t stride = sysconf (_ SC _page_size); While (1) {int I; struct timespec T1, T2; clock_gettime (clock_realtime, & T1); for (I = 0; I <nbytes; I ++ = STRIDE) {PTR [I] = Val ;}val ++; clock_gettime (clock_realtime, & T2); printf ("touched % d MB; in %. 6f sec \ n ", MB, timespec2float (T2)-timespec2float (T1); pause () ;}return 0 ;}compile the source program and create two soft links for this program, below: gcc-O2-O son-of-hog son-of-hog.c-lrtln-s son-of-hog-a1- s son-of-hog-B view the current memory: free-m total used free shared buffers cachedmem: 503 185 317 0 10 140-/+ buffers/cache: 35 468 swap: 1027 0 1027 note: at this point, we can see that there is MB of free memory, which includes MB of cached and 10 MB of buffers to clear the swap space (SWAP): swapoff-aswapon-afree-m total used free shared buffers cachedmem: 503 50 452 0 1 16-/+ buffers/cache: 33 470 swap: 1027 0 1027 we run hog-a to occupy MB of memory space, view the memory allocation time :. /hog-a 300 & [1] 2592 touched 300 MB; In 2.784745 sec note: the memory allocated for 2.7 MB shares more than seconds. there is a page change and cache reclaim time. using a SIGUSR1 for this process will wake up again and access the memory: Kill-usr1 % 1 touched 300 MB; In 0.010827 sec Note: only 10 ms is used here. run hog-B to see how much time it will take :. /hog-B 300 & [2] 2601 touched 300 MB; In 4.432349 sec Note: as physical memory is allocated, space is allocated to hog-B, the system performs replacement to allocate physical memory to hog-B and replace the memory occupied by hog-a with swap. send the SIGUSR1 signal to hog-B and wake up the program again. At this time, only 143 mspkill-usr1 hog-btouched 300 mb is used. In 0.143772 sec, we use the top command to view: top-p $ (pgrep hog-a)-P $ (pgrep hog-B) 1: def-06:12:26 up 10 min, 1 user, load average: 0.01, 0.10, 0.10 tasks: 2 Total, 0 running, 2 sleeping, 0 stopped, 0 zombiecpu (s): 0.0% us, 0.0% Sy, 0.0% Ni, 99.7% ID, 0.3% wa, 0.0% hi, 0.0% Si, 0.0% STMEM: 515600 k total, 509468 K used, 6132 K free, 956 K buffersswap: 1052248 k total, 162924 K used, 889324 K free, 11312 K cached1 PID user PR Ni virt res shr s % CPU % mem time + command 2601 root 18 0 301 m 300 m 428 s 0.0 0: 03. 32. /hog-B 300 2592 root 18 0 301 m 154 M 428 s 0.0 0: 02. 76. /hog-a 300 2 PID ppid time + % CPU % mem PR Ni s virt swap res UID Command 2601 2553. 32 0.0 59.7 18 0 S 301 m 1112 300 M 0 hog-B 2592 0: 02. 76 0.0 30.7 18 0 S 301 m 147 M 154 M 0 hog-A 3 PID % mem virt swap res code data SHR nflt mcm t s pr Ni % CPU command 2601 59.7 301 m 1112 300 M 4 300 m 428 235 0 s 18 0 0.0 hog-B 2592 30.7 301 m 147 M 154 M 4 300 m 428 0 0 s 18 0 0.0 hog-A 4 PID ppid uid user ruser tty time + % CPU % mem s command 2601 2553 0 root pts/0. 32 0.0 59.7 s hog-B 2592 2553 0 root pts/0. 76 0.0 30.7 s hog-A: 1) two programs use MB of virtual memory. (virt 301 m) 2) Because hog-A runs first, and then the occupied memory data is replaced with swap, it only occupies 154 MB of physical memory (res mb ), occupied 147 MB of swap space (SWAp ). 3) Because hog-B runs later, it uses the physical memory occupied by hog-A. At this time, it occupies 300 MB of physical memory (RES MB ), because it is replaced, 235 page errors (page fault count) are generated ).