Linux Memory Usage Tuning __linux

Some time ago when doing the player, encountered a problem, spent a long time, make a record, want to help people in need:

Player's broadcast video, whether it is manual switch video or to video playback completed, automatic switching video, a certain number of times after the black screen phenomenon, occasionally have a sound, problems occur, can not be restored, kernel output as follows log:

DMA free:71672kb min:616kb low:768kb high:924kb active_anon:0kb inactive_anon:0kb active_file:0kb Inactive_file:0kB UNEVICTABLE:0KB isolateds

Lowmem_reserve[]: 0 1244 1593 1593

Normal free:1249804kb min:4212kb low:5264kb high:6316kb active_anon:0kb inactive_anon:0kb Active_file:0kB inactive_ FILE:0KB unevictable:0kb o

Lowmem_reserve[]: 0 0 2794 2794

Highmem free:223000kb min:348kb low:640kb high:936kb active_anon:39248kb inactive_anon:140kb Active_file:61020kB INACTIVE_FILE:34108KB Unevio

Lowmem_reserve[]: 0 0 0 0

DMA:22*4KB 90*8kb 21*16kb 82*32kb 7*64kb 137*128kb 69*256kb 3*512kb 8*1024kb 5*2048kb 3*4096kB 0*8192kB 0*16384kB 0*32768 KB = 71672kB

NORMAL:1*4KB 53*8kb 16*16kb 6*32kb 6*64kb 4*128kb 1*256kb 1*512kb 2*1024kb 2*2048kb 3*4096kb 2*8192kB 4*16384kB 35*32768k B = 1249772kB

HIGHMEM:14*4KB 10*8kb 437*16kb 386*32kb 374*64kb 27*128kb 10*256kb 7*512kb 8*1024kb 7*2048kB 4*4096kB 2*8192kB 1*16384kB 3*32768KB = 223000B

23829 Total Pagecache Pages

0 pages in swap cache

Swap Cache Stats:add 0, delete 0, find 0/0

Free swap = 0kB

Total Swap = 0kB

524288 pages of RAM

387015 free Pages

73287 reserved Pages

1169 Slab Pages

8545 pages Shared

0 pages Swap Cached

Physical memory allocation error!

Physical memory allocation error!

Through log, the position to the output is the decoding drive, and the driver outputs the above information after the DMA request fails. In response to this phenomenon, according to the following several ideas of the investigation:

1. Memory leaks

Some log is added to the drive, and as the video is switched, the assigned address is somewhat larger until the final allocation fails. But a careful study found that allocation and release were paired, and that the memory usage information recorded in the drive was completely correct, and there was no indication that a memory leak had occurred.

2. Memory fragmentation

Officials have the same problem with the login, but there is no clear conclusion that the official should be not reproduced. Some people point out that the problem is that the continuous memory block that drives the request is larger, but the system has no corresponding memory to allocate, that is, the memory fragment. At the same time, there are two options: cache each request of memory, allocated, no longer returned to the system, this scenario I tested, found that will cause other video drivers have the same problem, not available; another scenario is that the memory fragmentation is due to the cache mechanism of the file system, to solve this problem, The memory consumed by the cache needs to be cleared as follows:

Echo 3 >/proc/sys/vm/drop_caches

After the test, sure enough, tested the hands of the test cases thousands of times, the test software, the results of the tragedy, the development department test cases on several occasions to detect the same problem, the program acclimatized.

3. Investigate Linux Memory usage

Starting from the memory fragmentation approach, we analyze what causes the drive DMA space to be fragmented. Some data found that the memory partition of Linux is divided into 3 pieces: Dma,low Memory and High Memory. The general 32-bit PC configuration is DMA zone located in 0x00010000-0x00ffffff, about 16mb,low Memory is 0x01000000-0x37bfdfff, about 800 MB, and then up the space to high Memory, This part of the access to jump through, performance is not as good as DMA zone and low Memory zone fast. And the strategy of memory allocation is from the top down, once the high memory space is insufficient, then to the low address space request memory. In our products, the total memory size is 512MB, some drive reserved 128MB,DMA zone configuration is 184MB, then the actual remaining low Memory zone only 200MB, and this 200MB to run the UI, media management, interconnection and so on a series of processes. Once the low Memory zone memory is not available, the system will request memory from the DMA zone, resulting in the memory fragmentation of the DMA space. And then, the free command is not able to see the correct memory information, because there is room to see, but one is occupied by the cache, the second is the total number of calculations, rather than the real continuous memory, then view the memory should use the system provided buddy info to see the real remaining memory block:

$cat/proc/buddyinfo

Node 0, Zone DMA 3 1 3 4 1 2 2 0 1 2 0 3 2 2

Node 0, Zone Normal 74 36 29 8 6 4 3 2 1 2 3 2 0 2

The output of the data from the left is the number of 4K memory block, followed by 2 to the rightmost 32M; In the case of problems, the system basically only small block of memory, and the problem is that some file access operations caused DMA zone space is used by the cache.

4. Control memory allocation algorithm

After you find the cause of the problem, investigate how you can prevent the system from allocating memory from the DMA zone. Find the lowmem_reserve_ratio that can be set to tell the system, DMA zone reserved space. Its parameters are 3 (system default: 256,32,32), and DMA zone related to the first, the value is divisor, with the DMA zone size divided by 256, the basic is the size of the reservation. After doing a few tests, found that only set to 1 to effectively protect the DMA zone is not occupied,:(! When set to 1, the video no longer has a black screen, but other problems quickly occur because the system has less memory available, the memory recovery algorithm is frequently executed, and performance drops dramatically.

5. Final programme

Because the total memory of the product is too small, and the DMA zone is too large, you need to adjust the DMA zone reserved space, but Lowmem_reserve_ratio has no adjustment space, so consider from the adjustment kernel up and down hand. Finally, the total size of the DMA zone in the kernel is adjusted from 184MB to the original 3/4, that is 138MB, get an approximate reasonable configuration, ensure the DMA and low memory can apply to enough space, after a long time testing, does not reproduce the related problems.