Location of memory statistics and memory leakage in linux

Memory Statistics and memory leakage problems in linux are located in product development. by measuring the total memory consumption of the current system, you can accurately evaluate the total memory required by the product, in this way, the appropriate memory chip and size are selected to reduce the cost of the product. When memory leakage occurs ,... information & Memory Statistics and memory leakage problems in linux are located in product development. by counting the total memory consumption of the current system, you can accurately assess the total memory size required by the product, so as to select the appropriate memory chip and size to reduce the cost of the product. In case of memory leakage, this problem is often left helpless. This article analyzes the process-related files in proc to accurately evaluate the memory size consumed by the system, it also provides a method to locate memory leakage issues. Linux's memory usage principle is: if the memory is sufficient, do not use the white space, try to use the memory to cache some files, so as to speed up the process, and when the memory is insufficient, the cache memory will be reclaimed through the corresponding memory recovery policy for the process to use. I. Analysis of total system memory. The meminfo file in the proc directory shows the usage of the current system memory. the available physical memory is memfree + buffers + cached. when memfree is insufficient, the kernel will write the cached and buffered memories back to the backup memory through the write-back mechanism (pdflush thread) to release the memory for the process to use, alternatively, you can manually explicitly release the cache memory echo 3>/proc/sys/vm/drop_caches is the overall usage of the current system memory on the Haisi platform, the system has consumed 29 MB of memory. next we will continue to analyze who has consumed the memory. # Cat/proc/meminfoMemTotal: 68956 kBMemFree: 18632 kBBuffers: 4096 kBCached: 17260 bytes: 0 kBActive: 21304 kBInactive: 19248 kBSwapTotal: 0 kBSwapFree: 0 kBDirty: 0 kBWriteback: 0 kBAnonPages: 19216 kBMapped: 2472 kBSlab: 6900 kBSReclaimable: 924 kBSUnreclaim: 5976 kBPageTables: 460 kBNFS_Unstable: 0 kBBounce: 0 kBCommitLimit: 62060 bytes: 28864 kBVmallocTotal: 442368 kBVmallocUs Ed: 46984 kBVmallocChunk: 393212 kB 2. process memory usage statistics in 32-bit operating systems, each process has 4 GB of virtual memory space, of which 0 ~ 3 GB is the private user space of each process, which is invisible to other processes in the system. 3 ~ 4 GB is the Linux kernel space, shared by all the processes and kernels of the system. By accessing the relevant files in/proc/{pid}/, you can understand the usage of virtual memory space of each thread and how much memory is consumed by each thread. Because our products are implemented in multi-thread mode, multiple threads share the user-State virtual address space of a process. The virtual address space contains several areas, mainly including the following areas: 1. the code segment of the current execution file, which is called the text segment. 2. the data segment of the execution file, which mainly stores global variables and static variables used by the execution file. 3. a heap that stores global variables and dynamically generated data. 4. stack for saving local variables and implementing function calls. 5. The mmap method is used to map the memory segments in the virtual address space. Therefore, you only need to view the user-State virtual address space allocation of any thread to know the total memory occupied by all threads in the same process.. You can view the/proc/{pid}/maps file to obtain the relevant virtual address space content. some typical contents in the Digest column are as follows: # cat/proc/568/maps00008000-0036a000 r-xp 00000000 00: 0e 236/home/hik/hicore00372000-003a5000 rw-p 00362000 00: 0e 236/home/hik/hicore003a5000-00e28000 rwxp 003a5000 00:00 0 [heap] 40000000-40005000 r-xp 00000000 094/lib/ld-uClibc.so.0416db000-41770000 rw-s c2005000 00: 0f 68/dev/mem b51fc000-b5200000 rwxp b51fc000 00:00 0 ....... Be1fc000-be200000 rwxp be1fc000 00:00 0be93b000-be950000 rwxp befeb000 00:00 0 [stack] line 1: r-xp shows that its permissions are read-only and executable. the memory address of this segment corresponds to the code segment of the execution file, the code segment of the program must be loaded into the memory for execution. Because it is read-only and will not be modified, it is shared throughout the system. Row 2: rw-p shows that the permission is read/write and cannot be executed. the memory address of this segment corresponds to the data segment of the execution file and stores the global and static variables used by the execution file. Row 3: From rwxp, we can see that the permission is read/write, executable, the address space increases upwards, and does not correspond to the file, it is a heap segment, and the process uses the memory applied for by malloc to be placed in the heap segment. Each process has only one heap segment. the memory applied by the main process or different threads is reflected in the heap segment of the process. The heap segment increases up to 1 GB, that is, 0x40000000. if it is greater than 1 GB, glibc uses mmap to apply for a memory for the heap. Row 4: the memory address of the shared library connected by the program. Fifth line: virtual address space mapped in mmap mode. Row 6 and row 7: The stack address segment of the thread. the stack size of each thread is 16 kB. Row 8: The Stack area of the process. Each thread has one stack segment. if there are multiple threads in the process, it contains multiple stack segments. III. statistics on the total memory of the current system 1. the total memory occupied by processes can be calculated using the maps table above. 2. when the system runs, the application-layer files will be mounted to the tmpfs file system, which may be around 13 MB in the Haisi system, this part of memory is measured in cache mode, but this part of memory cache cannot be released through the recycle policy or explicit call. 3. memory occupied by ramdisk of the root file system. 4. the size of the memory retained by the system. you can view/proc/sys/vm/min_free_kbytes to obtain or modify the memory size. 5. of course, when the system is running, some memory should be reserved for allocating skb during hard disk read/write cache or when the network load is high. generally, it must be over 30 MB. 4. inspiration for debugging memory leakage issues when a process applies for memory, the memory manager built in glibc actually receives the request. as the process requests for memory increase, the memory manager will fall into the kernel through system calls to allocate more memory to the process. For heap segment management, the kernel provides two systems to call brk and mmap. brk is used to change the heap top address, while mmap allocates a virtual address space for the process. When a process requests memory from glibc, if the requested memory quantity is greater than a threshold value, glibc uses mmap to allocate a virtual address space for the process, instead of using brk to extend the heap top pointer. By default, this threshold value is 128 KB, which can be modified using a function. # Include Int mallopt (int param, int value) Param values are M_MMAP_THRESHOLD and M_MMAP_MAX respectively. The Value is in bytes. M_MMAP_THRESHOLD is the memory application for applying for a large memory threshold value in glibc. if the memory application is greater than the threshold value, the memory manager will use the mmap system to call the request for memory. if the application is less than the threshold value, the memory manager uses the brk system to call the extended heap top pointer. M_MMAP_MAX indicates the maximum number of CIDR blocks allocated by mmap in the process. If you suspect a memory leak occurs during the debugging process, you can view the maps table of the process to check whether the virtual address space of the heap or mmap segment of the process continues to increase, if yes, memory leakage is likely to occur. if the virtual address space of the mmap segment continues to increase, you can also see the size of the virtual address space of each segment, therefore, you can determine the size of the memory you have applied, which can be used to locate debugging memory leakage issues.