Docker container Memory monitoring principle and application _docker

Docker Container Memory Monitoring

Linux Memory Monitoring

To understand how the Docker container memory is computed, first understand the concepts associated with memory in Linux.

Use the free command to view current memory usage.

[Root@localhost ~]$ free 
       total    used    free   shared  buffers   cached
Mem:   264420684 213853512  50567172  71822688  2095364 175733516-/+ buffers/cache
:  36024632 228396052
Swap:   16777212  1277964  15499248

Here are a few concepts:

1. MEM: Physical Memory
2. Swap: virtual memory. Data that can be stored on the hard disk
3. Shared: Share memory. exists in physical memory.
4. Buffers: For storing data to be output to disk (block device).
5. Cached: Storing data read from disk
   

can refer to here.

For the convenience of the explanation, I made a corresponding to the result of free.

[Root@localhost ~]$ free 
       total    used    free    shared  buffers  cached
Mem:   total_mem  used_mem  free_mem  shared_mem  buffer   cache
-/+ buffers/cache:real_used  Real_ Free
Swap:  total_swap used_swap  Free_swap

name	Description
Total_mem	Total Physical Memory
Used_mem	Amount of physical memory used
Free_mem	Amount of free physical memory
Shared_mem	Shared memory
Buffer	The amount of memory in the buffer
Cache	The amount of memory that the cache occupies
Real_used	Amount of memory actually used
Real_free	Amount of actual idle memory
Total_swap	Total Swap
Used_swap	Swap already in use
Free_swap	Free Swap

It is generally believed that buffer and cache can be used again to use the memory, so in the calculation of free memory, it will be removed.
So here are a few equations:

real_used = Used_mem-buffer-cache
real_free = free_mem + buffer + cache
Total_mem = Used_mem + free_mem

With this in view, let's look at the data source for free. In fact, its data source is from the/proc/memeinfo file.

[root@localhost ~]$ cat/proc/meminfo memtotal:264420684 kb memfree:50566436 KB buffers:2095356 KB Cached: 175732644 KB swapcached:123688 KB active:165515340 KB inactive:37004224 KB Active (anon): 92066880 KB Ina Ctive (anon): 4455076 KB Active (file): 73448460 KB Inactive (file): 32549148 KB unevictable:362900 KB mlocked:7469 
6 KB swaptotal:16777212 kb swapfree:15499248 KB dirty:2860 KB writeback:0 KB anonpages:24932928 kb mapped:58165040 KB shmem:71822688 KB slab:8374496 KB sreclaimable:8163096 KB sunreclaim:211400 KB K ernelstack:45824 KB pagetables:606296 KB nfs_unstable:0 KB bounce:0 KB writebacktmp:0 KB commitl imit:148987552 KB committed_as:114755628 KB vmalloctotal:34359738367 KB vmallocused:772092 KB vmallocchunk:3422     5428328 KB hardwarecorrupted:0 KB anonhugepages:22083584 KB hugepages_total:0 hugepages_free:0 HUGEPAGES_RSVD:   0 Hugepages_surp: 0 hugepagesize:2048 kb directmap4k:7168 KB directmap2m:2015232 KB directmap1g:266338304 kb
 

Docker

After talking about the memory of Linux, let's look at the memory monitoring of Docker. Docker itself provides a way to monitor the memory of the container through Docker stats.

The method is actually calculated by the value of the relevant data in the Cgroup.

Cgroup

The memory subsystem in Cgroup provides the following files for hierarchy.

[root@localhost ~]$ ll/cgroup/memory/docker/53a11f13c08099dd6d21030dd2ddade54d5cdd7ae7e9e68f5ba055ad28498b6f/Total dosage  0--w--w--w-1 Root 0 February 12:51 cgroup.event_control-rw-r--r--1 root root 0 May 17:07 cgroup.procs-rw-r--r--1 Root 0 February 12:51 memory.failcnt--w-------1 root root 0 February 12:51 memory.force_empty-rw-r--r--1 root root 0 3 Monthly 17:06 memory.limit_in_bytes-rw-r--r--1 root 0 February 12:51 memory.max_usage_in_bytes-rw-r--r--1 root root 0 2 Monthly 12:51 memory.memsw.failcnt-rw-r--r--1 root 0 March 17:06 memory.memsw.limit_in_bytes-rw-r--r--1 root 0 February 12:51 memory.memsw.max_usage_in_bytes-r--r--r--1 root 0 February 12:51 memory.memsw.usage_in_bytes-rw-r--r--1  Root 0 February 12:51 memory.move_charge_at_immigrate-rw-r--r--1 root root 0 February 12:51 memory.oom_control-rw-r--r-- 1 root 0 March 17:06 memory.soft_limit_in_bytes-r--r--r--1 root root 0 February 12:51 memory.stat-rw-r--r--1 root R Oot 0 February 12:51 Memory.swappiness-r--r--r--1 root 0 February 12:51 memory.usage_in_bytes-rw-r--r--1 root root 0 February 12:51 memory.use_hierarchy
 -rw-r--r--1 root 0 February 12:51 notify_on_release-rw-r--r--1 root root 0 February 12:51 tasks

The specific meaning of these files can be viewed in the relevant information cgroup memory.

This article mainly introduces several related to the Docker monitoring.

file name	Description
Memory.usage_in_bytes	Amount of memory used (including cache and buffer) (bytes), equivalent to Linux Used_meme
Memory.limit_in_bytes	Limited total memory (bytes), equivalent to Linux Total_mem
Memory.failcnt	Number of application memory failures count
Memory.memsw.usage_in_bytes	Used memory and swap (bytes)
Memory.memsw.limit_in_bytes	Limited memory and Swap capacity (bytes)
Memory.memsw.failcnt	Count of application memory and swap failures
Memory.stat	Memory-related state

The following is a sample of a container.

 [root@localhost 53a11f13c08099dd6d21030dd2ddade54d5cdd7ae7e9e68f5ba055ad28498b6f] $ cat Memory.usage_in_bytes 135021858816 [root@localhost 53a11f13c08099dd6d21030dd2ddade54d5cdd7ae7e9e68f5ba055ad28498b6f]$ Cat Memory.memsw.usage_in_bytes 135679291392 [ Root@localhost 53a11f13c08099dd6d21030dd2ddade54d5cdd7ae7e9e68f5ba055ad28498b6f]$ Cat Memory.stat Cache 134325506048 RSS 695980032 mapped_file 16155119616 pgpgin 21654116032 pgpgout 21705492352 swap 655171584 Inactive_anon 421 8880 active_anon 74202603520 inactive_file 8365199360 active_file 52449439744 unevictable 0 hierarchical_memory_limit 137438953472 hierarchical_memsw_limit 274877906944 total_cache 134325506048 total_rss 695980032 total_mapped_file 16155119616 total_pgpgin 21654116032 total_pgpgout 21705492352 total_swap 655171584 total_inactive_anon 4218880 Total_ Active_anon 74202603520 total_inactive_file 8365199360 total_active_file 52449439744 total_unevictable 0 

Memory.stat

Memory.stat contains the most abundant

Statistic	Description
Cache	Page caching, including TMPFS (shmem), in bytes
Rss	Anonymous and swap caching, excluding TMPFS (shmem), in bytes
Mapped_file	memory-mapped the file size of the map, including TMPFS (shmem), in bytes
Pgpgin	Number of pages stored in memory
Pgpgout	Number of pages read out of memory
Swap	Swap amount in bytes
Active_anon	Anonymous and swap caches in the active recently least used (LEAST-RECENTLY-USED,LRU) list, including TMPFS (shmem), in bytes
Inactive_anon	Anonymous and swap caches in the inactive LRU list, including TMPFS (shmem), in bytes
Active_file	File-backed memory in the active LRU list, in bytes
Inactive_file	File-backed memory in the inactive LRU list, in bytes
Unevictable	Unable to regenerate memory, in bytes
Hierarchical_memory_limit	The memory limit of the level containing the memory Cgroup, in bytes
Hierarchical_memsw_limit	Memory plus swap limit for levels containing memory Cgroup, in bytes

Active_anon + Inactive_anon = Anonymous memory + file cache for Tmpfs + swap cache

Active_file + inactive_file = Cache-size of Tmpfs

Docker Native Memory monitoring

Let's talk about the Docker stats of Docker native. Its concrete realization can be seen in the Libcontainer. It divides the memory monitoring of the container into Cache,usage,swap usage,kernel usage,kernel TCP usage.

The cache is obtained from the cache in the Memory.stat.

Usage uses the memory.usage_in_bytes and memory.limit_in_bytes to divide to calculate the usage rate. This way has a disadvantage, is not enough refinement, did not distinguish the cache part, can not really reflect the memory utilization rate. Because the cache is generally a reusable part of memory, it is generally counted into the available parts.

An improved calculation method that can be considered

The improved method excludes cache calculations when counting memory usage. Similar to the calculation of real_used in Linux will be buffer and cache exclusion.

Cache does not directly apply the cache in Memory.stat because it includes TMPFS, and TMPFS is the actual part of the memory used.

TMPFS is share memory, shared memory

Because there are in the Memory.stat

Active_file + inactive_file = Cache-size of Tmpfs

Therefore, you can calculate the amount of memory that is actually used

real_used = memory.usage_in_bytes-(RSS + active_file + inactive_file)

Thank you for reading, I hope to help you, thank you for your support for this site!