Docker Container Resource limits

Original address: https://docs.docker.com/engine/admin/resource_constraints/#configure-the-realtime-scheduler
Summary

In Docker, docker cpu usage,the container's resources are limited only by the host-side kernel resource scheduling.     However, you can add some resource control flags to control the container's MEMROY,CPU and block IO resource allocations when you start the container with the Docker Run command. This article will describe the details of the use of these control flags and their significance.
first, Memory memories the harm of 1.1 OomSometimes it is necessary to limit the memory usage of a container so that it consumes the host memory and causes the system memory to be strained. In a host-side Linux system, some critical tasks will no longer function properly if memory resources are consumed in a state of extreme scarcity, which will result in out of memory Exception (Oome). At this point the kernel chooses a process (typically a process that consumes a lot of memory) to kill to free up memory. Any process can be killed, including Docker or other important application processes, and if it is killed, it is likely that important processes in the system will affect the stability of the system.
Docker container cpu
Docker can minimize the risk of the system being killed when oom occurs by adjusting the oom priority of Docker daemon. By default, the container's oom priority is not adjusted, and the kernel tends to "kill" a container when oome occurs, rather than a Docker daemon or an important process in the system. If necessary, we can also pass the parameter--oom-score-adj (with a very small negative value behind) or the--oom-disable-kill parameter
To protect Docker daemon or an important container from being killed by the kernel as it happens when oom occurs.
Docker container cpu usage
For more information about the Linux kernel Oom management, refer to out of Memory Management
Docker cpu
We can also reduce the risk posed by Oome by the following ways: docker check cpu usage
(1) Perform adequate performance testing before the application is officially released to understand its memory consumption;
(2) To ensure that our application consumes a reasonable amount of resources;
(3) Limit the amount of memory used in the container, the following will be described in detail;
(4) Be aware that the Docker host side is configured with swap. Memory swapping can affect performance and speed, but it can play a role in buffering when system memory is tight.
(5) If possible, transform the container into a service process and then use the Service-level limit or node labels to ensure that the application has sufficient memory to run on the host.
1.2 Memory limits for containersThe memory resource limit for a container is either hard limit or soft limit.
For hard limit, the container memory must not exceed the set memory limit, for soft limit, unless the system memory is tight or the memory resource contention incentive and so on, otherwise the container can be unrestricted free application of memory under the condition of the system memory resource is ample.
The option flag for container memory resource limits, once set, can have an impact on the operation of the container.

docker stats cpu usage

The value of these option flags is formatted as positive integers plus units, such as B (bytes), K (kilobytes), M (megabytes), G (gigabytes).

Docker run cpu

Options	Describe
-M,--Memory	Memory limit, format is number plus unit, unit can be b,k,m,g. The minimum is 4M.
--memory-swap	The size of the exchangeable partition memory, see below for details--memory-swap (section)
--memory-swappiness	By default, the host-side kernel swaps the anonymous pages used by the container at a certain scale. You can adjust this scale by setting an integer value between 0~100 for this flag.
--memory-reservation	Memory resource soft limit, the value of which generally does not exceed the value of--memory (can actually be set to more than--memory, but so that the soft limit essence does not play a role). Soft limits generally work when system memory is tight, and it does not guarantee that the container's memory usage does not exceed this limit.
--kernel-memory	The core memory limit for the container, which is 4M minimum. Because core memory cannot be swapped out, containers that are heavily needed for core memory can cause strain on the host side, affecting either the host side or the other container reference "-"-kernel-memory section.
--oom-kill-disable	Use this flag to prevent an oom from being killed by the kernel when it occurs. To use this flag to ensure that the container is not killed by Oom killer You also need to set the-m/"-"-memroy option; If you do not set the-M option, the host side may be at risk of memory exhaustion, at which time the kernel will kill processes in the system to free up memory.
--oom-score-adj	Container was killed by OOM killer priority, range is [-1000, 1000], default is 0

For more details on cgroups and memory refer to document Memroy Resource Controller.
1.3 Memory-swapThe--MEMORY-SWAP flag is only meaningful if the--memory is also set. Use this flag container to swap excess memory to disk after exhausting its available memory resources. Of course, frequent memory swapping can lead to loss of performance.
The setting of this flag will have some of the following effects:
(1) If--memory-swap is set to a positive integer, the--memory flag must also be set. --memory-swap represents the total amount of "memory" + "swap partition";--memory represents the total amount of non-swap memory. For example:--memory= "300M" and--memory-swap= "1g", the memory available for the container is 300M, and the swap partition is 700M;
(2) If--memory-swap is set to 0, this flag is ignored;
(3) If the value of--memory-swap is equal to--memory (--memory is set to positive), its effect is equivalent to--memory-swap set to 0;
(4) If the--MEMORY-SWAP flag is not set, but--memory is set, the container can obtain a swap partition of twice times the--memory. For example, if--memory= "300M" and--memory-swap is not set, the container can actually swap partitions using 300M memory +600m;
(5) If--memory-swap is set to-1, the container's swap partition is not restricted by Docker and is limited only by the available resources of the host system.
1.4--memory-swappiness(1) 0 value means switch off the anonymous page;
(2) Set to 100 to indicate that all anonymous pages are exchangeable;
(3) If this flag is not set, the default value remains the same as the value of host.
1.5--kernel-memoryCore memory differs from user memory, it cannot be swapped to disk, it includes:
*Stack pages
*Slab pages
*Sockets Memory Pressure
*TCP Memory Pressure
--kernel-memory and--memory have a certain connection, it is in the context of--memroy limit core memory, that is, the consumption of core memory into the consumption of--memory.
Take a look at the following scenario:
*Unlimited memory, Unlimited kernel memory: default configuration;
*Unlimited memory, limited kernel memories: This configuration is optional if all cgroup requires more than the total amount of memory in the host system. In this case, you can limit the amount of core memory to not exceed the available memory for host hosts. The container needs to wait when it needs more memory.
*Limited Memories, umlimited kernel Memory: Only the total limit on the RAM, but not the individual limit kernel memory.
*Limited memory, Limited kernel memories: Limit both user and core memory. This is useful for memory-related problem-mode. (1) If the core memory limit is lower than the user memory limit, when the container core memory overrun will cause an oom, (2) if the core memory limit is higher than the user memory limit, it will not cause the container to oom.

Once any memory limit is opened, the host side will monitor the process's "high watermark" value, and we can then track which process (here is actually the container) memory overrun, which can be viewed on the host side via/proc/<pid>/status.

second, CPU resourcesThe container's CPU (time) resource is not limited by default, and we can set this limit manually for the container. Typically, you set the CPU time associated with the CFS schedule (most of the tasks are common tasks, that is, CFS scheduling). In Docker1.13 or later, the CPU time of the real-time scheduling task can be configured.
CPU allocation in 2.1 CFS schedulerCommon processes in Linux use the CFS algorithm for scheduling. The associated flags can be used to configure the CPU resources available to the container, and the container uses these flags to actually manipulate the Cgroup related configuration items on the host side of the container. The following table lists some of the flags:

Options	Description
--cpus=<value>	Represents the CPU resources that are allocated to the container. For example: The host system has 2 CPUs, we specify--cpu= "1.5" for the container, which guarantees that the container can use up to "1.5" CPUs resources. This is equivalent to the setting of--cpu-period= "1000000" plus--cpu-quota= "1500000". This flag is supported in Docker 1.3 or later.
--cpu-period=<value>	Specifies the period of the CFS schedule, which is typically used in conjunction with--cpu-quota. By default, the period is 1 seconds, expressed in microseconds, and the default value is generally used. 1.13 or higher is recommended instead of using the--CPUS flag.
--cpu-quota=<value>	The CPU time quota for a cycle in the CFS scheduler, which is the CPU time (in microseconds) that can be obtained per--cpu-period cycle container, cpu-quota/cpu-period. 1.13 or higher is recommended instead of using the--CPUS flag.
--cpuset-cpus	Specifies which CPU or CPU core the container is running on. The value can be a comma "," separated by each CPU number, or a "-" can specify a CPU range. In a multi-CPU system, the CPU is numbered starting with number No. 0. For example: 0-3 indicates that the container can use CPU0,CPU1,CPU2,CPU3, while 1,3 indicates that only CPU1 and CPU3 are used.
--cpu-shares	To set the weight of the container, the default weight of the process in the CFS schedule is 1024. In the case of limited CPU resources, the weight value will work, if the CPU is sufficient, each container can get enough CPU execution time, this time--cpu-shares equivalent to a soft limit.

Below is a look at the system with only 1 CPUs, the following settings can guarantee that each container can only allocate up to 50% of the CPU time.
(1) Docker 1.13 or later

Docker run-it--cpus= ". 5" Ubuntu/bin/bash

(2) Docker 1.12 or lower

$ docker run-it--cpu-period=100000--cpu-quota=50000 Ubuntu/bin/bash

2.2 Configuration of real-time schedulingIn Docker1.13 or later versions, you can configure the container for real-time scheduling. Before configuring a Docker daemon or container, the first thing to do is to ensure that the host kernel is configured accordingly.
kernel configuration on Hsot sideWe pass Zcat/proc/config.gz | grep config_rt_group_sched to confirm whether the host kernel enables config_rt_group_sched configuration items, or you can check the/sys/fs/cgroup/cpu.rt_runtime_ The US file exists for confirmation.
configuration of DOCKER daemonIf the container requires real-time scheduling (with real tasks), the--CPU-RT-RUNTIME flag is used to specify the maximum CPU time (in microseconds) that a real-time task in a container can occupy in a cycle when launching Docker daemon.
For example, with the default cycle of 1000000 us (1 seconds) in the system, setting--cpu-rt-runtime=950000 guarantees that real-time tasks in the container can get up to 950 milliseconds of CPU time in a single cycle, The remaining 50ms in the cycle can be assigned to non-real-time tasks.
On systems with SYSTEMD, if you want these configurations to take effect permanently, you can see control and configure Docker with SYSTEMD (https://docs.docker.com/engine/admin/systemd/).
configuration of a single container

When you start a container with Docker run, you can use flags to control the CPU resource allocation of the container.

Option	Description
-cap-add=sys_nice	The container has Cap_sys_nice permissions, which allows the container to modify its nice value, set the real-time scheduling policy, set CPU affinity, and so on.
--cpu-rt-runtime=<value>	A maximum CPU time that can be run in a real-time cycle, with the--CAP-ADD=SYS_NICE flag set
--ulimit rtprio=<value>	Container's highest real-time priority, need to set--cap-add=sys_nice flag

Take a look at the following example:

$ docker run--it--cpu-rt-runtime=95000 \
                  --ulimit rtprio=99 \
                  --cap-add=sys_nice \
                  Debian:jessie

The example above shows a container that runs from Debian:jessie with a cap_sys_nice privilege with real-time priority of 99, and a maximum realtime CPU time of 950ms in a period.