Use cgroups to limit cpu resources in Linux

This article describes how to use cgroups to restrict cpu resources in Linux to avoid a process occupying all CPU resources. For more information, see

This article describes how to use cgroups to manage cpu resources. Let's talk about the cpu usage of the control process. When running multiple programs on a machine that may consume a large amount of resources, we do not want a program to occupy all the resources, so that other programs cannot run normally, or the system is suspended for maintenance. At this time, cgroups can be used to control the resource usage of the process. Cpu resources are described here.

In cgroups, you can use cpu. cfs_period_us and cpu. cfs_quota_us to limit the cpu time that all processes in this group can use in a unit of time. The cfs is short for the Completely Fair Scheduler. Cpu. cfs_period_us is the time period. The default value is 100000, that is, milliseconds. Cpu. cfs_quota_us indicates the cpu time that can be used during this period. The default value is-1, which means no limit.

Run a cpu-consuming program

Copy codeThe code is as follows:
# Echo 'While True: pass' | python &
[1] 1532

The process occupies 100% of the cpu.

Copy codeThe code is as follows:
Pid user pr ni virt res shr s % CPU % mem time + COMMAND
1532 root 20 0 112 m 3684 1708 R 99.6 0.7. 42 python
...

Then let's make some restrictions on this process. Modify the restrictions of the/foo control group and add the process to the group.

Copy codeThe code is as follows:
Echo 50000>/sys/fs/cgroup/cpu/foo/cpu. cfs_quota_us
Echo 1532>/sys/fs/group/cpu/foo/tasks
As you can see, you only need to write the corresponding file to modify the settings. you only need to write the pid to the tasks file to add the process to the cgroup. Set cpu. cfs_quota_us to 50000, which is 100000 relative to cpu. cfs_period_us. Next, let's take a look at the effect.

Copy codeThe code is as follows:
Pid user pr ni virt res shr s % CPU % mem time + COMMAND
1532 root 20 0 112 m 3684 R 1708 50.2. 31 python
...

We can see that the cpu usage of a process has been successfully limited to 50%. The tested virtual machine has only one core. In multi-core scenarios, the values are different. In addition, cfs_quota_us can be larger than cfs_period_us, which is mainly used for multi-core scenarios. When there are n cores, the process in a control group can naturally use up to n times the cpu time.

These two values are restricted at the cgroups level, and the resources in the lower layer cannot exceed the upper layer. Specifically, the lower-level cpu. cfs_period_us value cannot be smaller than the upper-level value, and the cpu. cfs_quota_us value cannot be greater than the upper-level value.

Another set of cpu. rt_period_us and cpu. rt_runtime_us correspond to the restrictions of real-time processes, which may not be used at ordinary times.

In the cpu subsystem, cpu. stat is the statistics of resource restrictions made using the previous method. Nr_periods and nr_throttled are the total and restricted periods. Throttled_time is the total cpu usage time that is throttled by the control group.

There is also a cpu. shares, which is also used to limit cpu usage. However, it is quite different from cpu. cfs_quota_us and cpu. cfs_period_us. Cpu. shares does not limit the absolute cpu time that a process can use, but controls the quota between groups. For example

Copy codeThe code is as follows:
/Cpu. shares: 1024
/Cpu/foo/cpu. shares: 2048

When the processes in both groups are running at full capacity, the cpu occupied by the processes in/foo is twice that of the processes in. If you create another/foo/bar cpu. shares is also 1024 and there are also processes running at full capacity, the cpu usage ratio of/,/foo and/foo/bar is. As mentioned above, all of them are full. If the processes in other control groups are idle, the processes in a group can use all the CPUs. It can be seen that this method can make full use of resources while ensuring fairness.

In addition, you can specify the cpu cores that a process can use. The cpuset subsystem is the cpu core and memory nodes that can be used by processing processes, and other related configurations. Many configurations in this section are related to NUMA. Cpuset. cpus and cpuset. mems are used to restrict cpu cores and memory nodes that can be used by processes. In the two parameters, the cpu core and memory nodes are represented by IDs and separated by commas. For example, 0, 1, and 2. You can also use "-" to indicate the range, for example, 0-3. The two can be used together. For example, "0-, 7 ". Before adding processes, cpuset. cpus and cpuset. mems must be set at the same time and must be compatible; otherwise, errors may occur. For example

Copy codeThe code is as follows:
# Echo 0>/sys/fs/cgroup/cpuset/foo/cpuset. cpus
# Echo 0>/sys/fs/cgroup/cpuset/foo/cpuset. mems

In this way, processes in/foo can only use cpu0 and memory node 0. Use

Copy codeThe code is as follows:
# Cat/proc/ /Status | grep '_ allowed_list'

Verify the effect.

In addition to resource usage restrictions, cgroups also provides resource statistics. You can use it for cloud computing billing. There is a cpuacct subsystem dedicated for cpu resource statistics. Cpuacct. stat collects statistics on the cpu usage of process user and kernel states in the control group. The unit is USER_HZ, that is, the number of jiffies and cpu ticking. You can use getconf CLK_TCK to obtain the number of tick answers per second, usually 100. Divide the value you see into seconds.

Cpuacct. usage and cpuacct. usage_percpu are the cpu time consumed by processes in the control group, measured in nanoseconds. The latter is counted by cpu.

P.S. 2014-4-22

It is found that in SLES 11 sp2 and sp3, corresponding to kernel versions 3.0.13 and 3.0.76, writing the pid to the cpu subsystem into cgroup. procs does not actually take effect, and it only needs to be written into tasks. In other environments, kernel versions later or earlier are not found.