See how Cgroup controls CPU, memory, IO

Cgroup Introduction

Cgroup is the abbreviation for control group, which provides a mechanism for the task aggregation and partitioning of the Linux kernel, which organizes some tasks into one or more subsystems through a set of parameter sets.

CGROUPS is the acronym for Control groups, originally presented by Google engineers and later compiled into the Linux kernel.

Cgroups is the underlying foundation of the Resource Management Control section for IaaS virtualization (KVM, LXC, etc.), PAAs container sandbox (docker, etc.)

subsystem is a group that divides tasks into a specified attribute according to the Cgroup function, which is mainly used to realize the control of resources. In Cgroup, grouped task groups are organized in the form of hierarchies, where multiple subsystems form a structure similar to multiple trees in a data structure. Cgroup contains a number of orphaned subsystems, each subsystem represents a single resource, currently, Redhat supports 10 subsystems by default, but only 8 subsystems are mounted by default, Ubuntu 12.04 supports 8 subsystems by default, but only 5 subsystems are mounted by default.

Cgroup Subsystem Introduction

Of course, the user can customize the subsystem and mount it.

The following is a brief introduction to each subsystem:

1. The Blkio setting restricts the input and output control of each block device. For example: disks, CDs, USB, and so on.

2. The CPU uses the scheduler to provide CPU access to the Cgroup task.

3. CPUACCT generates CPU resource reports for cgroup tasks.

4. Cpuset if it is a multi-core CPU, this subsystem allocates a separate CPU and memory for the Cgroup task.

5. Devices allows or denies access to the device for the Cgroup task.

6. Freezer pause and Resume cgroup tasks.

7. Memory sets each cgroup and generates memory resource reports.

8. NET_CLS tags each network packet for Cgroup convenience.

9. NS namespace subsystem.

10. Perf_event adds the ability to monitor and track each group, which can monitor all threads belonging to a particular group and threads running on a specific CPU, which is useful for monitoring the entire group, as described in Http://lwn.net/Art icles/421574/

Libcgroup Tool Installation

Take CentOS 6.4 as an example

Yum Install Libcgroup

Service Cgconfig Start #开启cgroups服务

Chkconfig Cgconfig on #开机启动

[root@localhost /]# ls /cgroup/blkio  cpu  cpuacct  cpuset  devices  freezer  memory  net_cls

When Cgroup starts, it reads the contents of the configuration file/etc/cgconfig.conf, creating and mounting the specified Cgroup subsystem based on its contents.

Cgroup Configuration File Analysis

/etc/cgconfig.conf is the Cgroup Configuration tool Libcgroup used to define the Cgroup group, the parameter settings, and the configuration file for the mount point definition.

Mainly consists of Mount and group two sections.

(1) The syntax format for Mount section is as follows:

mount {    <controller> = <path>;    ...}########################################## controller：内核子系统的名称# path：该子系统的挂载点#########################################举个例子：mount {    cpuset = /cgroup/red;}上面定义相当于如下shell指令：mkdir /cgroup/redmount -t cgroup -o cpuset red /cgroup/red

(2) The syntax format for Group section is as follows:

Group <name> {[<permissions>] <controller> {<param name> = <param value>;    ... } ...}################################################################################# # Name: Specifies the name of the Cgroup# # Permissions: Optional, specify the permissions of the cgroup corresponding mount point file system, root user has all permissions.# # Controller: The name of the subsystem# # param name and param value: Subsystem properties and their property values################################################################################ #举个例子: Mount {# # Defines the Cgroup subsystem that needs to be created and its mount point, where the CPU is created with CPUACCT (statistics) two cgroup subsystem CPU =/mnt/cgroups/cpu;cpuacct =/mnt/CGROUPS/CPU;} Group Daemons/www {# # defines the DAEMONS/WWW (Web server process) group Perm {# # Defines permissions for this group task {uid = Root;gid = webmaster;                        } admin {uid = root;                gid = root; }} CPU {# # defines the properties and values of the CPU subsystem, which is the weight of the task belonging to the phrase Cpu.shares =1000; }}group Daemons/ftp {# # Definition Daemons/ftp (FTP process) group Perm {task {uid = Root;gid = Ftpmaster;        }admin {uid = Root;gid = root;}} CPU {# # The task that defines the phrase has a weight of cpu.shares =500; }} The above configuration file definition is equivalent to executing the following shell command:Mkdir/mnt/cgroups/cpumount-t Cgroup-o Cpu,cpuacct cpu/mnt/cgroups/cpumkdir /mnt/cgroups/cpu/daemonsmkdir /mnt/ Cgroups/cpu/daemons/wwwchown root:root /mnt/cgroups/cpu/daemons/www/*< Span class= "Hljs-keyword" >chown root:webmaster /mnt/cgroups/cpu/daemons/www/tasksecho 1000 > /mnt/cgroups/cpu/ Daemons/www/cpu.sharesmkdir /mnt/cgroups/cpu/daemons /ftpchown root:root /mnt/cgroups/cpu/daemons/ftp/* chown root:ftpmaster /mnt/cgroups/cpu/daemons/ftp/tasksecho 500 > /mnt/cgroups/cpu/ Daemons/ftp/cpu.shares         

For virtual machine VMS, application sandbox, cgroups technology selection comparison

Cgroups Resource Control cgroups management process CPU resources

Run a CPU-consuming script

X=0

While [True];d o

x= $x +1

Done

Top can see that this script basically accounts for 100% of CPU resources

PID USER PR NI VIRT RES SHR S%cpu%MEM time+ COMMAND

30142 Root 0 104m 2520 R 99.7 0.1 14:38.97 sh

The following uses Cgroups to control the CPU resources of this process

Mkdir-p/cgroup/cpu/foo/#新建一个控制组foo

echo 50000 >/cgroup/cpu/foo/cpu.cfs_quota_us #将cpu. Cfs_quota_us is set to 50000, compared to Cpu.cfs_period_us 100000 is 50%

echo 30142 >/cgroup/cpu/foo/tasks

Then top real-time statistics are as follows, CPU occupancy is nearly 50%, it seems cgroups about the control of the CPU has played an effect

PID USER PR NI VIRT RES SHR S%cpu%MEM time+ COMMAND 30142 Root 0 105m 2884 R 49.4 0.2 23:32.53 sh

There are other controls under the CPU control group Foo, and you can do more control over the CPU.

[Email protected] ~]# ls/cgroup/cpu/foo/

Cgroup.event_control cgroup.procs cpu.cfs_period_us cpu.cfs_quota_us cpu.rt_period_us cpu.rt_runtime_us cpu.shares Cpu.stat notify_on_release Tasks

Cgroups Managing Process Memory resources

Run a memory-consuming script that keeps growing memory

X= "a"

While [True];d o

x= $x $x

Done

Top see a steady rise in memory footprint

PID USER PR NI VIRT RES SHR S%cpu%MEM time+ COMMAND 30215 Root 0 871m 501m 1036 R 99.8 26.7 0:38.69 sh

30215 Root 0 1639m 721m 1036 R 98.7 38.4 1:03.99 sh

30215 Root 0 1639m 929m 1036 R 98.6 49.5 1:13.73 sh

The following uses Cgroups to control the memory resources of this process

Mkdir-p/cgroup/memory/foo

echo 1048576 >/cgroup/memory/foo/memory.limit_in_bytes #分配1MB的内存给这个控制组

echo 30215 >/cgroup/memory/foo/tasks

found that the previous script was killed.

[Email protected] ~]# sh/home/memory.sh

has been killed

Because this is a tough limit on memory, when the process tries to consume more memory than the cgroups limit, it triggers an out of memory, causing the process to be killed.

In fact, the memory usage of the process will have an estimate, then will give the process of the limit of 50% for example, unless a memory leak and other anomalies, will be killed because of the cgroups limit.

The cgroups Oom kill process can also be configured to be switched off by Memory.oom_control (oom_kill_disable 1), but even though the process is not killed directly, the process is in hibernation, unable to continue execution, and is still unavailable for service.

With regard to memory control, there are the following configuration files, the control of virtual memory, and the memory control of weighted weights, etc.

[Email protected]/]# ls/cgroup/memory/foo/

Cgroup.event_control Memory.force_empty memory.memsw.failcnt

Memory.memsw.usage_in_bytes memory.soft_limit_in_bytes memory.usage_in_bytes Tasks

Cgroup.procs memory.limit_in_bytes Memory.memsw.limit_in_bytes

Memory.move_charge_at_immigrate Memory.stat Memory.use_hierarchy

memory.failcnt memory.max_usage_in_bytes Memory.memsw.max_usage_in_bytes

Memory.oom_control memory.swappiness Notify_on_release

Cgroups managing Process IO Resources

Run a script that consumes IO

DD IF=/DEV/SDA Of=/dev/null &

See IO occupancy through iotop, disk speeds up to 284m/s

30252 BE/4 root 284.71 m/s 0.00 B/s 0.00% 0.00% DD IF=/DEV/SDA of=/dev/null

The following uses Cgroups to control the IO resource of this process

Mkdir-p/cgroup/blkio/foo

Echo ' 8:0 1048576 ' >/cgroup/blkio/foo/blkio.throttle.read_bps_device

#8:0 corresponds to the main device number and the secondary device number, which can be viewed through ls-l/DEV/SDA

echo 30252 >/cgroup/blkio/foo/tasks

And then through Iotop, the reading speed is actually reduced to 1m/s.

30252 BE/4 root 993.36 k/s 0.00 b/s 0.00% 0.00% DD IF=/DEV/SDA of=/dev/null

There are many other control planes and ways to Io, as follows

[Email protected] ~]# ls/cgroup/blkio/foo/

Blkio.io_merged blkio.io_serviced Blkio.reset_stats

blkio.throttle.io_serviced Blkio.throttle.write_bps_device blkio.weight Cgroup.procs

blkio.io_queued Blkio.io_service_time blkio.sectors

Blkio.throttle.read_bps_device Blkio.throttle.write_iops_device Blkio.weight_device Notify_on_release

Blkio.io_service_bytes Blkio.io_wait_time Blkio.throttle.io_service_bytes

Blkio.throttle.read_iops_device Blkio.time Cgroup.event_control Tasks

See how Cgroup controls CPU, memory, IO