Introduction to using cgroups to manage process disk I/O in Linux

This article mainly introduces how to use cgroups to manage disk I/O in Linux. For more information, see

Linux cgroups can also restrict and monitor disk io of processes. This function is implemented through the blkio subsystem.

There are many things in the blkio subsystem. However, most of them are read-only status reports, and only the following are writable parameters:

Copy codeThe code is as follows:
Blkio. throttle. read_bps_device
Blkio. throttle. read_iops_device
Blkio. throttle. write_bps_device
Blkio. throttle. write_iops_device
Blkio. weight
Blkio. weight_device

These are all disk io used to control the process. It is clearly divided into two categories, with throttle. as its name implies, it is a throttle that limits the traffic to a specific value. "Weight" is the distribution of io weights.

The four parameters of "throttle" can be identified by name. Use blkio. throttle. read_bps_device to limit the number of bytes that can be read per second. Run the I/O command first.

Copy codeThe code is as follows:
Dd if =/dev/sda of =/dev/null &
[1] 2750
Use iotop to check the current io

Copy codeThe code is as follows:
Tid prio user disk read disk write swapin io> COMMAND
2750 be/4 root 66.76 M/s 0.00 B/s 0.00% 68.53% dd if =/dev/sda of =/dev/null
...

Modify the resource limit and add the process to the control group.

Copy codeThe code is as follows:
Echo '8: 0 1048576 '>/sys/fs/cgroup/blkio/foo/blkio. throttle. read_bps_device
Echo 2750>/sys/fs/cgroup/blkio/foo/tasks
Here, is the primary device number and secondary device number of the corresponding block device. You can use the ls-l device file name. For example

Copy codeThe code is as follows:
# Ls-l/dev/sda
Brw-rw ----. 1 root disk 8, 0 Oct 24 11: 27/dev/sda
Here, 8 and 0 are the corresponding device numbers. Therefore, cgroups can impose different restrictions on different devices. Then let's take a look at the effect.

Copy codeThe code is as follows:
Tid prio user disk read disk write swapin io> COMMAND
2750 be/4 root 989.17 K/s 0.00 B/s 0.00% 96.22% dd if =/dev/sda of =/dev/null
...

It can be seen that the process's read per second is immediately reduced to about 1 MB. To lift the limit, write "0" to the file.

However, it should be noted that this method is useless for a large amount of io generated less than the sampling interval. For example, even if a peak value of 100 MB written per second is generated within one second, it is not restricted.

Let's take a look at blkio. weight. The throttle and weight methods of blkio are similar to the quota and shares methods of the cpu subsystem. they are both absolute restrictions and relative restrictions, in addition, you can make full use of resources when you are not busy. the weight value ranges from 10 to 1000.

It is a little effort to test the weight method. Because it is not an absolute limit, it will be affected by the file system cache. For example, if you want to test in a virtual machine, you need to disable the VM, such as the cache of VirtualBox that I use on the host machine. If you want to test the read io effect, you can use dd to create two large files:/tmp/file_1 and/tmp/file_2. Then set two weights.

Copy codeThe code is as follows:
# Echo 500>/sys/fs/cgroup/blkio/foo/blkio. weight
# Echo 100>/sys/fs/cgroup/blkio/bar/blkio. weight
Clear the file system cache before testing to avoid interfering with the test results.

Copy codeThe code is as follows:
Sync
Echo 3>/proc/sys/vm/drop_caches
In these two control groups, dd is used to produce the io test results.

Copy codeThe code is as follows:
# Cgexec-g "blkio: foo" dd if =/tmp/file_1 of =/dev/null &
[1] 1838
# Cgexec-g "blkio: bar" dd if =/tmp/file_2 of =/dev/null &
[2] 1839

Use iotop to check the effect

Copy codeThe code is as follows:
Tid prio user disk read disk write swapin io> COMMAND
1839 be/4 root 48.14 M/s 0.00 B/s 0.00% 99.21% dd if =/tmp/file_2 of =/dev/null
1838 be/4 root 223.59 M/s 0.00 B/s 0.00% 16.44% dd if =/tmp/file_1 of =/dev/null

Although the number of bytes read by the two processes per second is constantly changing, the general trend is still around, consistent with the configured weight ratio. Blkio. weight_device is sub-device. Enter the device number before writing.

The blkio subsystem also has many statistical items.

Blkio. time
Io access time for each backup, in milliseconds

Blkio. sectors
Number of fan areas for the entrant or for each slave node

Blkio. io_serviced
The number of I/O operations performed in each configuration, including read, write, sync, async, and total.

Blkio. io_service_bytes
Number of input/output nodes for each type of io

Blkio. io_service_time
Io time of each type, in microseconds

Blkio. io_wait_time
The waiting time of various types of io in the queue in each setup and backup

Blkio. io_merged
The number of times each type of io request is merged in each setup/backup

Blkio. io_queued
Number of current io requests in the queue in each setup and backup

Use these statistical items to better count and monitor the io of processes
Use

Copy codeThe code is as follows: echo 1> blkio. reset_stats
All statistics can be cleared.