Introduction to Linux Cgroups

Cgroups (Control Groups) is a mechanism provided by the Linux kernel that integrates (or separates) a series of system tasks and their subtasks into different groups of resources hierarchically based on demand, thus providing a unified framework for system resource management . Simply put, cgroups can restrict and document the physical resources used by the task group. Essentially, Cgroups is a series of hooks that the kernel attaches to the program, which triggers the corresponding hooks for resource tracking and throttling purposes when the program runs.

This article introduces the Ubuntu 16.04 system as an example of cgroups, all of which are demonstrated in this system.

Why do you want to know cgroups

It is necessary to understand cgroups technology in the era of virtualization technology represented by container technology. For example, we can easily limit the CPU, memory and other resources that a container can use, how is this implemented? By understanding cgroups Technology, we can explore the context of the entire resource-constrained system in a Linux system. This helps us to better understand and use Linux systems.

the main role of Cgroups

The main purpose of implementing cgroups is to provide a unified interface for resource management at different user levels. From resource control of a single task to virtualization at the operating system level, Cgroups offers four functions:

• Resource limitations: Cgroups can limit the total amount of resources that a task is going to have. For example, set the memory limit used by the task to run, and when it goes out, OOM.
• Priority allocation: By allocating the amount of CPU time slices and disk IO bandwidth, it is actually the same as controlling the priority of the task's operation.
• Resource statistics: Cgoups can count the amount of resources used by the system, such as CPU usage, memory usage, etc. This feature is ideal for current cloud products that are billed on a per-use basis.
• Task control: Cgroups can perform operations such as suspending, resuming, and so on.

Related Concepts

Task (Tasks) in a Linux system, the kernel itself is scheduled and managed not to differentiate between processes and threads, but to conceptually differentiate between processes and threads, depending on the parameters passed in at clone. Here, a task is used to represent a process or thread of the system.

The resource control in Cgroup (control group) Cgroups is implemented in Cgroup units. Cgroup represents a task group divided by a resource control standard that contains one or more subsystems. A task can be joined to a cgroup, or it can be migrated from one cgroup to another cgroup.

the subsystem in Subsystem (subsystem) cgroups is a resource scheduling controller (also called controllers). For example, the CPU subsystem can control the CPU's time allocation, and the memory subsystem can limit the amount of memory used. For example, I use Ubuntu 16.04.3, the kernel version is 4.10.0, the supported subsystem are as follows (cat/proc/cgroups):
Blkio limits the IO of the block device.
The CPU limits the allocation of CPU time slices, which are mounted in the same directory as CPUACCT.
Cpuacct generates a report that consumes CPU resources in Cgroup, and is mounted in the same directory as the CPU.
Cpuset assigns independent CPUs (multiprocessor systems) and memory nodes to the tasks in Cgroup.
devices allows or disables tasks in the Cgroup to access the device.
Freezer suspend/Resume tasks in the Cgroup.
hugetlb Limit the number of memory pages used.
Memory limits the amount of RAM available for tasks in Cgroup and automatically generates resource consumption reports.
net_cls uses the level identifier (CLASSID) to mark network packets, which allows the Linux traffic controller (TC Directive) to identify packets from a specific Cgroup task and make network restrictions.
Net_prio allows you to set the priority of network traffic (Netowork traffic) based on Cgroup.
perf_event allows the use of perf tools to monitor cgroup.
PIDs Limit the number of tasks.

The Hierarchy (hierarchy) hierarchy has a series of cgroup arranged in a tree-like structure, each of which is controlled by the corresponding subsystem of the binding. The Cgroup node in the hierarchy can contain 0 or more child nodes, which inherit the subsystem that the parent node mounts. There can be multiple tiers in an operating system.

cgroups File System interface

Cgroups provides the application interface as a file, we can view the cgroups default mount point via the Mount command:

Mount grep Cgroup

The first line of TMPFS indicates that the files in the/sys/fs/cgroup directory are temporary files that exist in memory.
The second line of the mount point/sys/fs/cgroup/systemd for the SYSTEMD system to cgroups support, related content The author will do a special introduction in the future.
The remaining mount points are the root-level hierarchies of each subsystem supported by the kernel.

It is important to note that in operating systems that use the SYSTEMD system, the/sys/fs/cgroup directory is mounted by the SYSTEMD during system startup and is mounted as a read-only type. In other words, the system does not recommend that we create a new directory under the/sys/fs/cgroup directory and mount other subsystems. This is not the same as the previous operating system.

Let's explore what's underneath the/sys/fs/cgroup directory and its subdirectories:

The/sys/fs/cgroup directory is the root directory of each subsystem. Let's take the memory subsystem for example and see what's in the memory directory.

These files are the root-level settings in the Cgroups memory subsystem. For example, the numbers in memory.limit_in_bytes are used to limit the maximum available memory for a process, and the weights used in memory.swappiness are preserved.

Since cgroups is using these files as an API, I can apply cgroups by creating or modifying the contents of these files. How do you do it specifically? For example, how can we limit the resources that a process can use? Next we'll show you how to use cgroups to limit the resources that a process can use by using a simple demo.

to view the cgroups that the process belongs to

You can use/proc/[pid]/cgroup to see which cgroup the specified process belongs to:

Each row contains three columns separated by a colon, and their meanings are:

• The ID of the Cgroup tree corresponds to ID one by one in the/proc/cgroups file.
• All subsystem that are bound to the Cgroup tree are separated by commas with multiple subsystem. Here name=systemd means no binding with any subsystem, just gave him a name called Systemd.
• The path of the process in the Cgroup tree, which is the cgroup that the process belongs to, and the path is relative to the mount point.

Since cgroups is using these files as an API, I can apply cgroups by creating or modifying the contents of these files. How do you do it specifically? For example, how can we limit the resources that a process can use? Next we'll show you how to use cgroups to limit the resources that a process can use by using a simple demo.

cgroups Tools

Before we introduce the application of cgroups through SYSTEMD, we use the cgexec in the Cgroup-bin Toolkit to demonstrate the demo. Ubuntu does not have the Cgroup-bin kit installed by default, please install it using the following command:

sudo Install Cgroup-bin

Demo: Limit the CPU available to the process

When we use cgroups, it's best not to directly modify their configuration files directly under the root directory of each subsystem. The recommended approach is to define different nodes in the subsystem tree for different requirements. For example, we can create a new directory named Nick_cpu in the/SYS/FS/CGROUP/CPU directory:

$ cd  /sys/fs/cgroup/sudomkdir nick_cpu

Then look at the contents of the new directory:

Is it a bit surprising that the Cgroups file system automatically creates these profiles when the file directory is created!

Let's limit the CPU cycle to one-tenth of the total through the following settings:

sudo su  echo100000 > nick_cpu/echo10000 > Nick_cpu/cpu.cfs_ Quota_us

Does the above two parameters look familiar? Yes, the "--cpu-period=100000--cpu-quota=200000" described in the article "Docker: Restricting the CPU available to the container" is implemented by them.

Then create a CPU-intensive program:

void Main ()
{    int  i, end;     1024x768 1024x768 1024x768 ;      for 0; I < end;)    {        ++;    }}

Save as file cputime.c compile and execute in different ways:

gcc cputime.c-sudosu time./ time Cgexec-g Cpu:nick_cpu./cputime

The time command can tell us how long it takes for the program to execute, and real is the time we actually feel. Using Cgexec, we can apply the Cgroup configuration nick_cpu we add to the process that runs the CPUTime program. Display, the default execution only needs about 2s. You need to run 23s after limiting CPU resources through cgroups.

Demo: Limit the memory available to the process

This time we will limit the maximum memory available to the process and create a directory nick_memory under/sys/fs/cgroup/memory:

$ cd  /sys/fs/cgroup/sudomkdir nick_memory

The following settings limit the free memory of the process to 300M maximum and do not use swap:

 1024x768*1024x768*314572800sudosu    echo 314572800 > nick_memory/echo 0 >  Nick_memory/memory.swappiness

Then create a program that constantly allocates memory, allocating memory five times, applying 100M at a time:

#include <stdio.h>#include<stdlib.h>#include<string.h>#defineChunk_size 1024 * 1024 * 100voidMain () {Char*p; inti;  for(i =0; I <5; i + +) {p=malloc(sizeof(Char) *chunk_size); if(p = =NULL) {printf ("fail to malloc!"); return ; }        //The memset () function is used to set the first n bytes of a specified memory to a specific valueMemset (P,0, chunk_size); printf ("malloc Memory%d mb\n", (i +1) * -); }}

Save the above code as a mem.c file, and then compile:

gcc mem.c-o mem

Execute the generated MEM program:

$./mem

At this point all goes well, then add the constraint just to try it out:

$ cgexec-g memory:nick_memory./mem

Because of insufficient memory and the use of swap is forbidden, the resource-constrained process is forced to kill when it requests memory.

Then use the stress program to test a similar scenario (Request 500M of memory via the stress program):

sudo 1 500000000 --vm-keep--verbose

The stress program can provide more detailed information, and the process is killed by receiving a SIGKILL (signal 9) signal.

In practice, it is often needed to limit the resources of both CPU and memory resources. The use of cgexec to implement such a use case is actually very simple, directly specify multiple-G options can be:

$ cgexec-g cpu:nick_cpu-g memory:nick_memory./cpumem

Summary

Cgroups is a feature provided by the Linux kernel and is not easy to understand because of the many concepts involved. This article attempts to demonstrate the use of cgroups with the simplest demo while introducing conceptual content. I hope the intuitive demo can help you understand cgroups.

Reference:

About the Linux Control Group