Basic concepts and architecture of Docker, basic concepts and architecture of docker

Basic concepts and architecture of Docker, basic concepts and architecture of docker

Introduction to Docker container Technology

Docker containers depend on two important features of the Linux kernel.

Namespaces namespace Control group

Understanding these two features can help us better understand the resource allocation and management of Docker containers.

1. Namespaces namespace

Many programming languages have the concept of namespaces, which can be considered as an encapsulation concept. Encapsulation actually implements code isolation. In the operating system, namespace provides system resource isolation. System resources include processes, networks, and file systems. In fact, one of the main purposes of implementing a namespace in Linux kernel is to implement a lightweight virtualization Service (container ). Processes in the same namespace can perceive each other's changes. They do not know anything about the processes in other namespaces. In this way, the process in the container can be illusion, as if it is in an independent system environment. In this way, they are independent and isolated from each other.

Docker uses five namespaces

PID (Process ID): provides the Process isolation NET (Network): manages the Network excuse IPC (InterProcess Communication): Manages cross-Process Communication access MNT (Mount ): manage mount points (that is, file System-related management) UTS (Unix Timesharing System): isolate the kernel and version ID

2. Control groups Control group

Isolated resources are managed by the Control group

Cgroups (Control groups) is a kind of physical resources (such as CPU, Memory, IO, etc.) that can be restricted, recorded, and isolated by the Linux kernel) is a Linux kernel function that manages any process in groups. CGroups is also the resource management method used by LXC for virtualization. It was initially proposed by google engineers and integrated into Linux Kernel 2.6.34 @ 2017 in 2007. CGroups is not completely new. It removes Process Management from the cpuset by Google's Paul Menage. CGroup itself provides the process grouping management function and interface basic structure, I/O or memory allocation Control and other specific resource management functions are implemented through this function. These specific resource management functions are called CGroup subsystems or controllers. The CGroup sub-system provides Memory-controlled Memory controllers and CPU controllers that control process scheduling. The Cgroup subsystem that can be used by the running kernel is confirmed by/proc/cgroup.

CGroup provides a CGroup Virtual File System as a user interface for grouping management and subsystem settings. To use CGroup, you must mount the CGroup file system. In this case, the mount option is used to specify which subsystem to use.

Linux uses multiple data structures to configure cgroups in the kernel and associate processes with cgroups nodes. How does Linux Enable User-State processes to use the cgroups function? The Linux kernel has a very powerful module called VFS (Virtual File System ). VFS can hide the details of a specific file system and provide a unified file system API interface for user-State processes. Cgroups also exposes the function to the user State through VFS. The connection between cgroups and VFS is called the cgroups file system.

Cgroups provides the following functions:

Restrict the number of resources that a process group can use ). For example, the memory subsystem can set a memory upper limit for the Process Group. Once the Memory used by the process group reaches the upper limit and the requested memory reaches the upper limit, OOM (Out Of memory) is triggered ). Prioritization ). It can set which process groups use larger CPU or disk IO resources. For example, you can use the CPU subsystem to allocate a specific CPU Share to a process group. Count the number of resources used by the process group (Accounting ). Calculate the number of system resources used by the process group, especially in the billing system. For example, you can use the Cpuacct subsystem to record the cpu time process group Isolation (Isolation) used by a process group ). For example, the Ns subsystem allows different process groups to use different namespaces for isolation. Different process groups have their own process, network, and file system mount spaces. Process Group Control ). It can suspend and resume a process group. For example, the Freezer subsystem can suspend and restore a process group.

The following are common cgroup subsystems:

Subsystem function

The blkio subsystem sets input/output limits for Block devices, such as physical devices (disks, solid state disks, USB, etc ).

The cpu subsystem uses the scheduler to provide access to the CPU cgroup task.

Cpuacct this subsystem automatically generates the CPU report used by the tasks in the cgroup.

Cpuset this subsystem allocates independent CPU (in multi-core systems) and Memory nodes for tasks in the cgroup.

The devices subsystem allows or denies access to devices by tasks in the cgroup.

Freezer: This subsystem suspends or resumes tasks in the cgroup.

The memory subsystem sets memory limits for tasks in the cgroup and automatically generates memory resource reports for those tasks.

The net_cls subsystem uses the classid to mark network packets, allowing the Linux Traffic Control Program (tc) to identify packets generated from a specific Cgroup.

Ns namespace subsystem.

Cgroup concepts

Task ). In Cgroups, a task is a process of the system. Control Group ). A control group is a group of processes divided according to certain standards. Resource Control in Cgroups is implemented in units of control groups. A process can be added to or migrated from one process group to another. Processes in a process group can use Cgroups to control resources allocated by the control group and are restricted by Cgroups. Hierarchy ). A control group can be organized into Hierarchical, which is a control family tree. The child node control group on the control group tree is the child of the parent node control group, inheriting the specific attributes of the parent control group. Subsystem (Subsystem ). A sub-system is a resource controller. For example, a CPU sub-system is a controller that controls CPU time allocation. The sub-system must be attached to a level to take effect. After a sub-system is attached to a level, all control groups at this level are controlled by this sub-system.

Cgroups hierarchy

Interaction

Every time you create a level in the system, all the tasks in the system are the default cgroup of that level (we call it root cgroup, which is automatically created when the level is created, the cgroup created later in this level is the initial Member of the descendant of this cgroup. A sub-system can be attached to a maximum of one level. Multiple subsystems can be attached to one level. A task can be a member of multiple cgroups, but these cgroups must be at different levels;

When a process (task) in the system creates a sub-process (task), the sub-task automatically becomes a member of the cgroup where the parent process is located. Then, the subtask can be moved to different cgroups as needed, but it always inherits the cgroup of its parent task at the beginning.

3. Docker container capabilities

File System isolation: each container has its own root file system process isolation: each container runs in its own process environment network isolation: the virtual network interfaces and IP addresses between containers are separated by resource isolation and grouping: Use Cgroups to independently allocate resources such as CPU and memory to each Docker container <