Docker (i): Docker Core Technology Preview

Docker is a LXC-based advanced container engine for PAAs vendors DotCloud open source, which is hosted on GitHub and is open source based on the go language development and compliance with the Apache 2.0 protocol. Docker provides a way to automate the deployment of software in a secure, repeatable environment, with the advent of a change in the way the cloud-based platform releases products. In order to better promote the development and dissemination of Docker in China, we decided to open the "in-depth Docker" column, inviting Docker-related preachers, developers, technical experts to tell the various aspects of Docker, let the reader have a better understanding of Docker, And be able to actively invest in the discussion and practice of new technology. In addition, welcome to join INFOQ Docker Technology Group Exchange Docker Best practice, QQ Group number: 482445072.

Extra: Cnutcon The global container Technology conference is in the hot application, the theme of this conference is "Analysis of container enterprise practice, focus on the container ecosystem Open Source project", click on the official website to learn more.

1. Background 1.1. From PAAs to container

In February 2013, former Gluster CEO Ben Golub and DotCloud's CEO Solomon Hykes sat down to chat, Solomon talked about the DotCloud container technology to be used in a separate open source, And then to open a new company around this technology to provide technical support. Solomon, 28, found that using Python to develop DotCloud's PAAs cloud, the use of LXC (Linux Container) technology could break the challenge of not having an easy collaboration between application development engineers and system engineers in the release process of products. This container container technology frees developers from the intricacies of daily deployment, allowing developers to focus on writing programs, and from a system engineer's point of view, they urgently need to be freed from the chaos of deployment documents, allowing system engineers to focus on the level of application expansion, Stable release of the solution. The more they talked, the more they felt it was a revolution in cloud technology, followed by the launch of the March 2013 Docker 0.1, which kicked off a change in the way the cloud-based platform released its products.

1.2 Docker Introduction

Docker is a container container engine built on LXC technology based on the Docker.inc company's open source, which is hosted on GitHub, open source based on the go language and compliant with the APACHE2.0 protocol. Docker held the Dockerconf 2014 technology conference in June 2014, attracting attention and technical support from industry-leading companies such as IBM, Google, and Redhat, regardless of the code activity on GitHub, Or is Redhat announcing the official support of Docker in RHEL7, giving the industry a signal that this is an innovative technology solution. Even Google's Compute Engine supports Docker running on it, and the domestic "BAT" pioneer Baidu App Engine (BAE) platform is also based on Docker as its PAAs Cloud Foundation.

Related Vendor Content

CoreOS chief evangelist, Red Hat company vice president confirms Cnutcon in the presentation of Docker on AWS: Running a container solution in a cloud environment Cnutcon exciting content preview: When did the first-line internet company's container application case sharing bat start using containers or Docker technology? What is the application scenario? Container Cluster management behind Docker-from Borg to Kubernetes (i)

Related Sponsors

Cnutcon Container Technology Conference, August 28, 2015-29th Crowne Plaza Hotel Beijing New Yunnan. Sign up now.

Docker is created to address the following issues:

1) Environmental management Complex: from a variety of OS to a variety of middleware to a variety of apps, a product can be successfully released, as developers need to care about too much, and difficult to manage, this problem in the software industry is common and need to face directly. Docker simplifies the deployment of multiple application instances, such as Web applications, background applications, database applications, big data applications such as Hadoop clusters, message queues, and so on, all of which can be packaged into an image deployment. ：

2) The advent of cloud computing: The success of AWS, leading developers to move applications to the cloud, solve the problem of hardware management, but the software configuration and management-related issues persist (AWS cloudformation is the industry standard in this direction, sample template can refer to here). The advent of Docker can help software developers broaden their mind and try new software management methods to solve this problem.

3) Changes in virtualization: The Cloud ERA uses standard hardware to reduce costs, using virtualization to meet the resource needs of users on demand and to ensure availability and isolation. However, whether it's KVM or Xen, Docker seems to be wasting resources because users need an efficient operating environment rather than an OS, Guestos is both wasteful and difficult to manage, and lightweight lxc more flexible and fast. ：

4) LXC Portability: LXC is already in the Linux 2.6 Kernel, but it is not designed for cloud computing, lacks standardized descriptive means and portability of containers, and determines the environment in which it is built is difficult to distribute and standardize management ( Relative to the concept of image and snapshot such as KVM). Docker has made a substantial and innovative approach to this issue.

1.3 Docker's Hello World

Take Fedora 20 as the host for example and install Docker-io directly:

$ sudo yum-y install Docker-io

Launch Docker backend Daemon:

$ sudo systemctl start docker

Run our first Hello World container:

$ sudo docker run-i-t Fedora/bin/echo Hello Worldhello World

You can see that the next line after the command line is run prints out the classic Hello World string.

2. Core Technology Preview

The Docker core is an operating system-level virtualization approach that may not be as intuitive to understand as a VM. We detail Docker's technical details from four aspects of the virtualization approach: isolation, quota / scalability, portability, and security .

2.1. Isolation: Linux Namespace (NS)

Each user instance is isolated from each other and does not affect each other. The method given by the general hardware virtualization method is the VM, and the method given by LXC is container, even finer is kernel namespace. Which pid, net, IPC, mnt,UTS, user and other namespace will container process, network, message, File system, UTS ("UNIX time-sharing system") and user space are isolated.

1) PID namespace

The process of different users is separated by PID namespace, and the same PID can be used in different namespace. The parent process for all LXC processes in Docker is the Docker process, and each LXC process has a different namespace. It is also possible to implement Docker in Docker conveniently because it allows nesting.

2) Net namespace

With PID namespace, the PID in each namespace can be isolated from each other, but the network port is also the port that shares the host. Network isolation is achieved through NET namespace, each net namespace has a separate network devices, IP addresses, IP routing tables,/proc/net directory. So that every container network can be isolated. Docker by default uses Veth to connect the virtual NIC in container with a Docker Bridge:docker0 on host.

3) IPC namespace

Process interactions in container are also based on common inter-process interaction methods (interprocess COMMUNICATION-IPC) of Linux, including common semaphores, message queues, and shared memory. Unlike VMS, however, container interaction between processes is actually a process interaction in the same PID namespace on the host, so you need to add namespace information to the IPC resource request-each IPC resource has a unique 32-bit ID.

4) mnt namespace

Similar to chroot, a process is placed into a specific directory for execution. MNT namespace allows different namespace processes to see different file structures so that each namespace process sees a file directory that is isolated. Unlike chroot, the information in/proc/mounts for each container in the namespace contains only mount point where the namespace is located.

5) UTS namespace

UTS ("UNIX time-sharing System") namespace allows each container to have a separate hostname and domain name so that it can be viewed as a separate node on the network rather than a process on the host.

6) User namespace

Each container can have a different user and group ID, which means that the program can be executed internally within the container with the user inside the container rather than the user on the host.

2.2 Quota/measurable-Control Groups (cgroups)

Cgroups implements quotas and metrics for resources. Cgroups is very simple to use, provide similar file interface, create a new group in the/cgroup directory, create a new task file in this folder, and write the PID to the file, to achieve the resource control of the process. Groups can limit Blkio, CPU, CPUACCT, cpuset, devices, freezer, memory, NET_CLS, NS Nine subsystems of the resources, the following is a detailed description of each subsystem:

1. Blkio this subsystem setting restricts the input and output control of each block device. For example: disks, CDs, USB, and so on.
2. CPU this subsystem uses the scheduler to provide CPU access to the Cgroup task.
3. CPUACCT generates CPU resource reports for cgroup tasks.
4. Cpuset if it is a multi-core CPU, this subsystem allocates a separate CPU and memory for the Cgroup task.
5. Devices allows or denies access to the device for the Cgroup task.
6. Freezer pause and Resume cgroup tasks.
7. Memory sets each cgroup and generates memory resource reports.
8. NET_CLS tags each network packet for Cgroup convenience.
9. NS namespace subsystem.

There is a certain relationship between the above nine subsystems. Please refer to the official documentation for details.

2.3 Portability: AUFS

AUFS (ANOTHERUNIONFS) is a Union FS, which simply supports mounting different directories under the same virtual file system (unite several directories into a, virtual filesystem) FileSystem, further understanding, AUFS supports setting readonly, ReadWrite, and whiteout-able permissions for each member directory (like Git Branch), while AUFS has a similar layering concept to the readonly permissions of B Ranch can be modified logically (incrementally, without affecting the readonly section). In general, the Union FS has two purposes, on the one hand can be implemented without the help of LVM, RAID multiple disk to the same directory, another more commonly used is a readonly branch and a writeable branch union together, Live C D is based on this method to allow the user to perform some write operations on the OS image, based on the invariant. The same is true of the container image that Docker built on AUFS, and then we'll start with the Linux example in container to introduce the use of Docker for AUFS features.

A typical startup Linux operation requires two Fs:bootfs + Rootfs:

Bootfs (boot file system) mainly contains bootloader and kernel, bootloader is mainly boot load kernel, when boot succeeds kernel is loaded into memory Bootfs is umount. Rootfs (Root file system) contains standard directories and files such as/dev,/proc,/bin, etc in a typical Linux system.

For different Linux distributions, Bootfs is basically the same, but the rootfs will be different, so the various distributions can be common bootfs such as:

After booting a typical Linux, first set Rootfs to ReadOnly, perform a series of checks, and then switch it to "ReadWrite" for the user to use. In Docker, the ROOTFS is also loaded and checked in ReadOnly mode at initialization, but then the union mount is used to mount a ReadWrite file system above the Rootfs of ReadOnly and allows the underlying FS ( The file system is set to readonly and superimposed upward so that a set of readonly and a writeable structure constitute a container of the running tenses, each of which is called a FS layer. Such as:

Thanks to the Aufs feature, each modification to the readonly layer File/directory will only exist in the upper writeable layer. Because there is no competition, multiple container can share the ReadOnly FS layer. So Docker calls the ReadOnly FS layer "image"-The entire rootfs is read-write for container, but in fact all the modifications are written to the top-level writeable layer, Image does not save user state and is used only for templates, new, and replicated.

The image on the upper layer relies on the lower image, so the image below is called the parent image in Docker, and the image without the parent image is called the base image. Therefore, to start a container,docker from an image, the image and dependent parent images and base image are loaded first, and the user's process runs in the layer of writeable. The specific container configuration of data information in all parent image, as well as the resource limits of the ID, network, and LXC management, constitutes a Docker conceptual container. Such as:

2.4 Security: AppArmor, SELinux, grsec

Security is always relative, there are three aspects to consider Docker security features:

1. The inherent security standards of Linux systems implemented by kernel namespaces and cgroups;
2. Secure interface for Docker Deamon;
3. Linux itself security hardening solutions, such as AppArmor, SELinux;

Because security is a very specific technology, it is not mentioned here, please refer directly to the official Docker documentation.

3. Latest sub-project introduction

Let's take a look at what other sub-projects in the Docker community are worth studying and studying. For this purpose, I have listed some interesting core projects to allow enthusiastic readers to quickly follow up on projects of interest to them:

1. Libswarm, Solomon Hykes, the CTO of Docker, introduced the new Lego bricks tool to the community at the Dockercon 2014 summit: It is an API for unifying the network interfaces of distributed systems. Libswarm to solve the problem is that distributed applications based on Docker have spawned multiple Docker-based service discovery (Serivce Discovery) projects such as ETCD, Fleet, Geard, Mesos, shipyard, Serf and so on, each solution has its own protocol and usage, and any one of them will be confined to a particular technology range. So Docker's CTO wanted to expose the generic API interface to distributed systems using LIBSWARM, breaking the established protocol limits. The project is still at an early stage of development and deserves to be involved.
2. Libchan, is a bottom-level network library that provides support for upper Libswarm. The equivalent of adding ZEROMQ or RABBITMQ to Docker, the benefit of implementing the network library itself is that Docker is optimized and lightweight. The general developer does not use it directly, and we use libswarm to interact with the container. Network engineers who like the bottom line may be interested in this, so take a look.
3. The core part of Libcontainer,docker technology is also listed separately because this piece of functionality is relatively independent, and the iterative upgrade of the function code is very fast. To learn more about Docker's latest support features, you should pay more attention to this module.

4. Summary

The Docker community is always in the face of technical challenges and calmly gives its own solutions. With the development of cloud computing, many important issues have not been properly addressed, and Docker is trying to get mainstream vendors to accept and apply it. At this point, the above Docker Technology Preview to this point, I also hope that readers can combine their own reality, try to use Docker technology. Because only on the basis of personal experience, such as Docker Cloud technology will produce greater value.

Docker (i): Docker Core Technology Preview