Docker Source Code Analysis (a): Docker architecture

1 Background 1.1 Docker introduction

Docker is a container engine project based on lightweight virtualization technology from Docker, the entire project is based on the go language and complies with the Apache 2.0 protocol. Today, Docker can quickly automate deployment of applications within the container, and can provide container resource isolation and security through kernel virtualization technologies (namespaces and cgroups, etc.). Because Docker is isolated through virtualization at the operating system layer, Docker containers run without the additional operating system overhead of similar virtual machines (VMS), increase resource utilization, and improve performance in areas such as IO.

Thanks to its many new features and the openness of the project itself, Docker has quickly gained the favor of many vendors in less than two years, including industry leaders such as Google, Microsoft, and VMware. Google launched the kubernetes in June this year to provide a scheduling service for Docker containers, and this August Microsoft announced support for Kubernetes on Azure, and then VMware, the traditional virtualization giant, announced a partnership with Docker. In mid-September this year, Docker gained $40 million in C-round financing to drive the development of distributed applications.

Looking at the current situation, Docker has a good future. This series of articles, from the source point of view, details the architecture of Docker, the operation of Docker, and the excellent features of Docker. This article is the first--docker architecture of the Docker source Analysis series.

1.2 Docker version Information

The analysis of Docker architecture is based on the results of Docker's corresponding version of source code and Docker, where Docker is the latest version of 1.2.

2 Docker architecture Analytics content Scheduling

The purpose of this article is to analyze the Docker architecture based on understanding the Docker source code. The analysis process is performed in the following three steps:

• Docker's general architecture diagram shows
• function and realization analysis of each module of Docker frame composition
• Take the execution of Docker commands as an example to illustrate the Docker run process

3 Docker General architecture diagram

Learning about Docker's source code is not a boring process, but it can be used to understand the design principles of Docker architecture. Docker is a C/s schema for users, and the back end of Docker is a very loosely coupled architecture, with a combination of modules and an organic mix to support the operation of Docker.

Here, we enclose the total Docker architecture, 3.1.

Figure 3.1 Docker general architecture diagram

3.1, it is not difficult to see that the user is using Docker client to establish communication with Docker daemon and send the request to the latter.

Docker Daemon, as the principal part of the Docker architecture, first provides the functionality of the server to allow it to accept requests from the Docker client, and the engine performs a series of tasks within Docker, each of which is in the form of a job.

During job operation, when container mirroring is required, the image is downloaded from Docker registry and the download image is stored as graph through the mirror management driver Graphdriver, and when the network environment needs to be created for Docker, The Docker container network environment is created and configured through network management-driven networkdriver, and is accomplished by execdriver when it is necessary to restrict the Docker container from running resources or executing user directives.

While Libcontainer is a separate container Management Pack, Networkdriver and Execdriver are implemented by Libcontainer to implement specific operations on the container.

When the command to run the container is executed, an actual Docker container is running, and the container has a separate file system, a separate and secure operating environment, and so on.

4 function and implementation analysis of each module in Docker architecture

Next, we will start with the Docker General architecture diagram, pull out each module within the architecture, and perform more detailed architectural analysis and functional elaboration of each module. The main modules are: Docker Client, Docker Daemon, Docker Registry, Graph, Driver, Libcontainer, and Docker container.

4.1 Docker Client

The Docker client is the customer used by the Docker architecture to establish communication with the Docker daemon. The executable file used by the user is Docker, and the Docker command-line tool can initiate numerous requests to manage container.

Docker client can establish communication with Docker daemon in the following three ways: Tcp://host:port,unix://path_to_socket and FD://SOCKETFD. For the sake of simplicity, this article uses the first method as a prototype for communicating the two. At the same time, the Docker client can set the parameters of the Secure Transport Layer Protocol (TLS) in the form of a command line flag parameter to ensure the security of the transmission when it establishes a connection with the Docker daemon and transmits the request.

After the Docker client sends the container management request, the Docker daemon accepts and processes the request, and when the Docker client receives the returned request and simply processes it, the Docker client completes its entire life cycle. When a container management request continues to be sent, the user must create the Docker Client again through the Docker executable file.

4.2 Docker Daemon

Docker Daemon is a system process that resides in the background in the Docker architecture, and is capable of accepting and processing requests sent by the Docker client. The daemon initiates a server,server in the background responsible for accepting requests sent by the Docker client, and after accepting the request, the server finds the appropriate handler to execute the request by Routing and distributing the dispatch.

The executable file used by Docker daemon startup is also Docker, same as the executable file Docker used to launch the Docker client. When the Docker command executes, the Docker daemon and Docker Client are identified by the parameters passed in.

The architecture of the Docker daemon can be broadly divided into the following three parts: Docker Server, Engine, and job. Daemon Architecture 4.1.

Figure 4.1 Docker Daemon Architecture

4.2.1 Docker Server

Docker server is a server dedicated to Docker client in the Docker architecture. The function of this server is to accept and dispatch requests sent by the Docker client for distribution. Schema 4.2 for Docker server.

Figure 4.2 Docker Server architecture

During the launch of Docker, a MUX was created through the package Gorilla/mux. Router, which provides the requested routing functionality. In Golang, Gorilla/mux is a powerful URL router and dispatch distributor. The MUX. Router adds a number of route entries, each of which consists of the HTTP request method (PUT, POST, get or delete), URL, handler.

If the Docker client accesses the Docker Daemon in HTTP form, the MUX is created. After router, Docker creates a httpsrv=http with the server's listening address and mux.router as a parameter. server{}, the final execution of Httpsrv.serve () is the request service.

During server service, the server accepts the Docker client access request on listener and creates a new Goroutine to service the request. In Goroutine, the request content is read first, then the parsing is done, then the corresponding routing item is found, then the corresponding handler is called to process the request, and finally the request is handler after the request is processed.

It is important to note that the operation of Docker server is performed by a job called "Serveapi" during the boot process of Docker. In principle, the operation of Docker server is one of many jobs, but in order to emphasize the importance of Docker server and the important features for the successor job service, the "Serveapi" job is extracted separately from the analysis and understood as Docker server.

4.2.2 Engine

The engine is the running engines in the Docker architecture and also the core modules of Docker running. It plays the role of the Docker container storage repository and manipulates these containers by performing a job.

During the design and implementation of the engine data structure, there is a handler object. The handler object stores handler processing access to a wide range of job-specific jobs. For example, one of the engine's handler objects is: {"create": Daemon. Containercreate,}, indicates that when the job named "Create" is run, it is daemon. Containercreate's handler.

4.2.3 Job

A job can be considered as the most basic work execution unit within the engine of the Docker architecture. Every job that Docker can do can be abstracted as a job. For example: Running a process inside a container, this is a job, creating a new container, which is a job, downloading a document from the Internet, this is a job, including the one previously mentioned in the Docker Server section, creating the Server service HTTP API, This is also a job, and so on.

Job designer, the job is designed to be similar to the UNIX process. For example, the job has a name, parameters, environment variables, standard input and output, error handling, return status, etc.

4.3 Docker Registry

Docker Registry is a repository for storing container images. The container image is the file schema and directory that is loaded to initialize the container when the container is created.

During Docker operation, Docker Daemon communicates with Docker registry, and implements search images, download images, upload images three functions, the job names of the three functions are "search", "pull" and "push" respectively.

In the Docker architecture, Docker can use the public Docker Registry, known as the Docker hub, so that when Docker acquires a container image file, it must access the Docker hub via the internet At the same time, Docker also allows users to build local private Docker Registry, which ensures that the capture of the container image is completed.

4.4 Graph

Graph plays the custodian of the downloaded container image in the Docker architecture and the record of the relationship between the downloaded container images. On the one hand, graph stores the local file system image with version information and, on the other hand, records the relationship of all file system images to each other through GRAPHDB. Graph's schema 4.3.

Figure 4.3 Graph Schema

Among them, Graphdb is a small graph database built on SQLite, which realizes the naming of nodes and the records of the relationship between nodes. It only implements a small subset of the majority of the graph databases, but provides a simple interface to represent the relationships between nodes.

At the same time, in the local directory of graph, for each container image, the specific information stored is: The container image of the metadata, the size of the container image information, and the container image represents the specific rootfs.

4.5 Driver

The driver is the driver module in the Docker architecture. With the driver drive, Docker enables customization of the Docker container execution environment. Due to the life cycle of Docker, not all operations are managed for Docker containers, there is access to Docker operations information, graph storage and recording, and so on. Therefore, in order to differentiate the management of Docker containers from the internal business logic of Docker daemon, a driver layer driver is designed to take over all this part of the request.

The implementation of Docker driver can be divided into the following three types of drivers: Graphdriver, Networkdriver, and Execdriver.

Graphdriver is primarily used to manage container mirroring, including storage and capture. That is, when the user needs to download the specified container image, Graphdriver stores the container image locally in the specified directory, and when the user needs to create the container's ROOTFS using the specified container image, Graphdriver obtains the specified container image from the local mirror storage directory.

Prior to the Graphdriver initialization process, there were 4 file systems or class file systems registered within them, namely Aufs, Btrfs, VFS, and Devmapper. DOCKER, at the time of initialization, extracts the specified type of DRIVER used by acquiring the system environment variable "docker_driver". All graph operations are then performed using the driver.

Architecture of Graphdriver 4.4:

Figure 4.4 Graphdriver Architecture

The purpose of Networkdriver is to complete the configuration of the Docker container network environment, which includes creating a network bridge for the Docker environment when Docker starts, creating a dedicated virtual NIC device for the Docker container when it is created, and assigning IP to the Docker container, Port mapping with the host, setting the container firewall policy, and so on. Architecture of Networkdriver 4.5:

Figure 4. 5 Networkdriver Architecture

Execdriver, as the execution driver of Docker container, is responsible for creating container running namespace, responsible for statistics and restriction of container resource usage, and responsible for the real running of process inside container. In the implementation of the Execdriver, it was possible to use the LXC driver to invoke the LXC interface to manipulate the configuration and lifecycle of the container, and now execdriver the native driver by default, independent of LXC. This is reflected in the Execdriverflag parameter that was loaded during the daemon startup process, which was set to "native" in the configuration file. This can be thought of as a big change in Docker's version 1.2, or a precursor to Docker's implementation across platforms. Execdriver Architecture 4.6:

Figure 4.6 Execdriver Architecture

4.6 Libcontainer

Libcontainer is a library implemented in the Docker architecture using the Go language design, which is intended to allow the library to directly access the container-related APIs in the kernel without relying on any dependencies.

Because of the existence of Libcontainer, Docker can call Libcontainer directly, and finally manipulate the namespace, cgroups, AppArmor, network devices, and firewall rules of the container. The completion of this series of operations does not need to rely on LXC or other packages. Libcontainer Architecture 4.7:

Figure 4.7 Libcontainer

In addition, Libcontainer provides a complete set of standard interfaces to meet the needs of the upper layer for container management. Or, Libcontainer blocks the direct management of containers on top of Docker. And because Libcontainer uses the Go cross-platform language development implementation, and itself can be accessed by a number of different programming languages, it is difficult to say that the future of Docker will be tightly tied to Linux. At the same time, Microsoft has added support for Docker in Azure, its well-known cloud computing platform, to see how open Docker is and how hot the industry is.

For the time being, Docker is likely to emerge in other container-based platforms due to the functionality of the Libcontainer and its loose coupling with the system, as well as the potential for new projects in the cloud computing world.

4.7 Docker Container

Docker container (Docker container) is the final manifestation of service delivery in the Docker architecture.

Docker orders the appropriate Docker containers according to the user's needs and instructions:

• By specifying the container image, the user allows the Docker container to customize the file system such as Rootfs.
• The user makes the Docker container use the specified computing resources by specifying a quota for the compute resources;
• By configuring the network and its security policy, the user makes the Docker container have an independent and secure network environment.
• The user makes the Docker container perform the specified work by specifying the command to run.

Docker Container 4.8:

Figure 4.8 Docker container

5 Docker Run case analysis

The previous section focuses on the introduction of the various parts of the Docker architecture. The content of this chapter will be analyzed in tandem with the Docker modules to analyze prototypes for Docker pull and Docker run two commands in Docker.

5.1 Docker Pull

The Docker pull command works by downloading the specified container image from Docker registry and storing it in a local graph for use in subsequent creation of the Docker container. Docker pull command execution Flow 5.1.

Figure 5.1 Docker pull Command execution flow

, the Red Arrows marked in the figure indicate the series of operations that Docker has done after the Docker pull command was initiated. Each of these steps is analyzed below.

(1) The Docker client accepts the Docker pull command, resolves the request and collects the request parameters, sends an HTTP request to the Docker Server,http request method as post, and the request URL is "/images/create". "+" XXX ";

(2) Docker Server accepts the above HTTP request and gives it to the MUX. Router,mux. Router the specific handler to execute the request through a URL and a request method;

(3) Mux. The router distributes the request route to the corresponding handler, specifically postimagescreate;

(4) In Postimagecreate This handler, a job called "pull" was created and started to execute;

(5) A job named "pull" executes the pullrepository operation, which is to download one or more of the image from the Docker registry;

(6) The job named "Pull" gives the downloaded image to Graphdriver;

(7) Graphdriver is responsible for storing the image, one side creates the graph object, and on the other hand records the relationship between the image in the graphdb.

5.2 Docker Run

The purpose of the Docker Run command is to run an instruction inside a new Docker container. When Docker executes this command, the work can be divided into two parts: first, the rootfs required to create the Docker container, and the second, the creation of the container's network, and so on, and really run the user instructions. Therefore, during the entire execution process, the Docker client sends two HTTP requests to the Docker server, and the second request is initiated depending on the return status of the first request. The Docker Run command executes process 5.2.

Figure 5.2 Docker Run command execution process

, the Red Arrows marked in the figure indicate the series of operations that Docker did after the Docker Run command was initiated. Each of these steps is analyzed below.

(1) The Docker client accepts the Docker Run command, resolves the request and collects the request parameters, sends an HTTP request to the Docker Server,http request method as post, and the request URL is "/containers/create". "+" XXX ";

(2) Docker Server accepts the above HTTP request and gives it to the MUX. Router,mux. Router the specific handler to execute the request through a URL and a request method;

(3) Mux. The router distributes the request route to the corresponding handler, specifically postcontainerscreate;

(4) In Postimagecreate This handler, a job called "create" was created and started to run the job;

(5) The job named "create" executes the container.create operation, which requires a container image to create a rootfs for the Docker container, called Graphdriver;

(6) Graphdriver get all the mirrors needed to create the Docker container rootfs from graph;

(7) Graphdriver will rootfs all images, load and install to the file directory specified by the Docker container;

(8) If all of the above operations are performed properly and no errors or exceptions are returned, the Docker client initiates a second HTTP request after receiving the Docker server return status. The request method is "POST", the request URL is "/containers/" +container_id+ "/start";

(9) Docker Server accepts the above HTTP request and gives it to the MUX. Router,mux. Router the specific handler to execute the request through a URL and a request method;

(Ten) Mux. The router distributes the request route to the corresponding handler, specifically Postcontainersstart;

(11) In Postcontainersstart This handler, the job named "Start" was created and executed;

(12) The job named "Start" completes the initial configuration work and begins to configure and create the network environment, calling Networkdriver;

Networkdriver need to create a network interface device for the specified Docker container, assign the Ip,port to it, and set up the firewall rules, and the corresponding operation is transferred to the NetLink package in Libcontainer;

NetLink Complete the network environment configuration and creation of the Docker container;

(15) Return to the job named "Start", after performing some auxiliary operations, the job began to execute user instructions, call execdriver;

(() Execdriver is called to initialize the operating environment inside the Docker container, such as namespace, resource control and isolation, and user command execution, and the corresponding operation is transferred to Libcontainer to complete;

Libcontainer is called to finish initializing the runtime environment inside the Docker container and eventually executes the command that the user requested to start.

6 Summary

This article starts with the source code of Docker 1.2, analyzes the architecture diagram of the Docker, analyzes the function and implementation of each module in the architecture diagram, and finally shows the internal operation of Docker through two Docker commands.

By learning from the Docker architecture, you can thoroughly deepen your understanding of Docker design, functionality, and value. At the same time, the use of Docker to achieve a user-customized distributed system, but also better to find the existing platform and Docker more ideal point of agreement. In addition, being familiar with Docker's existing architecture and design ideas can also bring more inspiration to the field of cloud computing PAAs, creating more practice and innovation.

7 References

Http://en.wikipedia.org/wiki/Docker_ (software)

Http://www.slideshare.net/rajdeep/docker-architecturev2

Https://github.com/docker/libcontainer

Http://www.infoq.com/cn/articles/docker-core-technology-preview

https://blog.docker.com/2014/03/docker-0-9-introducing-execution-drivers-and-libcontainer/

Https://crosbymichael.com/the-lost-packages-of-docker.html

Docker Source Code Analysis (a): Docker architecture