Docker Source Code Analysis (iii): Docker daemon Boot

1 Preface

Since its inception, Docker has led the technology boom in lightweight virtualization containers. In this trend, Google, IBM, Redhat and other industry leaders have joined the Docker camp. While Docker is still primarily based on the Linux platform, Microsoft has repeatedly announced support for Docker, from previously announced Azure support Docker and kubernetes, to today's announced next-generation windows The server's original ecosystem supports Docker. This series of moves by Microsoft is a metaphor for a compromise with the Linux world, which, of course, has to be re-aware of the huge impact of Docker.

Docker's influence is self-evident, but if you need to learn more about Docker's internal implementation, I think the most important thing is to understand Docker Daemon. In the Docker architecture, the Docker client communicates with the Docker daemon through a specific protocol, while the Docker daemon primarily hosts most of the work in the Docker runtime. This article is the "Docker Source Code Analysis" series of the third chapter--docker Daemon Chapter.

2 Docker Daemon Introduction

Docker Daemon is a daemon that runs in the background in the Docker architecture and can be broadly divided into Docker Server, engine, and job. Docker daemon can be thought of as accepting requests for Docker clients through the Docker server module, processing requests in the engine, and then creating the specified job and running according to the request type, and the operation process is possible in the following ways: to Docker Registry obtains the mirror, performs the localization operation of the container image through the Graphdriver, performs the configuration of the container network environment through the Networkdriver, executes the execution of the container's internal operation through the Execdriver, etc.

The following is the architecture of the Docker daemon:

3 Docker Daemon Source Analysis content arrangement

From the source point of view, this paper mainly analyzes the starting process of Docker daemon. Since the launch process of Docker daemon and Docker client is very similar, after introducing the start-up process, this article focuses on the most important aspects of the startup process: the implementation of Maindaemon () in the process of creating daemon.

4 Docker Daemon Start-up process

Since the launch of the Docker daemon and Docker client is done through the executable file Docker, the start-up process is very similar. When the Docker executable runs, the code runs through different command-line flag parameters, distinguishing between the two, and eventually running the respective sections.

When you start Docker daemon, you can generally use the following commands: Docker--daemon=true; Docker–d; Docker–d=true and so on. The main () function of Docker then parses the corresponding flag parameter of the above command, and finally completes the Docker daemon boot.

First, attach the launch flowchart for the Docker daemon:

Because of the Docker client chapter in the Docker source Analysis series, there has been a lot of previous work on the main () function in Docker (see Docker Client chapter), and the launch of Docker daemon will involve these tasks, Therefore, this article omitted the same part, but mainly for the follow-up and Docker daemon related content for in-depth analysis, that is, Maindaemon () of the specific source code implementation.

5 Maindaemon () specific implementation

With the Docker daemon flowchart, you can conclude that all of the work on Docker Daemon is included in the implementation of the Maindaemon () method.

In macro terms, Maindaemon () finishes creating a daemon process and makes it work.

From a functional point of view, Maindaemon () implements two parts: first, create a Docker runtime environment, and second, service the Docker Client to receive and process the corresponding request.

From the implementation details, the implementation process of Maindaemon () mainly consists of the following steps:

• The configuration initialization of the daemon, which is implemented in the Init () function, is performed before Maindaemon () is run, but is considered a prerequisite for the operation of the Maindaemon () due to the fact that this part is related to the operation of the Maindaemon ();
• Command line flag parameter check;
• Create an Engine object;
• Set up the engine signal capture and processing method;
• Loading builtins;
• Use Goroutine to load daemon objects and run them;
• Print Docker version and driver information;
• Creation and operation of the job "Serveapi".

The above steps are analyzed in more detail below.

5.0 Configuration Initialization

Before Maindaemon () runs, the config configuration information required for Docker daemon has been initialized. Implemented as follows, located in./docker/docker/daemon.go:

var (daemoncfg = &daemon. config{}) Func init () {daemoncfg.installflags ()}

First, declare a variable of type config in the daemon package named Daemoncfg. The Config object defines the configuration information required by the Docker daemon. When the Docker daemon is started, the DAEMONCFG variable is passed to the Docker daemon and is used.

The Config object is defined as follows (with the explanation of some attributes), located in the./docker/daemon/config.go:

Type Config struct {pidfile string//docker daemon The PID file of the owning process root string//docker runtime The root path used autorestart bool//has been enabled to support the restart of DNS when Docker run []string//docker uses the DNS server address Dnssearch []string//docker uses the specified DNS lookup domain name mirrors []string//Specified priority Docker registry mirror Enableipta            bles BOOL//Enable Docker iptables function enableipforward bool//enable Net.ipv4.ip_forward function enableipmasq BOOL//Enable IP spoofing technology Defaultip NET. IP//Bind container port when using default Ipbridgeiface string//Add container network to existing bridge BRIDGEIP string//Create Network Bridge IP address F IXEDCIDR string//Specifies the IP IPv4 subnet, which must be included in the bridge subnet intercontainercommunication BOOL//allows communication between containers on the same host graphdrive R string//docker Specific storage-driven graphoptions used by the runtime []string//Configurable storage driver Options Execdriver stri       ng//Docker runtime uses specific exec driver MTU INT//SET container network Mtudisablenetwork   BOOL//has been defined, followed by uninitialized enableselinuxsupport bool//Enable SELinux feature support context map[string][]string/ /defined, not initialized after}

After the daemoncfg has been declared, the init () function implements the assignment of each property in the Daemoncfg variable, with the following implementation: Daemoncfg.installflags (), located in./docker/daemon/ Config.go, the code is as follows:

Func (config *config) installflags () {flag. Stringvar (&config. Pidfile, []string{"P", "-pidfile"}, "/var/run/docker.pid", "Path to use for daemon pid file") flag. Stringvar (&config. Root, []string{"G", "-graph"}, "/var/lib/docker", "Path to use as the root of the Docker runtime") ... opts. Ipvar (&config. Defaultip, []string{"#ip", "-ip"}, "0.0.0.0", "Default IP address to use when binding container ports") opts. Listvar (&config. Graphoptions, []string{"-storage-opt"}, "Set Storage Driver Options") ...}

In the implementation of the Installflags () function, the key is to define some type of flag parameter and bind the value of the parameter to the specified property of the config variable, such as:

Flag. Stringvar (&config. Pidfile, []string{"P", "-pidfile"}, "/var/run/docker.pid", "Path to use for daemon pid file")

The meaning of the above statement is:

• Defines a flag parameter that is of type string;
• The flag is named "P" or "-pidfile";
• The flag has a value of "/var/run/docker.pid" and binds the value to the variable CONFIG. On the Pidfile;
• The flag describes the message "Path to use for daemon PID file".

At this point, the configuration information required for Docker Daemon is declared and initialized.

5.1 Flag parameter Check

Starting from this section, the Maindaemon () run analysis that really goes into Docker daemon.

The first step is the check of the flag parameter. Specifically, when the Docker command is parsed by the flag parameter, it determines whether the remaining parameters are 0. If 0, the Docker daemon startup command is correct, normal operation, if not 0, it means that when the Docker daemon is started, the extra parameters are passed in, the error prompt is output and the program exits. The specific code is as follows:

If flag. Narg ()! = 0 {flag. Usage () return}

5.2 Creating an Engine object

During Maindaemon () operation, after the flag parameter is checked, the engine object is created with the following code:

ENG: = engine. New ()

The engine is the running engines in the Docker architecture and is the core module for Docker operation. The engine plays the role of the Docker container storage repository and manages these containers in the form of a job.

In./docker/engine/engine.go, the engine structure is defined as follows:

Type Engine struct {handlers   map[string]handlercatchall   handlerhack       Hack//data for temporary hackery (see hack.go) ID         stringstdout     io. Writerstderr     io. Writerstdin      io. readerlogging    booltasks      sync. WAITGROUPL          sync. Rwmutex//Lock for Shutdownshutdown   Boolonshutdown []func ()//Shutdown Handlers}

Among them, the most important thing in the engine structure is the handlers attribute. The Handlers property is the map type, the key is a string type, and value is the handler type. Where the handler type is defined as follows:

Type Handler func (*job) Status

Visible, handler is a defined function. The parameter passed in by the function is the job pointer and returns to the status state.

After the introduction of engine and handler, now really into the implementation of the new () function:

Func New () *engine {eng: = &engine{handlers:make (Map[string]handler), ID:       utils. RandomString (), Stdout:   os. Stdout,stderr:   os. Stderr,stdin:    os. Stdin,logging:  True,}eng. Register ("Commands", func (Job *job) Status {for _, Name: = Range Eng.commands () {job. Printf ("%s\n", name)}return Statusok})//Copy existing global handlersfor k, V: = range Globalhandlers {Eng.handlers[k] = V}return Eng}

By analyzing the above code, you know that the new () function eventually returns an engine object. In the Code implementation section, the first job is to create an engine structure instance, Eng, and the second is to register the handler with the Eng object named commands, where handler is the temporarily defined function func (Job *job) status{}, The function is to print all the command names that have been registered through the job, and finally return the status Statusok; the third job is to globalhandlers all the handler in the defined variable are copied to the handlers property of the Eng object. Finally, the Eng object is successfully returned.

5.3 Setting the engine's signal capture

Back in the run of the Maindaemon () function, execute the following code:

Signal. Trap (Eng. Shutdown)

The function of this part of the code is: in the operation of the Docker daemon, set the trap specific signal processing method, the specific signal has sigint,sigterm and Sigquit, when the program captures the SIGINT or sigterm signal, perform the appropriate aftercare operations, Finally, ensure that the Docker daemon program exits.

The implementation of the code for this section is located in./docker/pkg/signal/trap.go. The implementation process is divided into the following 4 steps:

• Create and set up a channel for sending signal notifications;
• Defines an signals array variable with an initial value of OS. SIGINT, OS. SIGTERM; If the environment variable debug is empty, add the OS. Sigquit to signals array;
• Through Gosignal. Notify (c, signals ...) The Notify function is implemented to pass the received signal signal to C. It should be noted that only the signals listed in the signals will be transmitted to C, the rest of the signal will be directly ignored;
• Create a goroutine to handle the specific signal signal when the signal type is OS. Interrupt or syscall.sigterm, the specific execution method of the incoming trap function is executed, the formal parameter is cleanup (), and the argument is eng. Shutdown.

The definition of the Shutdown () function is located in./docker/engine/engine.go, and the main task is to do some aftercare for the closure of the Docker daemon.

Rehabilitation work is as follows:

• Docker daemon no longer receives any new job;
• Docker Daemon waits for all surviving jobs to complete;
• Docker Daemon calls all shutdown processing methods;
• When all handler are executed, or after 15 seconds, the Shutdown () function returns.

Due to the signal. Trap (Eng. Shutdown) function to implement the ENG in a specific implementation. Shutdown, after executing eng. After the shutdown, the OS is executed. Exit (0) to complete the immediate exit of the current program.

5.4 Loading Builtins

After setting the processing method for the specific signal of the trap for Eng, the Docker daemon implements the Builtins load. The code is implemented as follows:

If err: = Builtins. Register (Eng); Err! = Nil {log. Fatal (ERR)}

The main task of loading builtins is to register multiple handler for the engine so that the specified handler can be run later when the corresponding task is performed. These handler include: network initialization, Web API service, event query, version view, Docker registry authentication and search. The code implementation is located in./docker/builtins/builtins.go, as follows:

Func Register (Eng *engine. Engine) Error {if err: = Daemon (ENG); Err! = Nil {return err}if err: = Remote (ENG); Err! = Nil {return err}if err: = Event S.new (). Install (Eng); Err! = Nil {return err}if err: = Eng. Register ("version", dockerversion); Err! = Nil {return Err}return registry. NewService (). Install (Eng)}

The main 5 parts of the following analysis are implemented: Daemon (ENG), Remote (ENG), events. New (). Install (ENG), Eng. Register ("version", Dockerversion) and registry. NewService (). Install (Eng).

5.4.1 Register to initialize the network-driven handler

Daemon (ENG) implementation process, mainly for the Eng object registered a key "Init_networkdriver" handler, the value of the handler is the Bridge.initdriver function, the code is as follows:

Func Daemon (Eng *engine. Engine) Error {return eng. Register ("Init_networkdriver", Bridge. Initdriver)}

It should be noted that registering handler with the Eng object does not mean that the handler value function will be run directly, such as bridge. Initdriver, and would not run directly, but would bridge. The function entry for Initdriver, written in the Handlers property of Eng.

The specific implementation of Bridge.initdriver is located in./docker/daemon/networkdriver/bridge/driver.go, the main role of:

• Gets the address of the network device for the Docker service;
• Create a network bridge with the specified IP address;
• Configure network iptables rules;
• Several handler have also been registered for Eng objects, such as "Allocate_interface", "Release_interface", "allocate_port", "link".

5.4.2 registering the API service handler

Remote (ENG) Implementation process, mainly for the Eng object registered two handler, respectively, "Serveapi" and "acceptconnections". The code is implemented as follows:

Func remote (Eng *engine. Engine) Error {if err: = Eng. Register ("Serveapi", Apiserver. SERVEAPI); Err! = Nil {return Err}return eng. Register ("Acceptconnections", Apiserver. Acceptconnections)}

The registered two handler names are "Serveapi" and "acceptconnections" respectively, and the corresponding execution methods are apiserver respectively. Serveapi and Apiserver.acceptconnections, specifically implemented in./docker/api/server/server.go. Where Serveapi executes, it creates a goroutine to configure the specified HTTP by looping through a variety of protocols. Server, eventually requesting services for different protocols, while the acceptconnections is implemented primarily to inform the Init daemon that Docker daemon has been started and allows the Docker daemon process to accept requests.

5.4.3 Registering for events Handler

Events. New (). The implementation of the Install (Eng), which registers multiple event events for Docker, provides Docker users with an API that allows them to view the events information inside Docker, log information, and Subscribers_ Count information. The specific code is located in the./docker/events/events.go, as follows:

Func (e *events) Install (Eng *engine. Engine) Error {jobs: = Map[string]engine. handler{"Events":            e.get, "log":               e.Log, "Subscribers_count": e.subscriberscount,}for name, Job: = Range Jobs {if ERR: = Eng. Register (name, job); Err! = Nil {return Err}}return nil}

5.4.4 registered version of Handler

Eng. Register ("version", dockerversion) of the implementation process, the Eng object registration key is "version", Value is "dockerversion" execution method of handler, During the execution of the dockerversion, the version of Docker is written to the standard output of the job named version, the version of the Docker API, the GIT version, the Go Language runtime version, and the operating system version information. The specific implementation of Dockerversion is as follows:

Func dockerversion (Job *engine. JOB) engine. Status {V: = &engine. Env{}v.setjson ("Version", dockerversion. VERSION) V.setjson ("Apiversion", API. apiversion) V.set ("Gitcommit", dockerversion. Gitcommit) V.set ("Goversion", runtime. Version ()) V.set ("Os", runtime. GOOS) V.set ("Arch", runtime. Goarch) If kernelversion, err: = Kernel. Getkernelversion (); Err = = Nil {v.set ("Kernelversion", Kernelversion.string ())}if _, Err: = V.writeto (Job. Stdout); Err! = Nil {return job. Error (ERR)}return engine. Statusok}

5.4.5 Registered Registry's handler

Registry. NewService (). The Install (ENG) Implementation process is located at./docker/registry/service.go, which adds Docker registry information to the API information exposed by the Eng object. When registry. If the NewService () is successfully installed, two calls can be used by eng: "Auth", certified to the public registry, "Search", and the specified image is searched on the public registry.

The specific implementation of the install is as follows:

Func (S *service) Install (Eng *engine. Engine) error {eng. Register ("auth", S.auth) Eng. Register ("Search", S.search) return nil}

At this point, all builtins loading is complete, enabling the registration of specific handler to the Eng object.

5.5 Loading daemon objects with Goroutine and running

After executing the builtins load, go back to Maindaemon () execution, load the Daemon object through a goroutine and start running. The implementation of this link consists of three main steps:

• Create a Daemon object with the Daemoncfg and Eng objects initialized in the INIT function;
• Registering many handler with the Eng object through the install function of the Daemon object;
• After the Docker daemon is started, a job called "Acceptconnections" is run, and the main work is to send a "ready=1" signal to the init daemon to start accepting requests normally.

The code is implemented as follows:

Go func () {d, err: = Daemon. Maindaemon (Daemoncfg, ENG) If err! = Nil {log. Fatal (Err)}if err: = D.install (ENG); Err! = Nil {log. Fatal (Err)}if ERR: = Eng. Job ("Acceptconnections"). Run (); Err! = Nil {log. Fatal (Err)}} ()

The following is a breakdown of the work done in three steps.

5.5.1 Creating Daemon Objects

Daemon. Maindaemon (Daemoncfg, ENG) is the core part of creating Daemon object D. The main function is to initialize the basic environment of Docker daemon, such as processing config parameter, verifying system support degree, configuring Docker working directory, setting up and loading various driver, creating graph environment, verifying DNS configuration and so on.

Due to daemon. Maindaemon (Daemoncfg, ENG) is the core part of loading Docker daemon, and is too lengthy to arrange the fourth section of the Docker Source Analysis series.

5.5.2 register engine with daemon object handler

When you finish creating the Daemon object, Goroutine executes D. Install (ENG), specifically implemented at./docker/daemon/daemon.go:

Func (daemon *daemon) Install (Eng *engine. Engine) error {for name, method: = Range map[string]engine. handler{"Attach":            daemon. Containerattach,...... "Image_delete":      daemon. Imagedelete,} {if err: = Eng. Register (name, method); Err! = Nil {return err}}if err: = Daemon. Repositories (). Install (Eng); Err! = Nil {return Err}eng. Hack_setglobalvar ("Httpapi.daemon", daemon) return nil}

The implementation of the above code is divided into three parts:

• Registering numerous handler objects with the Eng object;
• Daemon. Repositories (). The Install (ENG) implements the registration of multiple image-related Handler,install to the Eng object located in the./docker/graph/service.go;
• Eng. Hack_setglobalvar ("Httpapi.daemon", daemon) implements adding a record to the Hack object of the map type in the Eng object, key is "Httpapi.daemon" and value is daemon.

5.5.3 running the Acceptconnections job

A job named "Acceptconnections" was last run inside Goroutine, and the primary role was to inform the Init daemon that Docker daemon could begin accepting requests.

This is the first time in the source analysis series involved in the operation of the specific job, the following simple analysis "acceptconnections" the job run.

You can see that the ENG is executed first. Job ("Acceptconnections"), return a job, and then execute Eng. Job ("Acceptconnections"). Run (), which is the run function of the execution job.

Eng. The implementation of the Job ("Acceptconnections") is located in./docker/engine/engine.go, as follows:

Func (Eng *engine) Job (name string, args ... string) *job {Job: = &job{eng:    eng,name:   Name,args:   args, Stdin:  newinput (), Stdout:newoutput (), Stderr:newoutput (), env:    &env{},}if Eng. Logging {job. Stderr.add (Utils. Nopwritecloser (Eng. STDERR))}if handler, exists: = Eng.handlers[name]; exists {Job.handler = handler} else if Eng.catchall! = Nil && name! = "" {Job.handler = Eng.catchall}return job}

From the above code, we first create a Job object of type job, in which the Eng property is the caller of the function Eng,name property is "Acceptconnections" and no parameters are passed in. In addition, in the handlers attribute of the Eng object, look for the value of the key "Acceptconnections" record, since the remote (ENG) in the load Builtins operation has registered such a record with Eng, key is " Acceptconnections ", value is apiserver. Acceptconnections. Therefore, the handler of the Job object is apiserver. Acceptconnections. Finally, the object job that has already been initialized is returned.

When the Job object is created, the run () function of the Job object is executed. The implementation of the Run () function is located in./docker/engine/job.go, which executes the specified job and blocks until the job execution is complete. For a Job object named "Acceptconnections," The Run code is Job.status = Job.handler (Job), Because the Job.handler value is apiserver.acceptconnections, the real execution is job.status = Apiserver. Acceptconnections (Job).

Enter the specific implementation of Acceptconnections, located in./docker/api/server/server.go, as follows:

Func acceptconnections (Job *engine. JOB) engine. Status {//Tell the init daemon we are accepting Requestsgo  systemd. Sdnotify ("Ready=1") if activationlock! = nil {close (Activationlock)}return engine. Statusok}

The focus is go systemd. The implementation of Sdnotify ("Ready=1"), located in./docker/pkg/system/sd_notify.go, is the primary function of notifying the init daemon that the boot of Docker daemon has been fully completed, and that the potential function is to make Docker Daemon begins accepting API requests sent by the Docker client.

At this point, you are finished loading the daemon object and running through Goroutine.

5.6 Printing Docker version and driver information

Back in the running flow of Maindaemon (), the other code inside the Maindaemon () function executes concurrently at the time of execution of the goroutine.

The first execution is to display the version information for Docker, as well as the specific information about the two drivers, Execdriver and Graphdriver, with the following code:

Log. Printf ("Docker daemon:%s%s; Execdriver:%s; Graphdriver:%s ", Dockerversion. Version,dockerversion. Gitcommit,daemoncfg.execdriver,daemoncfg.graphdriver,)

Creation and operation of the serveapi of 5.7 job

After printing part of the Docker details, Docker Daemon immediately creates and runs the job named "Serveapi", with the main role of letting Docker daemon provide API access services. The implementation code is located in./docker/docker/daemon.go#l66, as follows:

Job: = Eng. Job ("Serveapi", flhosts ...) Job. Setenvbool ("Logging", true) job. Setenvbool ("Enablecors", *flenablecors) job. Setenv ("Version", dockerversion. VERSION) job. Setenv ("Socketgroup", *flsocketgroup) job. Setenvbool ("Tls", *fltls) job. Setenvbool ("Tlsverify", *fltlsverify) job. Setenv ("Tlsca", *flca) job. Setenv ("Tlscert", *flcert) job. Setenv ("Tlskey", *flkey) job. Setenvbool ("Bufferrequests", True) if err: = job. Run (); Err! = Nil {log. Fatal (ERR)}

In the implementation process, first create a job named "Serveapi" and assign the value of Flhosts to Job.args. Flhost's role is primarily to provide Docker daemon with the protocol used and the address of the listener. Subsequently, Docker daemon set up a number of environment variables for the job, such as environment variables for the secure Transport layer protocol. Finally through the job. Run () The job that runs the Serveapi.

Since the handler,value of the key "Serveapi" in Eng is apiserver. Serveapi, so the job runs, Apiserver is executed. The Serveapi function, located at./docker/api/server/server.go. The role of the SERVEAPI function is primarily for all user-defined support protocols, and Docker Daemon creates a goroutine to initiate the corresponding http.server, respectively, for different protocol services.

Because HTTP is created and started. Server is an important part of the Docker architecture for Docker server, and the Docker Source Analysis series is analyzed in the fifth article.

At this point, you can assume that the Docker daemon has completed the initialization of the SERVEAPI job. Once the acceptconnections job is complete, it notifies the Init process that Docker daemon is ready to start providing the API service.

6 Summary

This paper analyzes the start of Docker daemon from the source point of view, and emphatically analyzes the realization of Maindaemon ().

Docker Daemon is the backbone of the Docker architecture and is responsible for the management of almost all operations within Docker. Learning the implementation of Docker Daemon can have a more comprehensive understanding of the Docker architecture. In summary, the operation of Docker, carrier for daemon, scheduling management by engine, task execution by job.

7 References

1. Http://www.infoq.com/cn/news/2014/10/windows-server-docker
2. Http://www.freedesktop.org/software/systemd/man/sd_notify.html
3. Http://www.ibm.com/developerworks/cn/linux/1407_liuming_init3/index.html
4. http://docs.studygolang.com/pkg/os/
5. https://docs.docker.com/reference/commandline/cli/

Docker Source Code Analysis (iii): Docker daemon Boot