Docker source Analysis (ii): Docker client creation and command execution

1. Preface

Today, Docker, the industry's leading lightweight virtualization container management engine, provides a new and convenient way for global developers to test and deploy software integration. When the team develops the software, Docker provides a reusable runtime environment, flexible resource configuration, convenient integration testing methods, and one-click Deployment. It can be said that the advantages of Docker in simplifying continuous integration, operational deployment of the most vividly, it completely let the developers from the continuous integration, operation and maintenance of the deployment of the liberation, the focus on real development.

However, it is not an easy job to maximize the functionality of Docker. In the context of a deep understanding of the Docker architecture, it is also necessary to master the use of the Docker client. The former can be described in the Docker Source analysis series of Docker architecture, and this article is mainly aimed at the latter, from the source point of view of the Docker client, to help developers understand the specific implementation of the Docker client, and ultimately better grasp the Docker How the client is used. This is the second--docker client chapter of the Docker source Analysis series.

2. Docker Client source Code Analysis Section arrangement

This article from the source point of view, the main analysis of the Docker client two aspects: Create and command execution. The entire analysis process can be divided into two parts:

The first section analyzes the creation of the Docker client. This part of the analysis can be divided into the following three steps:

• Analyze how to parse out command line flag parameters and request parameters in Docker commands through Docker commands;
• Analyze how to handle the specific flag parameter information and collect the configuration information required by the Docker client;
• Analyze how to create a docker Client.

The second part analyzes how to execute the Docker command on the basis of the existing Docker client. This part of the analysis can be divided into the following two steps:

• Analyze how to parse the request parameters in the Docker command to get the type of the corresponding request;
• Analyzes how the Docker client executes a specific request command and eventually sends the request to the Docker Server.

3. Creation of Docker Client

The creation of the Docker client is essentially the client that Docker users connect to Docker server through the executable file docker. The following three subsections describe the process of creating a Docker client.

The following flowchart runs for the entire Docker source code:

The flowchart allows the reader to understand more clearly the process of creating and executing requests by Docker client. It involves a number of unique nouns in the source code, which are explained and analyzed below.

3.1. Flag parameter resolution for Docker commands

As is well known, in the specific implementation of Docker, both the Docker server and the Docker client are created and started by the executable file Docker. It is especially important to understand how Docker executables differentiate between the two.

For both, first illustrate the difference. The Docker server starts with the command docker-d or Docker--daemon=true, while the Docker client launches as Docker--daemon=false PS, Docker pull name, and so on.

The parameters in the above Docker request can be divided into two categories: the first is the command-line parameter, which is the parameters required by the Docker program to run, such as:-D 、--daemon=true 、--Daemon=false, and so on; the second type is Docker sent to Docker The actual request parameters of the server, such as PS, pull name, and so on.

For the first class, we are accustomed to call it the flag parameter, in the Go Language standard library, also provides a flag packet, convenient for the command line parameter parsing.

After the above background, then entered the implementation of the Docker client created source code, located in./docker/docker/docker.go, the Go file contains the entire Docker's main function, that is, the entire Docker (regardless of Docker Daemon or Docker Client) running the portal. Part of the main function code is as follows:

Func Main () {    if reexec. Init () {      return    }    flag. Parse ()    //Fixme:validate daemon flags here ...    }

In the above code, first Judge Reexec. The return value of the Init () method, if true, exits the run directly, otherwise continues execution. Looking at the definition of Reexec.init () in./docker/reexec/reexec.go, it can be found that the return value of the code snippet is false because there is no initializer registration before the Docker runs.

Immediately after, the main function calls flag. Parse () parses the flag parameter in the command line. View source you can see that Docker defines multiple flag parameters in./docker/docker/flag.go and initializes it with the Init function. The code is as follows:

var (flversion = flag. Bool ([]string{"V", "-version"}, False, "Print version information and quit") Fldaemon = flag. Bool ([]string{"D", "-daemon"}, False, "Enable daemon mode") Fldebug = flag. Bool ([]string{"D", "-debug"}, False, "Enable debug mode") Flsocketgroup = flag.  String ([]string{"G", "-group"}, "Docker", "Group to assign the UNIX socket specified by-h when running in daemon mode use ' (The empty string) to disable setting of a group ") Flenablecors = flag. Bool ([]string{"#api-enable-cors", "-api-enable-cors"}, False, "enable Cors headers in the remote API") Fltls = FL Ag. Bool ([]string{"-tls"}, False, "use TLS; Implied by tls-verify flags ") Fltlsverify = flag. Bool ([]string{"-tlsverify"}, False, "use TLS and verify the remote (daemon:verify client, client:verify daemon)")//TH ESE is initialized in Init () below since their default values depend on Dockercertpath which isn ' t fully initialized unti L init () runs Flca *string Flcert *string FLKey *string flhosts []string) func init () {FLCA = flag. String ([]string{"-tlscacert"}, filepath. Join (Dockercertpath, Defaultcafile), "Trust only remotes providing a certificate signed by the CA given here") Flcert = f Lag. String ([]string{"-tlscert"}, filepath. Join (Dockercertpath, Defaultcertfile), "Path to TLS certificate file") FlKey = flag. String ([]string{"-tlskey"}, filepath. Join (Dockercertpath, Defaultkeyfile), "Path to TLS key file") opts. Hostlistvar (&flhosts, []string{"H", "-host"}, "the socket (s) to bind to with daemon mode\nspecified using one or more TC P://host:port, Unix:///path/to/socket, fd://* or FD://SOCKETFD. ")}

This involves a feature of the Golang, the execution of the INIT function. The characteristics of the INIT function in Golang are as follows:

• The init function is used for initialization of the package before the program executes, such as initializing variables, etc.
• Each package can have more than one init function;
• Each source file of a package can also have more than one init function;
• The sequence of execution of the INIT function within the same package is not explicitly defined;
• The init function of different packages determines the order of initialization according to the dependency relationship of the package import;
• The init function cannot be called, but is called automatically before the main function is called.

Therefore, before the main function is executed, Docker has defined many flag parameters and initialized many of the flag parameters. The command line flag parameters defined are: Flversion, Fldaemon, Fldebug, Flsocketgroup, Flenablecors, Fltls, Fltlsverify, Flca, Flcert, Flkey, and so on.

The following specific analysis Fldaemon:

• Definition: Fldaemon = flag. Bool ([]string{"D", "-daemon"}, False, "Enable daemon Mode")
• Type of Fldaemon is type bool
• The Fldaemon name is called "D" or "-daemon", and the name appears in the Docker command
• The default value of Fldaemon is False
• Fldaemon's help information for "Enable daemon mode"
• When accessing the value of Fldaemon, use the pointer * Fldaemon to dereference the access

When parsing the command-line flag parameter, the following languages are valid:

• -D,--daemon
• -d=true,--daemon=true
• -d= "true",--daemon= "true"
• -d= ' true ',--daemon= ' true '

When parsing to the first non-defined flag parameter, the command line flag parameter parsing is finished. For example, when executing Docker commands Docker--daemon=false--version=false PS, the flag parameter parsing mainly accomplishes two tasks:

• Complete the command line flag parameter parsing, named-daemon and-version flag parameter Fldaemon and flversion respectively obtained corresponding value, are false;
• When the parameter PS of the first non-flag parameter is encountered, the PS and all subsequent parameters are stored in flag. Args () for use when performing a specific request to the Docker client.

For an in-depth study of flag resolution, please refer to the source command line parameter flag.

3.2. Handling flag information and collecting configuration information for Docker client

With the relevant knowledge of the flag parameter parsing, the main function of Docker analysis is much easier to understand. In summary, the flag information processed in the source code is listed first, and the configuration information for the Docker client is collected, followed by one by one for this analysis:

• The flag parameters processed are: flversion, Fldebug, Fldaemon, fltlsverify, and Fltls;
• The configuration information collected for the Docker client is: Protoaddrparts (obtained through the flhosts parameter to provide the Docker client's communication protocol with the server and the communication address), Tlsconfig ( Obtained through a series of flag parameters, such as *fltls, *fltlsverify, to provide secure Transport layer protocol protection.

The flag parameter information, along with configuration information, is parsed.

At flag. The code after Parse () is as follows:

  If *flversion {    showversion ()    return  }

It is not difficult to understand that, when the flag parameter is parsed, if the flversion parameter is true, call Showversion () to display the version information and exit from the main function, otherwise, continue to execute.

  If *fldebug {    os. Setenv ("DEBUG", "1")  }

If the Fldebug parameter is true, create a system environment variable named debug with the Setenv function in the OS package and set its value to "1". Continue to execute down.

  If Len (flhosts) = = 0 {    defaulthost: = os. Getenv ("Docker_host")    if defaulthost = = "" | | *fldaemon {      //If We do not has a host, default to UNIX socket      defaulthost = FMT. Sprintf ("unix://%s", API. Defaultunixsocket)    }    If _, err: = API. Validatehost (Defaulthost); Err! = Nil {      log. Fatal (Err)    }    flhosts = Append (flhosts, defaulthost)  }

The above source code mainly analyzes internal variable flhosts. The role of flhosts is to provide the Docker client with the host object to connect to, and to provide the Docker server with the objects to listen on.

In the analysis process, first determine whether the flhosts variable length is 0, if so, through the OS package gets the value named docker_host environment variable, assign it to defaulthost. If Defaulthost is empty or Fldaemon is true, there is currently no defined host object, it is set to the UNIX socket by default and the value is API. Defaultunixsocket, the constant is located at./docker/api/common.go, and the value is "/var/run/docker.sock", so Defaulthost is "unix:///var/run/ Docker.sock ". After verifying the legitimacy of the defaulthost, append the value of Defaulthost to the end of Flhost. Continue to execute down.

  If *fldaemon {    Maindaemon ()    return  }

If the Fldaemon parameter is true, then executes the Maindaemon function, implements the Docker daemon start, if the Maindaemon function completes, exits the main function, the general Maindaemon function does not voluntarily terminate. Since this section describes the launch of the Docker client, it is assumed that the Fldaemon parameter is false and does not execute the above block of code. Continue to execute down.

  If Len (flhosts) > 1 {    log. Fatal ("Specify only One-h")  }  protoaddrparts: = Strings. SPLITN (Flhosts[0], "://", 2)

Above, if the length of the flhosts is greater than 1, the error log is thrown. The first element in the string array is then flhosts, divided by "://", and the divided two parts are placed into the variable protoaddrparts array. The role of Protoaddrparts is to resolve the protocol and address for establishing communication with Docker server, which is one of the essential configuration information for the Docker client creation process.

  var (    CLI       *client. DOCKERCLI    Tlsconfig TLS. Config  ) Tlsconfig.insecureskipverify = True

Since you have assumed that Fldaemon is false, you can assume that the main function is run for the purpose of creating and executing the Docker client. Create two variables here: one for type is client. The DOCKERCLI pointer to the object CLI, and the other for type is TLS. The object of config tlsconfig. and set the Insecureskipverify property of the Tlsconfig to true. The Tlsconfig object is created to ensure that the CLI follows the secure Transport Layer Protocol (TLS) when transmitting data. Secure Transport Layer Protocol (TLS) is used for confidentiality and data integrity between two communication applications. Tlsconfig is an optional configuration information during the Docker client creation process.

  If we should verify the server, we need to load a trusted CA  if *fltlsverify {    *fltls = True    certpool: = X5 09.NewCertPool ()    file, err: = Ioutil. ReadFile (*FLCA)    if err! = Nil {      log. Fatalf ("couldn ' t read CA cert%s:%s", *flca, Err)    }    Certpool.appendcertsfrompem (file)    Tlsconfig.rootcas = Certpool    tlsconfig.insecureskipverify = False  }

If the fltlsverify flag parameter is true, it means that the server-side security needs to be verified, and the Tlsconfig object needs to load a trusted CA file. The path to the CA file is the value of the *FLCA parameter, and eventually the assignment of the Rootcas property in the Tlsconfig object is completed and the Insecureskipverify property is set to False.

If TLS is enabled, try to load and send client certificates  if *fltls | | *fltlsverify {    _, Errcert: = OS. Stat (*flcert)    _, Errkey: = OS. Stat (*flkey)    if Errcert = = Nil && Errkey = nil {      *fltls = True      cert, err: = TLS. Loadx509keypair (*flcert, *flkey)      if err! = Nil {        log. Fatalf ("couldn ' t load X509 key pair:%s. Key encrypted?", err)      }      tlsconfig.certificates = []tls. Certificate{cert}}}  

If one of the two flag parameters of the FLTLS and Fltlsverify is true, then a certificate that needs to be loaded and sent to the client side is indicated. Finally, the content of the certificate is given to the Tlsconfig certificates property.

At this point, the flag parameter has been fully processed and the configuration information required by the Docker client has been collected. The content is then created and executed by the Docker client.

3.3. The creation of the Docker client

The Docker client is created in the case of existing configuration parameter information, using the Newdockercli method in the client package to create an instance CLI, the source code is implemented as follows:

  If *fltls | | *fltlsverify {    CLI = client. NEWDOCKERCLI (OS. Stdin, OS. Stdout, OS. Stderr, Protoaddrparts[0], protoaddrparts[1], &tlsconfig)  } else {    CLI = client. NEWDOCKERCLI (OS. Stdin, OS. Stdout, OS. Stderr, Protoaddrparts[0], protoaddrparts[1], nil)  }

If the flag parameter Fltls is TRUE or fltlsverify is true, the TLS protocol is required to secure the transport, so the Tlsconfig parameter is passed in when the Docker client is created, otherwise the Docker client is created as well. , but Tlsconfig is nil.

The implementation of the NEWDOCKERCLI function in the client package can be described in detail in./docker/api/client/cli.go.

Func newdockercli (in IO. Readcloser, out, err io. Writer, proto, addr string, Tlsconfig *tls. Config) *dockercli {  var (    isterminal = False    terminalfd uintptr    Scheme     = "http"  )  if Tlsconfig! = Nil {    scheme = "https"  } if in! =  Nil {    If file, OK: = out. *os. File); OK {      TERMINALFD = file. Fd ()      isterminal = term. Isterminal (TERMINALFD)    }  }  if Err = = Nil {    err = out  }  return &dockercli{    Proto:      Proto,    addr:       addr, in    :         in, out    : Out        ,    err:        err,    Isterminal:isterminal,    TERMINALFD:TERMINALFD,    tlsconfig:  tlsconfig,    scheme:     scheme,  }}

Overall, the creation of DOCKERCLI objects is simple, the more important DOCKERCLI properties are proto: Transport protocol, Addr:host destination address, Tlsconfig: The configuration of the Secure Transport layer protocol. If Tlsconfig is not NULL, then the secure Transport layer protocol is required, the scheme of the Dockercli object is set to "https", and the configuration of the input, output, and error display is finally returned to the object.

By calling the NEWDOCKERCLI function, the program eventually finishes creating the Docker Client and returns to the main function to continue execution.

4. Docker command execution

Main function execution so far, the following needs to be performed for the Docker command: request parameters in the created Docker client,docker command (stored in flag after flag parsing). ARG ()). That is, you need to use Docker client to parse the request parameters in the Docker command and eventually send the appropriate request to the Docker Server.

4.1. Docker Client Parsing request command

The Docker Client resolution request command works on the first completion of the Docker command execution section, directly into the source code section after the main function:

If err: = CLI. CMD (flag. Args () ...); Err! = Nil {    If sterr, OK: = Err. ( *utils. STATUSERROR); OK {      if sterr. Status! = "" {        log. Println (Sterr. Status)      }      os. Exit (Sterr. StatusCode)    }    log. Fatal (err)  }

Consult the above source code, you can find, as previously said, the first analysis stored in flag. The specific request parameter in Args (), the function that executes is the CMD function of the CLI object. Enter the CMD function of the./docker/api/client/cli.go:

CMD executes the specified Commandfunc (CLI *dockercli) cmd (args ... string) error {  If Len (args) > 0 {    metho D, exists: = Cli.getmethod (args[0])    if!exists {      fmt. Println ("Error:command not Found:", Args[0])      return CLI. Cmdhelp (args[1:] ...)    }    Return method (args[1:] ...)  }  Return CLI. Cmdhelp (args ...)}

The code comment shows that the CMD function executes the specific instruction. In the implementation of the source code, first determine whether the length of the request parameter list is greater than 0, if not, the description does not request information, return the help information of the Docker command, if the length is greater than 0, the request information, first by requesting the first element in the parameter list args[0] To get a concrete method. If the method above does not exist, the help information for the Docker command is returned, and if so, call the concrete method, the parameter is args[1] and all subsequent request parameters.

Or take a specific Docker command as an example, Docker–daemon=false–version=false pull Name. Through the above analysis, the following procedures can be summed up:

(1) After parsing the flag parameter, the docker request parameter "pull" and "Name" are stored in flag. Args ();

(2) Create a good Docker client to execute the CLI for CLI,CLI. CMD (flag. Args () ...);

In the CMD function, the name of the method is obtained by args[0] that is "pull", executing Cli.getmethod (args[0]);

(3) in the Getmothod method, the value "Cmdpull" is obtained by processing the final return methods;

(4) Final execution method (args[1:] ...) That is Cmdpull (args[1:] ...).

4.2. Docker Client Execution Request command

The previous section, through a series of command parsing, finally found a specific way to execute the command, this section focuses on how the Docker client handles and sends requests through the execution method.

Because of the different request content, the execution process is roughly the same, this section still illustrates the process in one example: Docker pull NAME.

When the Docker client executes the above request command, it executes the cmdpull function, passing in a parameter of args[1:] .... The source code is specific to the Cmdpull function in./docker/api/client/command.go.

The following one-by-one analysis cmdpull source implementation.

(1) An object cmd of type flagset is defined by the Subcmd method in the CLI package.

CMD: = Cli. Subcmd ("Pull", "Name[:tag]", "pulling an image or a repository from the registry")

(2) A flag of type string is defined for the Cmd object, named "#t" or "#-tag", and the initial value is null.

Tag: = cmd. String ([]string{"#t", "#-tag"}, "", "Download Tagged Image in a repository")

(3) The args parameter parsing, the parsing process, first extract whether there is a tag to match the flag parameter, if any, assign it to the tag parameter, the remaining parameters are stored in cmd. Narg (); If none, all parameters are stored in cmd. Narg ().

If err: = cmd. Parse (args); Err! = Nil {return nil}

(4) Determine the parameter list after the flag parsing, if the number of parameters in the parameter list is not 1, it is necessary to pull multiple image,pull command is not supported, then call the error-handling method cmd. Usage (), and returns nil.

If Cmd. Narg ()! = 1 {cmd. Usage () return nil    }

(5) Create a variable V of the map type, which is used to hold the URL parameters required for the pull mirror, then assign the first value of the parameter list to the remote variable and add the remote as the value of the key fromimage to V; Finally, if there is a tag message, Add the tag information to V as the value of the key "tag".

var (  v      = URL. values{}  remote = cmd. ARG (0)) V.set ("FromImage", remote) if *tag = = "" {  v.set ("tag", *tag)}

(6) Resolves the host address of the mirror and the name of the mirror through the remote variable.

  Remote, _ = parsers. Parserepositorytag (remote)    //Resolve the Repository name from FQN to hostname + Name    hostname, _, Err: = Registry . Resolverepositoryname (remote)    if err! = Nil {      return err    }

(7) Obtain the authentication configuration information required to communicate with the Docker server through the CLI object.

Cli. Loadconfigfile ()    //Resolve The Auth config relevant for this server    authconfig: = Cli.configFile.ResolveAuthConfig (hostname)

(8) A function named pull is defined, and the parameter type passed in is registry. Authconfig, the return type is error. The main contents of the function execution block are: Cli.stream (...) Part. This section specifically initiates a POST request to the Docker server with a URL of "/images/create?". +v.encode (), the requested authentication information is: map[string][]string{"X-registry-auth": Registryauthheader,}.

   Pull: = Func (Authconfig registry. Authconfig) Error {      buf, err: = json. Marshal (authconfig)      if err! = Nil {        return err      }      registryauthheader: = []string{        base64. Urlencoding.encodetostring (BUF),      }      return Cli.stream ("POST", "/images/create?" +v.encode (), nil, cli.out, map[string][]string{      "  x-registry-auth": Registryauthheader,      })    }

(9) Since the previous step simply defines the pull function, this step calls the execution of the pull function, and if successful, returns, and if the error is returned, the appropriate error handling. If the return error is 401, you need to log in first, go to the login link, after completion, continue to execute the pull function, if completed will eventually return.

If err: = Pull (authconfig); Err! = Nil {  if strings. Contains (Err. Error (), "Status 401") {    fmt. Fprintln (cli.out, "\nplease login prior to pull:")    if err: = CLI. Cmdlogin (hostname); Err! = Nil {      return err    }        authconfig: = Cli.configFile.ResolveAuthConfig (hostname) return pull        ( authconfig)  }  return err}

This is the entire execution of pull requests, and the execution of other requests is similar to the process. In summary, most of the request execution process is the initial processing of the parameters in the command line about the request, and the corresponding auxiliary information is added to the Docker server to send the Docker client and Docker server agreed API requests through the specified protocol.

5. Summary

From the source point of view, this article analyzes the complete process of starting from a docker executable file to creating a Docker Client and ultimately sending it to a Docker server request.

The author believes that learning and understanding Docker client-related source code implementation, not only allows users to master the use of Docker commands, but also allows users in special circumstances have the ability to modify the Docker client's source code, so that they meet the specific needs of their system, Maximize the value of Docker's open mind to achieve the purpose of customizing the Docker client

6. References

1. Http://www.infoq.com/cn/articles/docker-command-line-quest
2. http://docs.studygolang.com/pkg/
3. http://blog.studygolang.com/2013/02/%E6%A0%87%E5%87%86%E5%BA%93-%E5%91%BD%E4%BB%A4%E8%A1%8C%E5%8F%82%E6%95%B0% e8%a7%a3%e6%9e%90flag/
4. https://docs.docker.com/reference/commandline/cli/

Docker source Analysis (ii): Docker client creation and command execution