"Docker Combat" dockerfile multi-stage construction principle

After the Docker 17.05 release, a new Dockerfile multi-stage build was added. The so-called multi-stage build, in effect, allows multiple from directives to appear in a dockerfile. What is the point of doing this?

Why does the old version of Docker not support multiple from directives

Prior to the 17.05 release of Docker, only one from instruction was allowed in the Dockerfile, which had to be said from the nature of the image.

In the Docker concept brief, we mention that you can simply understand that the Docker image is a compressed file that contains the program you need and a filesystem. In fact, this is not rigorous, Docker image is not just a file, but a bunch of files, the most important file is the layer .

In Dockerfile, most directives generate a layer, such as the two examples below:

# example one, foo image of Dockerfile

# There are already several layers in the base image

From ubuntu:16.04

# The Run command adds a layer where Git software is installed

RUN apt-get update \

&& apt-get install-y--no-install-recommends git \

&& Apt-get clean \

&& rm-rf/var/lib/apt/lists/*

# example two, bar mirror Dockerfile

from Foo

# The Run command will add a layer in which Nginx is installed

RUN apt-get update \

&& apt-get install-y--no-install-recommends nginx \

&& Apt-get clean \

&& rm-rf/var/lib/apt/lists/*

Assuming that the underlying image ubuntu:16.04 already exists on Layer 5 and is packaged into a mirrored foo using the first Dockerfile, Foo has 6 layers and is packaged as a mirrored bar with a second dockerfile, and the bar has 7 layers.

If other images such as ubuntu:16.04 are not counted, how many layers are saved in the system if there are only Foo and bar two mirrors in the system?

is 7 stories, not 13, because Foo and bar share 6 layers. Layer sharing mechanism can save a lot of disk space and transmission bandwidth, such as your local already have Foo image, and pull the bar mirror from the Mirror warehouse, pull only the last layer of the local not to be able to, do not need to pull the entire bar mirror strands. But how is layer sharing implemented?

It turns out that each layer of the Docker image only records file changes, and when the container starts, Docker calculates each layer of the mirror and finally generates a filesystem, called a federated mount. If you are interested in this, you can get to know AUFS.

The various layers of Docker are relevant, and in the process of joint mounting, the system needs to know on what basis to add new files. Then this requires that a Docker image can have only one starting layer and only one root. So, in Dockerfile, only one from instruction is allowed. Because multiple from directives can cause multiple roots, they are not possible. But why did the Docker 17.05 release allow Dockerfile to support multiple from instructions, or did IT support multiple roots?

The meaning of multiple from directives

Multiple from directives are not intended to generate multiple root layer relationships, and the last generated image is still based on the last from, and the previous from is discarded, so what is the meaning of the previous from?

Each from instruction is a build phase, multiple from is a multi-stage build, although the last image generated can only be the result of the last phase, but it is possible to copy the files from the predecessor phase to the back stage, which is the greatest significance of multi-stage construction.

The biggest use scenario is to separate the build environment from the running environment, for example, before we need to build a Go language program, then we need to use the Go command and other compile environment, our dockerfile may be this:

# Go language Environment base image

From golang:1.10.3

# Copy the source to the mirror

COPY server.go/build/

# Specify working directory

Workdir/build

# When compiling the image, run go build to compile the build server program

RUN cgo_enabled=0 goos=linux goarch=amd64 goarm=6 go build-ldflags '-w-s '-o server

# Specify container run-time Portal program server

entrypoint ["/build/server"]

The underlying image golang:1.10.3 is huge because it contains all the go language compiler tools and libraries, and when run we just need to compile the server program on the line, do not need compile-time compiler tool, the final generation of a large volume image is a waste.

The solution to using the impulse cloud is to separate the program compilation from the mirroring package, use the build service of the pulse cloud, choose the Add construct Go language build tool, and compile in the build step.

Finally, the compilation interface is copied to the image, then the Dockerfile base image does not need to include the GO compilation environment:

# no need to go language compilation environment

From scratch

# Copy the compilation results into the container

COPY Server/server

# Specify container run-time Portal program server

entrypoint ["/server"]

Tip: Scratch is a built-in keyword, not a real-world image. From scratch will use a completely clean file system that does not contain any files. Because the go language compiles without running, you do not need to install any runtime libraries. The from scratch can minimize the last generated image, which contains only the server program.

After the Docker 17.05 release, there is a new solution that can be solved directly by a dockerfile:

# compilation phase

From golang:1.10.3

COPY server.go/build/

Workdir/build

RUN cgo_enabled=0 goos=linux goarch=amd64 goarm=6 go build-ldflags '-w-s '-o server

# Run Phase

From scratch

# Copy compilation results from the compilation phase to the current mirror

COPY--from=0/build/server/

entrypoint ["/server"]

The mystery of this Dockerfile lies in the--from=0 parameter of the copy instruction, which is the first stage when copying a file from the front stage to the current stage with multiple from statements. In addition to using numbers, we can also give the stage a name, such as:

# Compile stage named builder

From Golang:1.10.3as Builder

# ... Omitted

# Run Phase

From scratch

# Copy compilation results from the compilation phase to the current mirror

COPY--from=builder/build/server/

Even more powerful, copy--from can be copied not only from the predecessor phase, but also directly from an existing image. Like what

From ubuntu:16.04

COPY--from=quay.io/coreos/etcd:v3.3.9/usr/local/bin/etcd/usr/local/bin/

We directly copy the ETCD image of the program into our image, so that in the generation of our program image, we do not need to compile the source code ETCD, directly to the official compilation of the program Files to come.

Some programs do not have apt source, either the version of Apt source is too old, or simply provide the source code to compile themselves, when using these programs, we can easily use the existing Docker image as our base image. However, our software may sometimes need to rely on multiple such files, we can not simultaneously use both nginx and ETCD mirror as our base image (does not support Dogan), in this case, using COPY--from is very convenient and practical.