Understanding Docker's multi-stage mirroring build

This is a creation in Article, where the information may have evolved or changed.

Docker technology has been in existence for more than 4 years since 2013. For developers who have embraced and used Docker technology in their daily development work, building Docker images is a commonplace. But does this mean that Docker's image-building mechanism is relatively perfect? No, Docker is still continuously optimizing the image building mechanism. No, from the release of the Docker 17.05 release this year, Docker began supporting the multi-stage build of container mirroring (multi-stage build).

What is a mirrored multi-stage build? Just give the concept definition too abrupt, here first sell a Xiaoguanzi, we first from the daily development used in the image of the construction of the method and the image of the problems encountered in the construction of the problem.

First, homogeneous image construction

A common scenario in which we do image building is when the application is compiled directly on the developer's own developer or server, and the compiled binaries are then penetrated into the image. This situation generally requires that the build environment is compatible with the base image used by the image, such as: I compiled the app on Ubuntu 14.04 and applied the app to an image based on the Ubuntu series base image. This build is what I call "homogeneous image building" because the application's build environment is compatible with the environment in which it is deployed: the applications I compiled under Ubuntu 14.04 can be basically seamlessly based on the ubuntu:14.04 and later versions of base image images (for example: 16.04 , 16.10, 17.10, etc.), but in a non-fully compatible base image, such as in CentOS, it may fail.

1, homogeneous image construction example

Here is an example of the construction of a homogeneous image (subsequent chapters are based on this example), note: Our compilation environment is Ubuntu 16.04 x86_64 virtual machine, Go 1.8.3 and Docker 17.09.0-ce.

We use one of the most common HTTP servers in the Go language as an example:

// github.com/bigwhite/experiments/multi_stage_image_build/isomorphism/httpserver.gopackage mainimport (        "net/http"        "log"        "fmt")func home(w http.ResponseWriter, req *http.Request) {        w.Write([]byte("Welcome to this website!\n"))}func main() {        http.HandleFunc("/", home)        fmt.Println("Webserver start")        fmt.Println("  -> listen on port:1111")        err := http.ListenAndServe(":1111", nil)        if err != nil {                log.Fatal("ListenAndServe:", err)        }}

Compile this program:

# go build -o myhttpserver httpserver.go# ./myhttpserverWebserver start  -> listen on port:1111

This example looks simple and does not have a few lines of code, but behind the go net/http package does a lot of things at the bottom, including a lot of system calls that reflect the "coupling" of the application to the operating system, which is reflected in subsequent explanations. Next we'll build a docker image for this program, and start a Myhttpserver container based on this image. We chose ubuntu:14.04 as base image:

Github.com/bigwhite/experiments/multi_stage_image_build/isomorphism/dockerfilefrom Ubuntu:14.04COPY./ Myhttpserver/root/myhttpserverrun chmod +x/root/myhttpserverworkdir/rootentrypoint ["/root/myhttpserver"] Execution Build: # Docker build-t Myrepo/myhttpserver:latest. Sending build context to Docker daemon 5.894MBStep 1/5: from ubuntu:14.04---> Dea1945146b9step 2/5: COPY./myhttpse Rver/root/myhttpserver---> 993e5129c081step 3/5: RUN chmod +x/root/myhttpserver---> Running in 104d84838ab2--- > ebaeca006490removing Intermediate Container 104d84838ab2step 4/5: workdir/root---> 7afdc2356149removing interm Ediate container 450ccfb09ffdstep 5/5: Entrypoint/root/myhttpserver---> Running in 3182766e2a68---> 77f315e15f14 Removing intermediate container 3182766e2a68successfully built 77f315e15f14successfully tagged myrepo/myhttpserver: latest# Docker imagesrepository TAG IMAGE ID CREATED sizemyrepo/myhttpse RVer Latest 77f315e15f14 seconds ago 200mb# Docker run Myrepo/myhttpserverwebserver start-listen On port:1111

The above is the most basic image build method.

Next, we may encounter the following requirements:
* Setting up a Go program's build environment can be time-consuming, especially for go applications that rely on many third-party open-source packages, which can take a long time to download. We'd better pack all these things into a builder image for the Go program build;
* We see that the size of the myrepo/myhttpserver image we built above is 200MB, which seems a bit too "big". Although Docker on each host node has the ability to cache image layer, we still want to build a leaner and shorter image.

2. With Golang Builder image

The Docker Hub offers an official Golang image repository with Go dev environment, and we can use this Golang builder image directly to help build our app image For some go applications that rely more on third-party packages, we can also customize our own dedicated builder image with this Golang image as the base image.

We build our Golang-builder image based on golang:latest, the base image, and we write a dockerfile.build for build Golang-builder Image:

// github.com/bigwhite/experiments/multi_stage_image_build/isomorphism/Dockerfile.buildFROM golang:latestWORKDIR /go/srcCOPY httpserver.go .RUN go build -o myhttpserver ./httpserver.go

Build Golang-builder image under the same directory:

# docker build -t myrepo/golang-builder:latest -f Dockerfile.build .Sending build context to Docker daemon  5.895MBStep 1/4 : FROM golang:latest ---> 1a34fad76b34Step 2/4 : WORKDIR /go/src ---> 2361824677d3Removing intermediate container 01d8f4e9f0c4Step 3/4 : COPY httpserver.go . ---> 1ff14bb0bc56Step 4/4 : RUN go build -o myhttpserver ./httpserver.go ---> Running in 37a1b76b7b9e ---> 2ac5347bb923Removing intermediate container 37a1b76b7b9eSuccessfully built 2ac5347bb923Successfully tagged myrepo/golang-builder:latestREPOSITORY              TAG                 IMAGE ID            CREATED             SIZEmyrepo/golang-builder   latest              2ac5347bb923        3 minutes ago       739MB

Next, we build our final app image based on the myhttpserver that has been built in Golang-builder:

# docker create --name appsource myrepo/golang-builder:latest# docker cp appsource:/go/src/myhttpserver ./# docker rm -f appsource# docker rmi myrepo/golang-builder:latest# docker build -t myrepo/myhttpserver:latest .

The logic of this command is to copy the completed Myhttpserver to the host's current directory from the container appsource based on golang-builder image and delete the temporary container Appsource and the Golang-builder image built above; the final step, like the first example, builds the final image based on the Myhttpserver already built in the local directory. For convenience, you can also put this series of commands into a makefile.

3. Use a smaller size alpine image

Builder image does not help us for the final application of the image "weight loss", the size of Myhttpserver image still stays at 200MB. To "Lose weight", we need a smaller base image, we chose Alpine. Alpine image has a size less than 4M, plus the size of the app, and the size of the final app image can be reduced to less than 20M.

In conjunction with builder image, we only need to change the base image of Dockerfile to Alpine:latest:

// github.com/bigwhite/experiments/multi_stage_image_build/isomorphism/Dockerfile.alpineFrom alpine:latestCOPY ./myhttpserver /root/myhttpserverRUN chmod +x /root/myhttpserverWORKDIR /rootENTRYPOINT ["/root/myhttpserver"]

To build the version Alpine app Image:

# docker build -t myrepo/myhttpserver-alpine:latest -f Dockerfile.alpine .Sending build context to Docker daemon  6.151MBStep 1/5 : FROM alpine:latest ---> 053cde6e8953Step 2/5 : COPY ./myhttpserver /root/myhttpserver ---> ca0527a62d39Step 3/5 : RUN chmod +x /root/myhttpserver ---> Running in 28d0a8a577b2 ---> a3833af97b5eRemoving intermediate container 28d0a8a577b2Step 4/5 : WORKDIR /root ---> 667345b78570Removing intermediate container fa59883e9fdbStep 5/5 : ENTRYPOINT /root/myhttpserver ---> Running in adcb5b976ca3 ---> 582fa2aedc64Removing intermediate container adcb5b976ca3Successfully built 582fa2aedc64Successfully tagged myrepo/myhttpserver-alpine:latest# docker imagesREPOSITORY                   TAG                 IMAGE ID            CREATED             SIZEmyrepo/myhttpserver-alpine   latest              582fa2aedc64        4 minutes ago       16.3MB

16.3mb,size did come down! We start a container based on the image to see if the app is running with any problems:

# docker run myrepo/myhttpserver-alpine:lateststandard_init_linux.go:185: exec user process caused "no such file or directory"

The container failed to start! Why is it? Because Alpine image is not an isomorphic image of the Ubuntu environment. We'll explain in detail below.

Second, the heterogeneous image construction

Our image Builder:myrepo/golang-builder:latest is based on the golang:latest image. Golang base image has two templates: Dockerfile-debain.template and Dockerfile-alpine.template. Golang:latest is based on the Debian template and is compatible with Ubuntu. The Myhttpserver for dynamically shared link libraries is as follows:

 # ldd myhttpserver    linux-vdso.so.1 =>  (0x00007ffd0c355000)    libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007ffa8b36f000)    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007ffa8afa5000)    /lib64/ld-linux-x86-64.so.2 (0x000055605ea5d000)

The Linux distribution of the Debian department uses glibc. But Alpine is different, Alpine uses the implementation of MUSL libc, so when we run the above container, the loader fails to exit because it cannot find myhttpserver dependent libc.so.6.

This build environment is incompatible with the operating environment, which I call "heterogeneous image building". So how do we solve this problem? We continue to look at:

1. Static construction

In mainstream programming languages, go is one of the most portable, especially after go 1.5, go to runtime in the C code is used to rewrite, the dependence on libc has been minimized, but there are still some feature to provide two versions of the implementation: C implementation and go implementation. And by default, in the case of Cgo_enabled=1, both the program and the precompiled standard library adopt the C implementation. A detailed discussion of this can be found in the article "Also on portability of Go", which I wrote earlier. Therefore, the use of different libc implementation of the Debian system and the Alpine system is incompatible with the nature of the situation. To solve this problem, we first consider a static build of the Go program and then put the statically built go app into Alpine image.

Let's modify dockerfile.build and add cgo_enabled=0 when compiling the go source file:

// github.com/bigwhite/experiments/multi_stage_image_build/heterogeneous/Dockerfile.buildFROM golang:latestWORKDIR /go/srcCOPY httpserver.go .RUN CGO_ENABLED=0 go build -o myhttpserver ./httpserver.go

Build this builder Image:

# docker build -t myrepo/golang-static-builder:latest -f Dockerfile.build .Sending build context to Docker daemon  4.096kBStep 1/4 : FROM golang:latest ---> 1a34fad76b34Step 2/4 : WORKDIR /go/src ---> 593cd9692019Removing intermediate container ee005d487ad5Step 3/4 : COPY httpserver.go . ---> a095eb69e716Step 4/4 : RUN CGO_ENABLED=0 go build -o myhttpserver ./httpserver.go ---> Running in d9f3b3a6c36c ---> c06fe8dccbadRemoving intermediate container d9f3b3a6c36cSuccessfully built c06fe8dccbadSuccessfully tagged myrepo/golang-static-builder:latest# docker imagesREPOSITORY                     TAG                 IMAGE ID            CREATED             SIZEmyrepo/golang-static-builder   latest              c06fe8dccbad        31 seconds ago      739MB

Next, we build our final app image based on a statically connected myhttpserver that's already built in Golang-static-builder:

# docker create --name appsource myrepo/golang-static-builder:latest# docker cp appsource:/go/src/myhttpserver ./# ldd myhttpserver    not a dynamic executable# docker rm -f appsource# docker rmi myrepo/golang-static-builder:latest# docker build -t myrepo/myhttpserver-alpine:latest -f Dockerfile.alpine .

Run the new Image:

# docker run myrepo/myhttpserver-alpine:latestWebserver start  -> listen on port:1111

Note: We can use strace to prove that the go is only using Go's own runtime implementation, but not the code in LIBC.A:

# CGO_ENABLED=0 strace -f go build httpserver.go 2>&1 | grep open | grep -o '/.*\.a'  > go-static-build-strace-file-open.txt

Open the Go-static-build-strace-file-open.txt file to view the contents of the file, you will not find libc.a this file (under Ubuntu, General LIBC.A lying/usr/lib/x86_64-linux-gnu/ below), this shows that go build does not attempt to go to the open libc.a file and get the symbolic definition in it at all.

2. Using Alpine Golang Builder

Our Go app runs in the container of Alpine based, and we can use Alpine Golang Builder to build our app (no static link required). As mentioned earlier, Golang has alpine templates:

REPOSITORY                   TAG                 IMAGE ID            CREATED             SIZEgolang                       alpine              9e3f14138abd        7 days ago          269MB

The Dockerfile content of Alpine version Golang Builder is as follows:

//github.com/bigwhite/experiments/multi_stage_image_build/heterogeneous/Dockerfile.alpine.buildFROM golang:alpineWORKDIR /go/srcCOPY httpserver.go .RUN go build -o myhttpserver ./httpserver.go

The follow-up operation is not the same as the previous Golang Builder: Build our app with Alpine Golang builder and break it into alpine image, not here.

Three, multi-stage image construction: enhance the developer experience

Before Docker 17.05, we were all building mirrors like the one above. You will find that even with the heterogeneous Image Builder pattern, we maintain two dockerfile, and also perform some other things like copy application from container, clean build container and build outside of the Docker build command. Image, and so on. The Docker community saw the problem and implemented the multi-stage mirroring build mechanism (multi-stage).

Let's take a look at the dockerfile used by the multi-stage build for the example above:

//github.com/bigwhite/experiments/multi_stage_image_build/multi_stages/DockerfileFROM golang:alpine as builderWORKDIR /go/srcCOPY httpserver.go .RUN go build -o myhttpserver ./httpserver.goFrom alpine:latestWORKDIR /root/COPY --from=builder /go/src/myhttpserver .RUN chmod +x /root/myhttpserverENTRYPOINT ["/root/myhttpserver"]

After reading this dockerfile content, your first bright is not to merge the previous two dockerfile together, each dockerfile alone as a "stage"! This is the case, but this Docker also has some new grammatical forms that are used to establish the link between the various "stages". For such a dockerfile, we should know the following points:

• The dockerfile that supports multi-stage build establishes an intrinsic connection between the previous build phases, allowing the latter phase to be built using the artifacts from the previous phase to form a chain of the build phase;
• The end result of the multi-stages build produces only one image, avoiding the redundancy of multiple temporary images or temporary container objects, which is exactly what we need: we only have the result.

Let's use multi-stage to build the above example:

# docker Build-t Myrepo/myhttserver-multi-stage:latest. Sending build context to Docker daemon 3.072kBStep 1/9: from Golang:alpine as builder---> 9e3f14138abdstep 2/9: WOR KDIR/GO/SRC---> Using cache---> 7a99431d1be6step 3/9: COPY httpserver.go. ---> 43a196658e09step 4/9: RUN go build-o myhttpserver/httpserver.go---> Running in 9e7b46f68e88---> 90dc73 912803Removing Intermediate Container 9e7b46f68e88step 5/9: from alpine:latest---> 053cde6e8953step 6/9: Workdir/ro ot/---> Using cache---> 30d95027ee6astep 7/9: COPY--from=builder/go/src/myhttpserver. ---> F1620b64c1bastep 8/9: RUN chmod +x/root/myhttpserver---> Running in e62809993a22---> 6be6c28f5fd6removin G Intermediate Container e62809993a22step 9/9: Entrypoint/root/myhttpserver---> Running in e4000d1dde3d---> 639c Ec396c96removing Intermediate container e4000d1dde3dsuccessfully built 639cec396c96successfully tagged myrepo/ myhttserver-multi-stage:latest# Docker IMagesrepository TAG IMAGE ID CREATED sizemyrepo/myhttserver-m Ulti-stage latest 639cec396c96 about an hour ago 16.3MB

Let's run this image:

# docker run myrepo/myhttserver-multi-stage:latestWebserver start  -> listen on port:1111

Iv. Summary

The multi-stage mirroring build allows developers to build smaller images in a single, dockerfile, and easy way, with a good experience and easier access to automated systems such as CI/CD. However, the current multi-stage build can only be supported in Docker 17.05 and later versions. If you want to learn and practice this function, but there is no environment, you can use the experimental environment provided by Play-with-docker.

Play with Docker Labs

All of the above sample code can be downloaded here.

Weibo: @tonybai_cn
Public Number: Iamtonybai
Github.com:https://github.com/bigwhite

, Bigwhite. All rights reserved.

Related Posts:

1. Kuberize Ceph RBD API Service
2. Graceful exit of service program in Docker container
3. Explore the support of Docker containers for shared memory
4. Discussion on methods of modifying system variables in Docker container
5. Kubernetes cluster pod uses host's local time zone setting