The three---the book of his own writing Docker the Union File system Introduction

Union File System UnionFS

UnionFS is a file system service designed for the LINUX,FREEBSD and NetBSD operating systems to combine other file systems into a federated mount point. It uses branch to "transparently" Overwrite files and directories of different file systems to form a single, consistent file system. These branches are either read-only or read-write, so when you write to this virtual federated file system, the system is actually written into a new file. It seems that the virtual federated file system can operate on any file, but it does not change the original file, because UnionFS used an important management technology called write-time replication.
Write-time Replication (Copy-on-write, hereinafter referred to as Cow), also known as implicit sharing, is a resource management technique for efficient replication of modifiable resources. The idea is that if a resource is duplicated, but without any modification, there is no need to create a new resource immediately; This resource can be shared by the old and new instances. Creating a new resource occurs during the first write operation, which is when the resource is modified. This resource-sharing approach can significantly reduce the consumption of unmodified resource replication, but will also add and subtract a small amount of overhead when resource modifications are made.
With a classic example, Knoppix, a Linux distribution for Linux demos, CD-ROM teaching, and commercial product demonstrations, is to put a CD-ROM or DVD and a presence in a writable device (eg, U disk) on the file system called Knoppix.img together. So any changes to the CD/DVD file will be applied to the USB disk, do not change the original CD/DVD content. Aufs

Aufs, the English full name is advanced multi-layered unification filesystem, once also called acronym multi-layered unification Multi-layered unification filesystem. Aufs completely rewrites the early UnionFS 1.x, whose primary purpose is reliability and performance, and introduces new features, such as load balancing of writable branches. Some implementations of AUFS have been incorporated into the UnionFS 2.x version. how Docker is using Aufs

Aufs is the first type of storage drive Docker chooses. The AUFS has a fast boot container that efficiently leverages storage and memory, until now AUFS is still a storage-driven type supported by Docker. Next we'll explain how Docker uses AUFS to store images and containers. image layer and Aufs

Each Docker image is made up of a series of read-only layers. The contents of image layers are stored in the/var/lib/docker/aufs/diff directory of the Docker hosts filesystem. The/var/lib/docker/aufs/layers directory stores the metadata of how image layer stacks these layer. The
prepares an ECS with Docker 1.11.2 installed. Executes the Ls/var/lib/docker/aufs/diff command without pulling any mirrors and starting any container, discovering that the directory does not store any content. Pull the ubuntu:15.04 mirror, and then execute the ls/var/lib/docker/aufs/diff command again. We can see that the results of Docker pull show ubuntu:15.04 mirrors a total of 4 layers, and there are four corresponding storage file directories in the results of the Execute Ls/var/lib/docker/aufs/diff command. One thing to note here is that since Docker 1.10, the storage-mirrored layer folder in the diff directory is no longer the same as the mirror ID. Last cat/var/lib/docker/aufs/layers/ 6bb19cb345da470e015ba3f1ca049a1c27d2c57ebc205ec165d2ad8a44e148ea commands are listed on the stack in the 6bb19cb345da470e015ba3f1ca049a1c27d2c57ebc205 Ec165d2ad8a44e148ea layer beneath the layers.

$ docker Pull ubuntu:15.04 15.04:pulling from Library/ubuntu 9502adfba7f1:pull complete 4332ffb06e4b:pull complete 2f93 7cc07b5f:pull Complete A3ed95caeb02:pull Complete digest:sha256 : 2fb27e433b3ecccea2a14e794875b086711f5d49953ef173d8a03e8707f1510f status:downloaded Newer image for ubuntu:15.04 $ ls  /var/lib/docker/aufs/diff 208319b22189a2c3841bc4a4ef0df9f9238a3e832dc403133fb8ad4a6c22b01b
9c444e426a4a0aa3ad8ff162dd7bcd4dcbb2e55bdec268b24666171904c17573  6bb19cb345da470e015ba3f1ca049a1c27d2c57ebc205ec165d2ad8a44e148ea
F193107618deb441376a54901bc9115f30473c1ec792b7fb3e73a98119e2cf77 $ ls/var/lib/docker/aufs/mnt  208319b22189a2c3841bc4a4ef0df9f9238a3e832dc403133fb8ad4a6c22b01b
9c444e426a4a0aa3ad8ff162dd7bcd4dcbb2e55bdec268b24666171904c17573  6bb19cb345da470e015ba3f1ca049a1c27d2c57ebc205ec165d2ad8a44e148ea F193107618deb441376a54901bc9115f30473c1ec792b7fb3e73a98119e2cf77 $ cat/var/lib/docker/aufs/layers/ 6bb19cb345da470e015ba3f1ca049a1c27d2c57ebc205ec165d2ad8a44e148ea 9C444E426A4a0aa3ad8ff162dd7bcd4dcbb2e55bdec268b24666171904c17573
F193107618deb441376a54901bc9115f30473c1ec792b7fb3e73a98119e2cf77
 208319b22189a2c3841bc4a4ef0df9f9238a3e832dc403133fb8ad4a6c22b01b

Next, we will create a mirror named Changed-ubuntu, based on the ubuntu:15.04 mirror image. This mirror only adds a file in the mirrored/TMP folder that writes "Hello World". You can use the following dockerfile to implement:

From ubuntu:15.04

 RUN Echo ' Hello World ' >/tmp/newfile

In the terminal CD to the Dockerfile location above, execute Docker build-t changed-ubuntu. command to Build mirror.

$docker build-t Changed-ubuntu.
Sending Docker daemon 10.75 KB step
1:from ubuntu:15.04
 ---> d1b55fd07600 step
2:run E Cho ' Hello world ' >/tmp/newfile
 ---> Running in c72100f81dd1
 ---> 9d8602c9aee1
removing Intermediate container c72100f81dd1
successfully built 9d8602c9aee1

Then perform Docker images view the current mirror, and you can see the newly generated changed-ubuntu.

$docker images
REPOSITORY          TAG                 IMAGE ID            CREATED              SIZE
changed-ubuntu      latest              9d8602c9aee1 about        a minute ago   131.3 MB
ubuntu              15.04               d1b55fd07600-        months ago        131.3 MB

Using the Docker History changed-ubuntu command, you can clearly see which image layers is used by the Changed-ubuntu mirror. You can see from the output that the 9D8602C9AEE1 image layer is at the top, only 12B in size, created by the/bin/sh-c echo "Hello World" >/tmp/newfile command. That is, Changed-ubuntu mirroring takes up only 12Bytes of disk space, which proves how efficiently disk space is used by AUFS. The following four-layer image layers is a shared 4 image layers that makes up the ubuntu:15.04 mirror image. The layers of the "missing" tag is that after Docker 1.10, the image history data of a mirrored image layers is stored in a file, which is a normal behavior that Docker officials think.

 $docker history Changed-ubuntu IMAGE CREATED by SIZE COMMENT 9d8602c9aee1 4 minutes ago/bin/sh-c echo "Hello World" &G T /tmp/newfile b d1b55fd07600 months ago/bin/sh-c # (NOP) CMD ["/bin/bash"] 0 B <missi  Ng> months ago/bin/sh-c sed-i ' s/^#\s*\ (deb.*universe\) $/1.879 KB <missing> 10 Months ago/bin/sh-c echo ' #!/bin/sh ' >/usr/sbin/polic 701 B <missing> months ago/ Bin/sh-c # (NOP) ADD file:3f4708cf445dc1b537 131.3 MB 

Next we continue to look at the layers storage information, from the output we can see the/var/lib/docker/aufs/diff directory and/var/lib/docker/aufs/ The MNT directory has one more folder 9f122dbaa103338f27bac146326af38a2bcb52f98ebb3530cac828573faa3c4e. When using cat/var/lib/docker/aufs/layers/ 9f122dbaa103338f27bac146326af38a2bcb52f98ebb3530cac828573faa3c4e command to view its metadata, you can see that the layers in front of it is Ubuntu : 15.04 The 4 image layers used by the mirror. Further probing the/var/lib/docker/aufs/diff/9f122dbaa103338f27bac146326af38a2bcb52f98ebb3530cac828573faa3c4e folder and discovering that it stores one/ Tmp/newfile file, only one line in the file, "Hello world." At this point, we have fully analyzed how image layer and Aufs are shared by files and folders for mirrored storage.

$ Ls/var/lib/docker/aufs/diff 208319b22189a2c3841bc4a4ef0df9f9238a3e832dc403133fb8ad4a6c22b01b  9c444e426a4a0aa3ad8ff162dd7bcd4dcbb2e55bdec268b24666171904c17573
F193107618deb441376a54901bc9115f30473c1ec792b7fb3e73a98119e2cf77  6bb19cb345da470e015ba3f1ca049a1c27d2c57ebc205ec165d2ad8a44e148ea
9F122DBAA103338F27BAC146326AF38A2BCB52F98EBB3530CAC828573FAA3C4E $ ls/var/lib/docker/aufs/mnt  208319b22189a2c3841bc4a4ef0df9f9238a3e832dc403133fb8ad4a6c22b01b  9c444e426a4a0aa3ad8ff162dd7bcd4dcbb2e55bdec268b24666171904c17573
F193107618deb441376a54901bc9115f30473c1ec792b7fb3e73a98119e2cf77  6bb19cb345da470e015ba3f1ca049a1c27d2c57ebc205ec165d2ad8a44e148ea 9F122DBAA103338F27BAC146326AF38A2BCB52F98EBB3530CAC828573FAA3C4E $ cat/var/lib/docker/aufs/layers/
9f122dbaa103338f27bac146326af38a2bcb52f98ebb3530cac828573faa3c4e
6bb19cb345da470e015ba3f1ca049a1c27d2c57ebc205ec165d2ad8a44e148ea
9c444e426a4a0aa3ad8ff162dd7bcd4dcbb2e55bdec268b24666171904c17573 F193107618deb441376a54901bc9115f30473c1ec792b7fb3e73A98119e2cf77 208319b22189a2c3841bc4a4ef0df9f9238a3e832dc403133fb8ad4a6c22b01b $ cat/var/lib/docker/aufs/diff/ 9f122dbaa103338f27bac146326af38a2bcb52f98ebb3530cac828573faa3c4e/tmp/newfile Hello World

container layer and Aufs

Docker uses Aufs's cow technology to achieve image layer sharing and reduce disk space consumption. Cow means that once a file has only a small portion of it changed, AUFS also needs to copy the entire file. This design can have a certain impact on the performance of the container, especially if the files to be copied are large, or are located beneath many image layers, or aufs require a deep search of the directory structure tree. But do not worry too much, for a container, each image layer only need to copy at most once. Subsequent changes will be made on the first copy of the container layer.
When a container is started, Docker creates a read-only init layer for it to store content related to the environment in the container, and Docker creates a read-write for it to perform all writes.
Container layer's mount directory is also/var/lib/docker/aufs/mnt. Container's metadata and configuration files are stored in the/var/lib/docker/containers/directory. The container read-write layer is stored in the/var/lib/docker/aufs/diff/directory. Even if the container stops, the read-write layer still exists, so the restart container will not lose data, and the read-write layer will be deleted only when a container is deleted.
Then we still use the experiment to prove the above conclusion. First, the number of existing containers is 0, and no data is found in the/var/lib/docker/containers directory. Finally, look at the Aufs mount for the next system, with only one config file.

$ docker ps-a
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES

$ ls/var/lib/ Docker/containers

$ ls/sys/fs/aufs/

Start a Changed-ubuntu container.

$docker run-dit changed-ubuntu Bash
fb5939d878bb0521008d63eb06adea75e6af275855f11879dfa3992dfdaa5e3f

$ Docker ps-a
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
FB5939D878BB        changed-ubuntu      "bash"              seconds ago up      seconds                           amazing_babbage

Looking at the/var/lib/docker/aufs/diff directory found that there are two more folders below, F9CCF5CAA9B7324F0EF112750CAA14203B557D276CA08C78C23A42A949E2BFC8 is the read-write layer that Docker creates for the container, F9ccf5caa9b7324f0ef112750caa14203b557d276ca08c78c23a42a949e2bfc8-init is the init layer of the read-only created by Docker for the container.

$ Ls/var/lib/docker/aufs/diff
208319b22189a2c3841bc4a4ef0df9f9238a3e832dc403133fb8ad4a6c22b01b  9f122dbaa103338f27bac146326af38a2bcb52f98ebb3530cac828573faa3c4e  F9ccf5caa9b7324f0ef112750caa14203b557d276ca08c78c23a42a949e2bfc8-init
6bb19cb345da470e015ba3f1ca049a1c27d2c57ebc205ec165d2ad8a44e148ea  F193107618deb441376a54901bc9115f30473c1ec792b7fb3e73a98119e2cf77
9c444e426a4a0aa3ad8ff162dd7bcd4dcbb2e55bdec268b24666171904c17573  F9ccf5caa9b7324f0ef112750caa14203b557d276ca08c78c23a42a949e2bfc8

/VAR/LIB/DOCKER/AUFS/MNT directory changes are consistent with/var/lib/docker/aufs/diff.

$ ls/var/lib/docker/aufs/mnt
208319b22189a2c3841bc4a4ef0df9f9238a3e832dc403133fb8ad4a6c22b01b  9f122dbaa103338f27bac146326af38a2bcb52f98ebb3530cac828573faa3c4e  F9ccf5caa9b7324f0ef112750caa14203b557d276ca08c78c23a42a949e2bfc8-init
6bb19cb345da470e015ba3f1ca049a1c27d2c57ebc205ec165d2ad8a44e148ea  F193107618deb441376a54901bc9115f30473c1ec792b7fb3e73a98119e2cf77
9c444e426a4a0aa3ad8ff162dd7bcd4dcbb2e55bdec268b24666171904c17573  F9ccf5caa9b7324f0ef112750caa14203b557d276ca08c78c23a42a949e2bfc8

The/var/lib/docker/aufs/layers/directory has more files with the same name as the two file directories above, and the Cat command allows you to see clearly the records that rely on layer.

$ls/var/lib/docker/aufs/layers 208319b22189a2c3841bc4a4ef0df9f9238a3e832dc403133fb8ad4a6c22b01b  9f122dbaa103338f27bac146326af38a2bcb52f98ebb3530cac828573faa3c4e
F9ccf5caa9b7324f0ef112750caa14203b557d276ca08c78c23a42a949e2bfc8-init  6bb19cb345da470e015ba3f1ca049a1c27d2c57ebc205ec165d2ad8a44e148ea
F193107618deb441376a54901bc9115f30473c1ec792b7fb3e73a98119e2cf77  9c444e426a4a0aa3ad8ff162dd7bcd4dcbb2e55bdec268b24666171904c17573 F9ccf5caa9b7324f0ef112750caa14203b557d276ca08c78c23a42a949e2bfc8 $ cat/var/lib/docker/aufs/layers/
F9ccf5caa9b7324f0ef112750caa14203b557d276ca08c78c23a42a949e2bfc8
F9ccf5caa9b7324f0ef112750caa14203b557d276ca08c78c23a42a949e2bfc8-init
9f122dbaa103338f27bac146326af38a2bcb52f98ebb3530cac828573faa3c4e
6bb19cb345da470e015ba3f1ca049a1c27d2c57ebc205ec165d2ad8a44e148ea
9c444e426a4a0aa3ad8ff162dd7bcd4dcbb2e55bdec268b24666171904c17573
F193107618deb441376a54901bc9115f30473c1ec792b7fb3e73a98119e2cf77 208319B22189A2C3841BC4A4EF0DF9F9238A3E832DC403133FB8AD4A6C22B01B $ cat/vAr/lib/docker/aufs/layers/f9ccf5caa9b7324f0ef112750caa14203b557d276ca08c78c23a42a949e2bfc8-init
9f122dbaa103338f27bac146326af38a2bcb52f98ebb3530cac828573faa3c4e
6bb19cb345da470e015ba3f1ca049a1c27d2c57ebc205ec165d2ad8a44e148ea
9c444e426a4a0aa3ad8ff162dd7bcd4dcbb2e55bdec268b24666171904c17573
F193107618deb441376a54901bc9115f30473c1ec792b7fb3e73a98119e2cf77 208319b22189a2c3841bc4a4ef0df9f9238a3e832dc403133fb8ad4a6c22b01b

Create a new folder in the/var/lib/docker/containers/directory that is the same as Containerid, holding the metadata and config files for the container.

$ ls/var/lib/docker/containers/
fb5939d878bb0521008d63eb06adea75e6af275855f11879dfa3992dfdaa5e3f

$ ls/ var/lib/docker/containers/fb5939d878bb0521008d63eb06adea75e6af275855f11879dfa3992dfdaa5e3f/
Config.v2.json  fb5939d878bb0521008d63eb06adea75e6af275855f11879dfa3992dfdaa5e3f-json.log  Hostconfig.json  Hostname  hosts  resolv.conf  resolv.conf.hash  SHM

Next we look at the mount from the system Aufs and a si_fe6d5733e85e4904 folder in the/sys/fs/aufs/directory. We can see clearly from the cat/sys/fs/aufs/si_fe6d5733e85e4904/* command that this is the layer permission of the container we just started, Only the top F9CCF5CAA9B7324F0EF112750CAA14203B557D276CA08C78C23A42A949E2BFC8 layer is the Read-write permission.

 $s/sys/fs/aufs/config si_fe6d5733e85e4904 $ cat/sys/fs/aufs/si_fe6d5733e85e4904/*/var/ lib/docker/aufs/diff/f9ccf5caa9b7324f0ef112750caa14203b557d276ca08c78c23a42a949e2bfc8=rw/var/lib/docker/aufs/ diff/f9ccf5caa9b7324f0ef112750caa14203b557d276ca08c78c23a42a949e2bfc8-init=ro+wh/var/lib/docker/aufs/diff/ 9f122dbaa103338f27bac146326af38a2bcb52f98ebb3530cac828573faa3c4e=ro+wh/var/lib/docker/aufs/diff/ 6bb19cb345da470e015ba3f1ca049a1c27d2c57ebc205ec165d2ad8a44e148ea=ro+wh/var/lib/docker/aufs/diff/ 9c444e426a4a0aa3ad8ff162dd7bcd4dcbb2e55bdec268b24666171904c17573=ro+wh/var/lib/docker/aufs/diff/ f193107618deb441376a54901bc9115f30473c1ec792b7fb3e73a98119e2cf77=ro+wh/var/lib/docker/aufs/diff/ 208319b22189a2c3841bc4a4ef0df9f9238a3e832dc403133fb8ad4a6c22b01b=ro+wh the 70/run/shm/aufs.xino of the "

Finally, mention the Aufs how to delete a file for container. If you delete File1,aufs, a. wh.file1 file is generated at the Read-write layer of container to hide read-only files from all file1 layers. At this point, we have clearly described and validated how Docker uses AUFS to manage container layers. Write your own aufs .

Let's start by using simple commands to create a Aufs file system, and feel how to use aufs and cow to implement file management.
First create a Aufs folder under your lab directory, and then create a mnt folder in the Aufs directory to mount points. Next, create a folder called Container-layer in the Aufs directory, which has a file named Container-layer.txt, which contains the I am container layer. Similarly, continue to create 4 folders named Image-layer N in the Aufs directory (n values are 1 4, respectively), which has a folder named Image−layer {n} (n takes a value of 1~4) with a name of Image-layer{n}. TXT file, the file content is I am image layer${n}. Check the contents of the file using the following command:

$ cd/home/qinyujia/aufs

$ ls
container-layer  image-layer1  image-layer2  image-layer3  Image-layer4  mnt

$ cat container-layer.txt
I am container layer

$ cat image-layer1/image-layer1.txt< C11/>i am Image Layer 1

$cat image-layer2/image-layer2.txt
I am Image Layer 2

$ cat Image-layer3/image-lay Er3.txt
I am image Layer 3

$ cat image-layer4/image-layer4.txt
I am Image Layer 4

The file directory to be Federated is ready, and then we will use the Container-layer and 4 folders named Image-layer${n} in a aufs way to mount the MNT directory we just created. In the Mount Aufs command, we do not specify permissions for the 5 folders to be mounted, and the default behavior is that the first directory dirs specified on the left is the Read-write permission, followed by the read-only permissions.

$ sudo mount-t aufs-o dirs=./container-layer:./image-layer4:./image-layer3:./image-layer2:./image-layer1 none./mnt< c0/>$ Tree mnt
mnt
├──container-layer.txt
├──image-layer1.txt
├──image-layer2.txt
├── Image-layer3.txt
└──image-layer4.txt

Do you remember that we used to view the read and write permissions of the files in the system Aufs directory? Here we still use the cat/sys/fs/aufs/si_fe6d5733e85e5904/* command to confirm the permissions for each directory in the newly-mount file system. (Note that si_fe6d5733e85e5904 should be a newly created system for this mnt mount point, not the one mentioned in Docker and Aufs), which we can see clearly from the output, Only the Container-layer folder is Read-write, and the rest are read-only permissions.

$ cat/sys/fs/aufs/si_fe6d5733e85e5904/*
/home/qinyujia/aufs/container-layer=rw
/home/qinyujia/aufs/ Image-layer4=ro
/home/qinyujia/aufs/image-layer3=ro
/home/qinyujia/aufs/image-layer2=ro
/home/ Qinyujia/aufs/image-layer1=ro
/home/qinyujia/aufs/container-layer/. Aufs.xino

Here's an interesting operation to add a line of text "write to Mnt ' Image-layer1.txt" at the end of the Mnt/image-layer1.txt file. According to the cow technology we introduced above, we can guess what kind of behavior it will produce.

$ ECHO-E "\nwrite to Mnt ' s image-layer1.txt" >>/mnt/image-layer4.txt

We used the cat command to view the contents of the Mnt/image-layer4.txt file and found that the content really changed from "I am Image Layer 4" to
"I am Image Layer 4

Write to mnt ' s image-layer1.txt ' because MNT is just a dummy mount point because we continue to find out where the file is modified.

$ cat./mnt/image-layer4.txt
I am Image Layer 4

write to mnt ' s image-layer1.txt

We looked at the contents of the Image-layer4/image-layer4.txt file and found that it did not change.

$ cat Image-layer4/image-layer4.txt
I am Image Layer 4

Next, when we check the Container-layer folder, we find a file called Image-layer4.txt, the contents of the file is
"I am Image Layer 4

Write to Mnt ' s image-layer1.txt. Which is when we try to mnt/ Image-layer4.txt file for write operation, the system first in the MNT directory to find the file named Image-layer4.txt, copy it to the Read-write level Container-layer directory, and then the Container-layer directory of I Mage-layer4.txt file for write operation. To this end, we successfully completed a small demo, the implementation of their own Aufs file system.

$ ls container-layer/
container-layer.txt  image-layer4.txt

$cat container-layer/image-layer4.txt
I am Image Layer 4

write to