Docker Storage Driver Detailed introduction _java

Docker Storage Driver Detailed introduction

Recently do project, during the Docker storage driver will not, so find information on the Internet, and resolved, here on the record.

Objective

1. Understand how Docker is stored
2. Docker image and container directory structure on host
3. Docker image and container contents are stored differently from the configuration
   

Docker is an open source application container engine, which mainly utilizes the Linux kernel namespace to implement sandbox isolation and cgroup resource constraints. Docker is a lightweight Linux container for unified development and deployment, trying to solve the "dependency hell" problem, combining dependent services and components, similar to containers used by ships, for rapid installation deployments.

1. Basic architecture of Docker-client and daemon

Let us first understand the basic structure of Docker and start-up process, in fact, Docker adopted the C/s architecture, including client and server. Docker Daemon accepts requests from customers as a server and processes those requests (creating, running, submitting containers). The client and the server are on a machine, communicating through the restful API. Specific to the use of the process, that is, after the implementation of the service Docker start, the host to generate Docker Deamon Daemon, running in the background and waiting to receive messages from the client (that is, the input of the Docker command, such as Docker pull xxx , Docker run ..., docker commit xxx), and realize the interaction with Docker Deamon. When you start the Docker service, you can see the Docker process.

Default

[Root@localhost ~]# Ps-aux | grep Docker
root  11701 0.0 0.4 359208 16624?  SSL 21:05 0:00/usr/bin/docker-d-H fd://--selinux-enabled--insecure-registry 186.100.8.216:5000
root  11861 0.0 0.0 113004 2256 pts/0 s+ 23:01 0:00 grep--color=auto Docker

Description of this, mainly after the designated file system, the need to first in the/etc/sysconfig/docker configuration specific storage driver (this will write a dedicated blog), and then start Docker Daemon, It cannot be manipulated by the parameters of the Run command. It can also be set directly on the host command line via Docker–d.

2. Docker storage mode-storage Driver

The core of the Docker model is the effective use of layered mirroring mechanism, mirroring can be inherited through layering, based on the underlying image (without the parent mirror), can produce a variety of specific application mirrors. Different Docker containers can share some of the basic file system layer, coupled with their own unique change layer, greatly improve the efficiency of storage. The main mechanism is the layered model and the different directories to be mounted under the same virtual file system (unite several directories into the a single fictitious filesystem, from this article). Several different storage drivers are used for mirrored storage Docker, including: Aufs,devicemapper,btrfs and Overlay (from the official website). Here's a simple introduction to different storage drivers.

Aufs

Aufs (ANOTHERUNIONFS) is a federated file system. AUFS supports the ability to set read-only (readonly), read-write (ReadWrite), and write (whiteout-able) permissions for each member directory (such as a branch of Git), while the AUFS has a concept that resembles layering. A branch of read-only permission can be modified logically (without affecting the read-only portion). Aufs the only storage driver can implement and share runtime libraries between containers, so when you run hundreds of them with the same program code or runtime, Aufs is a pretty good choice.

Device Mapper

Device Mapper is a mapping framework mechanism from logical devices to physical devices provided in the Linux 2.6 kernel, under which users can easily develop management strategies for storage resources according to their needs (see). Device Mapper driver will create a 100G simple file containing your mirrors and containers, each of which is limited to 10G volumes (note: This is a sparse file created automatically using loopback, specifically/var/lib/docker/ The data and metadata under Devicemapper/devicemapper can be dynamically expanded). Can adjust the size of the Docker container, specific reference) you can start Docker daemon with the parameter-s to specify driver, that is, you can docker-d-s Devicemapper set Docker storage driver. Turn off the Docker service first and execute the command:

Default

[Root@localhost ~]# docker-d-S devicemapper
info[0000] +job serveapi (unix:///var/run/docker.sock) 
INFO[0000 ] Listening for HTTP on Unix (/var/run/docker.sock) 
info[0000] +job init_networkdriver ()     
info[0000]-job init_ Networkdriver () = OK (0)   
info[0000] Loading containers:start.     
....
INFO[0000] Loading containers:done.     
INFO[0000] Docker daemon:1.4.0 4595d4f/1.4.0; execdriver:native-0.2; Graphdriver:devicemapper 
info[0000] +job acceptconnections ()      
info[0000]-job acceptconnections () = OK (0)

In addition, Docker can specify –storage-opt parameters when starting the container, but now only Devicemapper can accept parameter settings. Later there will be targeted blog display.

BTRFS

Btrfs Driver can be very efficient in Docker builds. But as Devicemapper does not support shared storage between devices (to participate in the official website). Btrfs supports the creation of snapshots (snapshot) and clones (clone), as well as the ability to manage multiple physical devices conveniently. (For more information, refer to IBM's introduction to Btrfs)

Overlay

Overlay is similar to AUFS, but performance is better than AUFS, has good memory utilization, has now merged into the Linux kernel 3.18. Specific use of the command: Docker–d–s overlay

Official website Note:it is currently unsupported on btrfs or no Copy on Write filesystem and should the only being used over EXT4 .

3 Docker directory structure

The two most important concepts of Docker are mirrors and containers. So where are the pull of our mirrors? Where are the contents of our operations modifications stored when the mirrored run container is started? Because the specific drive is different, so the final implementation effect is different. Below we take device Mapper storage driver as an example to analyze the storage structure of the lower Docker.

1. Enter the/var/lib/docker directory, listing the contents:

Default

[Root@localhost ~]# cd/var/lib/docker/
[root@localhost docker]# ls
containers devicemapper execdriver graph Init linkgraph.db repositories-devicemapper TMP Trust volumes

According to the contents of the catalogue, it is obvious to see that the Devicemapper driver is used.

Note: The folders shown below are all under/var/lib/docker.

2. Which folder does the mirrored file of pull exist under? Reference

Pull's mirrored information is saved in the Graph folder, and the mirrored content exists under Devicemapper/devicemapper/data.

3. Where is the start-up container running?

The container configuration information that was started is stored in the containers, and the execdriver/native/is checked.

The contents of the operation in the container are kept under Devicemapper/devicemapper/data.

4. Graph's Role

acts as custodian of downloaded container mirrors in the Docker schema, and a record of relationships between downloaded container mirrors. Graph's local directory, for each container mirror, the information that is stored is: The container's mirrored metadata (JSON), the container mirror size (layersize) information, and the specific rootfs that the container mirror represents.

5. Experiment Test:

-Initial no start container:

Default

[Root@localhost docker]# ll containers/Total
0

-Start a container:

Default

[Root@localhost docker]# Docker run-i-T--rm centos:7/bin/bash
[root@187a8f9d2865/]#

The uuid=187a8f9d2865 of the container being started

-Before starting the container, view the actual size of the file under/var/lib/docker/devicemapper/devicemapper/

Default

[root@bhdocker216 docker]# du-h devicemapper/devicemapper/*
2.1G devicemapper/devicemapper/data
3.5M Devicemapper/devicemapper/metadata

-View on host hosts

Default

[root@bhdocker216 docker]# ls containers/
187a8f9d2865c2ac***91b981

View the contents of the startup container under the UUID folder:

Default

[root@bhdocker216 containers]# ll 187a8f9d2865c2ac***91b981 total
-rw-------. 1 root 273 Mar 5 23:59 187a8 F9d2865***-json.log
-rw-r--r--. 1 root 1683 Mar 5 23:58 config.json
-rw-r--r--. 1 root 334 Mar 5 23:58 H Ostconfig.json
-rw-r--r--. 1 root Mar 5 23:58 hostname
-rw-r--r--. 1 root 174 Mar 5 23:58 hosts
- Rw-r--r--. 1 root 5 23:58 resolv.conf
-Add files to the startup container and view.

Create a file in the running container first:

Default

[root@8a1e3ad05d9e/]# dd if=/dev/zero of=floppy.img bs=512 count=5760 5760+0
Records in
5760+0 Records out< c13/>2949120 bytes (2.9 MB) copied, 0.0126794, 233 MB/s

Then view the files under/var/lib/docker/devicemapper/devicemapper/:

Default

[root@bhdocker216 docker]# du-h devicemapper/devicemapper/*
5.5G devicemapper/devicemapper/data
4.6M Devicemapper/devicemapper/metadata

The size of this place is a bit out of the box because the # dd If=/dev/zero of=test.txt bs=1m count=8000 was first executed, creating a 8G size file that was too slow for me to terminate, but it was clear to see the operation in the running container, Two folders have changed (increased).

-View graph, in the case of only pull a mirror (Ubuntu14.10), where there are 7 long UUID named directory, this is how to come?

With the Docker Images–tree listing the mirrored tree structure, we can see the mirrored hierarchical storage structure. The final Ubuntu (7th layer) is based on the 6th layer changes, that is, in this logical tree the nth layer is based on the n-1 layer changes, n-layer dependent on the image of the n-1 layer. The No. 0 layer, size 0, is called base image.

-What is the contents of the Graph/uuid directory?

Default

[Root@localhost graph]# ll 01bf15a18638145eb***-H total
8.0K
-rw-------. 1 root 1.6K Mar 5 18:02 JSON-
R W-------. 1 Root 9 Mar 5 18:02 layersize

View the contents of the Layersize: the size of the number representation layer (in units: B). JOSN: Save this mirrored metadata (such as: Size,architecture,config,container,**parent uuid**, etc.).

-View the Devicemapper/devicemapper folder

There are two folders data and metadata, in fact device mapper driver is the image and container files are stored in the **data** file. You can view the size of data and metadata by Docker info. You can also view the actual size of these two sparse files with du–h (which is useful above).

-Execdriver

Default

[root@bhdocker216 docker]# ls execdriver/native/
8A1E3AD05D9E66A455E683A2C***2437BDCCCDFDFA
//To view the contents inside:
[root@bhdocker216 8a1e3ad05d9e66a455e***]# ls
container.json State.json

-Volumes

The volumes without the-v parameter is empty, tested if the boot container is added with the-v argument, a UUID is displayed under the Volumes folder and a global search is performed in the host, which is found only under volumes and is not related to the mirror and the container's UUID.

Default

[root@bhdocker216 docker]# Find/-name 86eb77f9f5e25676f100***d5a
/var/lib/docker/volumes/86eb77f9f5e25676f100 D5A
//view contents:
[root@bhdocker216 volumes]# ls 86eb77f9f5e25676f100***d5a
config.json
[ root@bhdocker216 volumes]# cat 86eb77f9f5e25676f100***d5a/config.json 
{"ID": " 86EB77F9F5E25676F100A89BA727BC15185303236AAE0DCF4C17223E37651D5A "," Path ":"/home/data "," Isbindmount ": true," Writable ": true}

Table Description of Folder action

Make a summary, organize a table,/var/lib/docker under the different folders under the description of the role:

Thank you for reading, I hope to help you, thank you for your support for this site!