Docker configuration Btrfs and device mapper storage driver settings

Docker configuration Btrfs and device mapper storage driver settings

Operating Environment:

Docker-v

Docker version 1.12.1, build 23cf638

Uname-a

Linux ceph-6-29 3.10.0-327.28.3.el7.x86_64 #1 SMP Thu 19:05:49 UTC x86_64 x86_64 x86_64 gnu/linux\

One: The first time the Docker installation is complete, the default storage device is the loop loopback device, which creates a 100G for storing data, and a 2G sparse file for storing the metadata, which is then appended to the loopback block devices/dev/loop0 and/DEV/LOOP1, respectively. The thin pool is then created based on the loopback block device.

1.1: View Loopback block Device information

[Email protected] ~]# losetup-a

/DEV/LOOP0: [2245]:1937889 (/var/lib/docker/devicemapper/devicemapper/data)

/DEV/LOOP1: [2245]:1937890 (/var/lib/docker/devicemapper/devicemapper/metadata)

1.2: View loopback device Size

[[email protected] ~]# Docker info

.。。。。。。。。。。。。。。。

Server version:1.12.1

Storage Driver:devicemapper

Pool Name:docker-8:197-537316283-pool

Pool blocksize:65.54 KB

Base Device size:10.74 GB=============data default block size

backing FILESYSTEM:XFS

Data File:/dev/loop0

Metadata File:/dev/loop1

Data Space used:11.8 MB

Data Space total:107.4 GB

Data Space available:107.4 GB

Metadata Space used:581.6 KB

Metadata Space total:2.147 GB"=============metadata default block size

Metadata Space available:2.147 GB

Thin Pool Minimum Free space:10.74 GB

Udev Sync Supported:true

From the above information can be seen, Base Device only 10G space, we found during the test, repeated release of the image, it is easy to put the space stained, if you want to increase the space to perform operations as follows:

1.3: Close Docker

Systemctl Stop Docker

1.4: Delete Docker directory files, do not delete, but sometimes may cause the Docker Service service to start.

rm-rf/var/lib/docker/*

1.5: Adjust the size of loopback device files and mirrors, for example, we set back the environment device file size to 100G, the metadata file size is 4G, the base image size is 20G:

In addition, –storage-opt has other parameters, comparing Dm.fs specified file system (default is EXT4), etc.

Vim/usr/lib/systemd/system/docker.service

Add the boot entry content as follows:

Execstart=/usr/bin/dockerd--storage-opt dm.basesize=100g dm.loopdatasize=50g--storage-opt Dm.loopmetadatasize=4G

1.6: Start Docker service

Systemctl Daemon-reload

Systemctl Start Docker

1.7: Verify Docker configuration information in rough:

Docker info

.。。。。。。。。。。。。。。

Server version:1.12.1

Storage Driver:devicemapper

Pool Name:docker-8:197-537315670-pool

Pool blocksize:65.54 KB

Base device size:107.4 "=============base device default block size

backing FILESYSTEM:XFS

Data File:/dev/loop0

Metadata File:/dev/loop1

Data Space used:53.74 MB=============data default block size

Data Space total:53.69 GB

Data Space available:53.63 GB

Metadata Space used:532.5 KB

Metadata Space total:4.295 "=============metadata default block size

Metadata Space available:4.294 GB

Thin Pool Minimum Free space:5.369 GB

1.8: The above is directly modify the configuration to adjust the size of the storage space, verify that the configuration changes take effect;

Second: the creation of storage space by DD file mode to adjust storage space;

2.1: Verify Docker configuration information online:

Docker info

.。。。。。。。。。。。。。。

Server version:1.12.1

Storage Driver:devicemapper

Pool Name:docker-8:197-537315670-pool

Pool blocksize:65.54 KB

Base device size:107.4 "=============base device default block size

backing FILESYSTEM:XFS

Data File:/dev/loop0

Metadata File:/dev/loop1

Data Space used:53.74 MB=============data default block size

Data Space total:53.69 GB

Data Space available:53.63 GB

Metadata Space used:532.5 KB

Metadata Space total:4.295 "=============metadata default block size

Metadata Space available:4.294 GB

Thin Pool Minimum Free space:5.369 GB

1.3: Close Docker

Systemctl Stop Docker

1.4: Delete Docker directory files, do not delete, but sometimes may cause the Docker Service service to start.

rm-rf/var/lib/docker/*

1.5: Create a storage space file

Create a Directory

Mkdir-p/var/lib/docker/devicemapper/devicemapper

Cd/var/lib/docker/devicemapper/devicemapper

Create Your pool:

DD If=/dev/zero of=/var/lib/docker/devicemapper/devicemapper/data bs=1g count=0 seek=80

DD If=/dev/zero of=/var/lib/docker/devicemapper/devicemapper/metadata bs=4k count=0 seek=1024000

1.6: Verify Storage space size

[Email protected] devicemapper]# Ls-lsh

Total 0

0-rw-r--r--1 root root 80G Sep 21:35 data

0-rw-r--r--1 root root 4.0G Sep 21:36 metadata

1.7: View Startup service profile:

Vim/usr/lib/systemd/system/docker.service

The Startup item content is default:

Execstart=/usr/bin/dockerd

1.8: Start Docker service

Systemctl Daemon-reload

Systemctl Start Docker

1.9: Verify the configuration:

[[email protected] devicemapper]# Docker info

containers:0

running:0

paused:0

stopped:0

images:0

Server version:1.12.1

Storage Driver:devicemapper

Pool Name:docker-8:197-537315670-pool

Pool blocksize:65.54 KB

Base device size:10.74 "=============base device default block size

backing FILESYSTEM:XFS

Data File:/dev/loop0

Metadata File:/dev/loop1

Data Space used:11.8 MB=============data default block size

Data Space total:107.4 GB

Data Space available:107.4 GB

Metadata Space used:704.5 KB

Metadata Space total:4.194 "============= Metadata default block size

Metadata Space available:4.194 GB

Thin Pool Minimum Free space:10.74 GB

Udev Sync Supported:true

2.0: Verify view info information, the spatial information has not changed, this configuration, this seems not very reliable, above is in the foreign query information when you see someone do this, for reference only:

The above two ways of demonstration, basic to Docker storage, but not recommended in the production environment use, use loopback equipment, performance is not said, is not too stable, always out of some wonderful problems, we suffer from it, all find data total comb below two kinds of configuration Docker The use of storage space has now been tested in production systems to verify reliability.

The following may involve a lot of storage-related knowledge, please inquire about the relevant information, this article does not elaborate, currently, Docker supports AUFS, Btrfs, Device Mapper, OVERLAYFS, ZFS Five kinds of storage drivers. As the Docker official says, there is no single driver for all scenarios, and choosing the right storage driver for different scenarios can effectively improve Docker performance.

AUFS: The earliest Docker supported the storage boot, the patch to join the kernel, the problem is much more! The AUFS can transparently cover one or more of the existing filesystem's layered file systems, merging multiple layers into a single layer representation of the file system.

Devicemapper: Copy-on-write (CoW). Cow is copy-on-write, which means to copy only when it is necessary to write, this is a modified scenario for existing files.

The Btrfs:btrfs driver, Docker build is very fast,-but the devicemapper device does not share executable memory.

OVERLAYFS: is a very fast trade union file system. It now merges in the main Linux kernel as 3.18.0.

Specific reference information is as follows:

https://docs.docker.com/engine/reference/commandline/dockerd/

http://dockone.io/article/1513

The following mainly describes how to quickly and easily configure the use of Btrfs and Devicemapper Drive;

Three: Directly store Docker data using bare devices, linking the real disk to the data stored on Docker data.

3.1: Configuration requirements,

A: The system needs to have an unused disk or partition;

B: Master the partition common commands

3.2: View system partition information:

# LSBLK

NAME maj:min RM SIZE RO TYPE mountpoint

SDA 8:0 0 558.4G 0 disk

。。。。。。。。。。。。。。。。。。

├─SDA4 8:4 0 1K 0 part

├─sda5 8:5 0 355.9G 0 part/dbdata

├─sda6 8:6 0 80G 0 Part ======= This demo uses partitioning

。。。。。。。。。。。。

3.3: View info information before modifying:

[[email protected] ~]# Docker info

。。。。。。。。。。。。。。。。。。。。。

Images:5

Server version:1.12.1

Storage Driver:devicemapper

Pool Name:docker-8:2-269225707-pool

Pool blocksize:65.54 KB

Base Device size:10.74 GB

backing FILESYSTEM:XFS

Data File:/dev/loop0

Metadata File:/dev/loop1

Data Space used:1.944 GB

Data Space total:107.4 GB

Data Space available:98.49 GB

Metadata Space used:2.834 MB

Metadata Space total:2.147 GB

Metadata Space available:2.145 GB

Thin Pool Minimum Free space:10.74 GB

3.4: Close Docker

Systemctl Stop Docker

3.5: Delete Docker directory files, do not delete, but sometimes may cause the Docker Service service to start.

rm-rf/var/lib/docker/*

3.6: System partition;

What I'm going to do is use Fdisk to delete the/dev/sda6 partition and divide the remaining space reproduction into two partitions, with the following information:

dm.datadev=/dev/sda6 ==>70g

dm.metadatadev=/dev/sda7 ==>10g

3.7: Add the boot entry content as follows:

Vim/usr/lib/systemd/system/docker.service

Modify the startup entry as follows:

Execstart=/usr/bin/dockerd--storage-opt dm.datadev=/dev/sda6--storage-opt dm.metadatadev=/dev/sda7--storage-opt dm.blocksize=512k--storage-opt dm.basesize=80g

3.8: Start Docker service

Systemctl Daemon-reload

Systemctl Start Docker

3.9: Verify the effect of the modification:

[[email protected] docker]# Docker info

。。。。。。。。。。。。。。。。

images:0

Server version:1.12.1

Storage Driver:devicemapper

Pool Name:docker-8:2-403544979-pool

Pool blocksize:524.3 KB

Base Device size:85.9 GB"============= storage location changed

backing FILESYSTEM:XFS

Data file:/dev/sda6"============= storage location changed

Metadata file:/dev/sda7"============= storage location changed

Data Space used:79.17 MB

Data Space total:85.9 GB

Data Space available:85.82 GB

Metadata Space used:737.3 KB

Metadata Space total:10.74 GB

Metadata Space available:10.74 GB

Thin Pool Minimum Free space:8.59 GB

With the above configuration, Docker will be able to store data to bare devices, the principle of speed should be good, this way the disadvantage is that the use of disk space is not good monitoring. Some people use LVM to do this and then configure it, and then add the configuration

Dockerd--storage-opt dm.min_free_space=10% to ensure the use of space, I tested the direct use of bare devices, this parameter after the Docker can not start. I think it's a bit cumbersome to add a layer of LVM to the middle, and that's not the way to do it.

Four: Configuration using Btrfs file system;

This way configuration is relatively simple, file system mounted system after the Docker data is written to the system after the mount point, the system monitoring can monitor the disk space, can realize the alarm.

4.1: Configuration requirements,

A: The system needs to have an unused disk or partition;

B: Master the partition common commands

4.2: Pre-Modified Docker info information is as follows:

Pool Name:docker-8:197-270105280-pool

Pool blocksize:65.54 KB

Base Device size:10.74 GB

backing FILESYSTEM:XFS

Data File:/dev/loop0

Metadata File:/dev/loop1

Data Space used:768.3 MB

Data Space total:107.4 GB

Data Space available:106.6 GB

Metadata Space used:1.79 MB

Metadata Space total:2.147 GB

Metadata Space available:2.146 GB

Thin Pool Minimum Free space:10.74 GB

Udev Sync Supported:true

4.3: View system partition information:

SDM 8:192 0 278.9G 0 disk

├.......................

├─SDM2 8:194 0 80G 0 Part ======= This demo uses partitioning

......................

[Email protected] ~]#

4.4: Format File system

Install the base package

Yum Install btrfs-*

Format/DEV/SDM2 as a Btrfs file system:

Mkfs.btrfs-f/DEV/SDM2

4.5: View the partition UUID to prepare for mounting:

Blkid

/dev/sdm1:uuid= "4f00e697-43a9-4088-9f47-363238ddd36c" type= "XFS"

/dev/sdm2:uuid= "15b78ce6-5ddd-42e3-a3e0-fd226e3dbd4a" uuid_sub= "66494e3f-4708-4376-a416-c769bfa96651" TYPE= " Btrfs

4.5: Close Docker

Systemctl Stop Docker

4.6: Delete docker directory files, do not delete, but sometimes may cause the Docker Service service to start.

rm-rf/var/lib/docker/*

4.7: Add system boot Auto mount:

Vim/etc/fstab

Uuid=15b78ce6-5ddd-42e3-a3e0-fd226e3dbd4a/var/lib/docker btrfs Defaults 0 0

After saving exit:

Executive Mount-a

To view the system partition Mount condition:

[Email protected] ~]# df-h

Filesystem Size used Avail use% mounted on

..............................

/DEV/SDM2 80G 2.4G 77G 3%/var/lib/docker

4.8: Add the boot entry content as follows:

Btrfs configuration options: Btrfs.min_space, the remaining space is less than this value, the system will give the alarm information;

Vim/usr/lib/systemd/system/docker.service

Execstart=/usr/bin/dockerd-s btrfs-g/var/lib/docker--storage-opt btrfs.min_space=10g

4.9: Start Docker Service

Systemctl Daemon-reload

Systemctl Start Docker

4.10: Configuration Information Verification:

[[email protected] ~]# Docker info

................

Server version:1.12.1

Storage Driver:btrfs <================ Storage Information

Build Version:btrfs v3.19.1

Library version:101

Logging Driver:json-file

Cgroup DRIVER:CGROUPFS

Plugins:

Volume:local

Network:host Bridge NULL Overlay

Swarm:inactive

Runtimes:runc

Default Runtime:runc

The above 3-4 steps describe the configuration of Btrfs and device mapper storage driver settings, the shortcomings of which you crossing.

Reference Documentation:

Docker five storage driving principles and application scenarios and performance test comparison: http://dockone.io/article/1513

Persistent storage mode for Docker containers: http://dockone.io/article/1283

Http://vitan.github.io/docker/2016/01/22/aufs-vs-devicemapper.html

Http://www.cnblogs.com/itcomputer/p/5667917.html

Https://github.com/snitm/docker/tree/master/daemon/graphdriver/devmapper

This article from "Kang Jianhua" blog, reproduced please contact the author!

Docker configuration Btrfs and device mapper storage driver settings