Docker Device Mapper using DIRECT-LVM

First, Device MAPPER:LOOP-LVM

Default CentOS7 the device Mapper devices used by Docker use the loopback device by default, and the backend is the automatically generated sparse file, as follows:

# ls -lsh /var/lib/docker/devicemapper/devicemapper/总用量 510M508M -rw-------. 1 root root 100G 10月 30 00:00 data1.9M -rw-------. 1 root root 2.0G 10月 30 00:00 metadata

data [hold data] and metadata [hold metadata] size from the output you can see that the initialization defaults to 100G and 2G size, are sparse files, the use of how much occupied.

During the initialization process, Docker creates two sparse files, data and metadata, attached to the loopback device, and /dev/loop0 /dev/loop1 then creates the thin pool based on the loopback device. Default one container maximum storage data not exceeding 10g[Note: The default size after Docker 1.8 is already 100G, and it is recommended that the production environment be monitored for container pool size. ], if you need to adjust then you need to modify the /etc/sysconfig/docker configuration file to add relevant options to --storage-opt adjust (refer to man Docker for details on STORAGE DRIVER options specification).

# Docker Infocontainers:2Images:13Storage Driver:devicemapper Pool name:docker-253:1-100673362-pool Pool Blocksize:65.54 KB backing FILESYSTEM:XFS data file:/dev/loop0 Metadata file:/dev/loop1 Data Space used:533.5 MB Data Space Total:107.4 GB Data Space Available:40.75 GB Metadata Space used:1.221 MB Metadata Space Total:2.147 GB Metadata Space Available:2.146 GB Udev Sync Supported:True Data Loop file:/var/lib/docker/devicemapper/devicemapper/data Metadata loop file:/var/lib/docker/devicemapper/ Devicemapper/metadata ...# lsblk ... loop07:00100G0 loop└─docker-253:1-100673362-pool252:00100G0 dm├─docker-253:1-100673362-61f1302169c719e4f671942d6158bba061a0b5081c98d40e8ca9749f1a521ca4252:1010G0 dm└─docker-253:1-100673362-79c4340c3e06584d4e3630ad4a9b3a768066a52b0a04c9cb7bffa0b45bec8747252:2010G0 DMLOOP17:102G0 loop└─docker-253:1-100673362-pool252:0 0  100G 0 dm├─docker-253:1-100673362- 61F1302169C719E4F671942D6158BBA061A0B5081C98D40E8CA9749F1A521CA4  252: 1 0 10g 0 Dm└─docker-253:1-100673362-79c4340c3e06584d4e3630ad4a9b3a768066a52b0a04c9cb7bffa0b45bec8747  252:2 0 10g 0 DM ...             

Second, Device MAPPER:DIRECT-LVM

DIRECT-LVM is also using LVM, device mapper the DM-THINP kernel module, using DIRECT-LVM no longer using loopback devices, directly using raw partitions (no filesystem), in medium load and high density environments Will have a better performance advantage, in addition to the official is not recommended production environment system using the default LVM thin pool, as for the two types of specific performance comparison can refer to comprehensive overview of Storage Scalability in Docker this Article.

CentOS7 from docker-1.6.2-14.el7.centos.x86_64.rpm the beginning docker-storage-setup to provide tools to facilitate the configuration of DIRECT-LVM, so it is recommended to use the tool configuration. Of course, you can manually configure the LVM, add the relevant configuration options, but the process is more cumbersome, as to how to manually configure the author did not try, but this article attached to the relevant deployment of reference articles, there is a need to see, here only the use of docker-storage-setup configuration DIRECT-LVM.

Sample configuration file location /usr/lib/docker-storage-setup/docker-storage-setup , you can view a detailed description of the relevant configuration in it, or by man docker-storage-setup getting help, here are a few key options:

• Data_size=40%free
  • Defines the size of the creation DATA thin pool, which defaults to 40% of VG
• min_data_size=2g
  • Define DATA pool minimum, default is 2G, if VG is less than 2G creation fails
• chunk_size=512k
  • Define the CHUNK size of the thin pool, default 512k
• Auto_extend_pool=yes
  • Defines whether to automatically expand the thin pool size by default for auto-expansion
• Pool_autoextend_threshold=60
  • Defines the percentage of auto-expansion, which defaults to 60% for the current pool, 100 for disable, and a minimum of 50LVMTHIN-LVM thin provisioning
• Pool_autoextend_percent=20
  • Define the size of each expansion, the default is 20%, that is, the current pool size is 100G, then automatically expand 20G, the size of the expansion after 120G, 100 means disable

The actual configuration of the auto-expansion after Docker-storage-setup is set in the /etc/lvm/profile/ directory:

Because the LVM volume group being tested is named DOCKER-VG, Docker-storage-setup automatically generates the configuration file as/etc/lvm/profile/docker-vg--docker-pool-extend.profile

# cat /etc/lvm/profile/docker-vg--docker-pool-extend.profileactivation {        thin_pool_autoextend_threshold=60        thin_pool_autoextend_percent=20}

Use verifiable lvs -o+seg_monitor monitoring status.

Example

The host adds a piece of equipment first, and in this case the device is named to /dev/vdc add a docker-storage-setup configuration file:

# SystemctlStop Docker # Stops the currently running Docker# cat/etc/sysconfig/docker-Storage-setupDEVS=/DEV/VDC # A quoted, space-separated list of devices Span class= "Hljs-keyword" >to be used. If you add more than one device, separate Devs=vg=docker-vg # the volume group to use for Docker storage. setup_lvm_thin_pool=yes# docker-< Span class= "Hljs-keyword" >storage-setup # Perform setup operation, the related LVM will automatically create        span>                

After the creation is complete, you can view the relevant configuration

# cat /etc/sysconfig/docker-storage     # 配置已经自动修改DOCKER_STORAGE_OPTIONS=-s devicemapper --storage-opt dm.fs=xfs --storage-opt dm.thinpooldev=/dev/mapper/docker--vg-docker--pool# grep ‘sysconfig/docker-storage‘ /usr/lib/systemd/system/docker.serviceEnvironmentFile=-/etc/sysconfig/docker-storage # 可以看出 docker 启动会自动加载该配置文件

Delete source data and start Docker

# cat /etc/sysconfig/docker-storage-setup# DEVS=/dev/vdc     # 注释该行VG=docker-vgSETUP_LVM_THIN_POOL=yes# rm -rf /var/lib/docker# systemctl restart lvm2-monitor # 确保 lvm2-monitor 服务运行# systemctl start docker

# Docker Infocontainers:39Images:98Storage Driver:devicemapper Pool Name:docker--vg-docker--pool# This has become a related device file Pool Blocksize: 524.3 kB backing Filesystem: <unknown> Data file:metadata file:data Space Used: 14.16 GB data Space total: 64.35 GB data space Available: 50.19 GB Metadata space used: 4.702 MB Met Adata Space total: 109.1 MB Metadata Space Available: 104.3 MB Udev Sync supported: true Library Version: 
                   
                    1.02
                    93-rhel7 (2015-01-)      
                      ... 

Third, reference

• Official Docker Documentation:
  • Docker and the Device Mapper storage driver
• Performance-Related:
  • Random I/O performance improves when increasing THINP chunksize
  • Comprehensive overview of Storage Scalability in Docker
  • Performance Tuning of Docker and RHEL Atomic
• Deployment Related:
  • Managing Storage with Docker formatted Containers on Red Hat Enterprise Linux and Red Hat Enterprise Linux Atomic Host
  • Friends Don ' t let Friends Run Docker on Loopback in Production
  • Docker-storage-setup
  • Docker and Devicemapper ' s Thinpool in RHEL 7
  • Setting up Storage

Docker Device Mapper using DIRECT-LVM