Use Device Mapper to change the size of the Docker container

Jérôme Petazzoni (Docker evangelist) Translator: Mark Shao (senior engineer, EMC China)

If you use Docker on CentOS, Rehl, Fedor, or other Linux distributions that do not have AUFS support by default, you may need to use Device Mapper's storage plug-in. By setting this plugin as the default, it will store all your containers in a short 100G file and limit the maximum of 10GB per container. This article will show you how to break this limit and move the storage of the container to a specified partition or LVM volume.

How it works

To really understand what we're going to do, let's start by understanding how the Device Mapper plugin works.

It is based on the "thin target" feature of Device Mapper. It is actually a snapshot of the target block device, which is called "lite" because it allows thin provisioning. Thin provisioning means that you have a pool of available storage blocks, and then you can create blocks of any size (virtual disks, if necessary) from that pool, which will be marked as used (or taken away from the pool) after you actually read and write.

This means that you can over-use the pool, such as creating thousands of 10GB volumes in a 100GB pool, or even a 100TB volume inside a 1GB pool. As long as your actual read and write block capacity is not larger than the size of the pool, you can do OK.

In addition, the way to streamline your goals is to take snapshots. This indicates that you can create a shallow copy of the existing volume at any time. In the eyes of the user, as you have two volumes, they can be individually modified independently of each other. Even if you make a complete copy, it is instantaneous (even for large volumes) in time, and they do not reuse the storage two times. Additional storage occurs only when any of the volumes are changed, and then the thin target allocates a storage from the pool quickly.

In essence, "thin target" actually uses two storage devices: one (large) is the storage block pool itself, and there is a small storage of some meta data. These metadata include volumes, snapshots, and the mapping information for each volume's block or snapshot and block in the storage pool.

When Docker uses the Device Mapper storage plug-in, it /var/lib/docker/devicemapper/devicemapper/data /var/lib/docker/devicemapper/devicemapper/metadata creates two files (if they don't exist) and stores the corresponding storage pool and related metadata. This is handy and you don't need to do any installation work (you don't need extra partitions to store Docker containers, or build LVM or something like that). However it has also two disadvantages:

-Storage pool will have a default capacity of 100GB

-It will be supported by sparse files. From the point of view of the efficiency of disk usage, this is good (just like in a thin pool of volumes, it starts out small and only uses the disk's storage blocks when it actually needs to be written). But from a performance point of view it is not so good, because the VFS added some extra burden, especially "the first time to write."

Before you know how to resize a container, let's try to add more space to the pool.

We need a bigger pool.

警告: The following actions will remove all of your containers and mirrors, making sure you have backed up the previous data!

Remember that when the data and meta-information files don't exist, Docker creates them, so the solution is simple: create them in Docker before you start them!

1. Stop the Docker daemon because we're going to reset our storage plug-in, and if we remove the file while it's running, the bad stuff will happen.

2. Wipe away /var/lib/docker . Warning: As mentioned earlier, this operation will remove all of your containers and mirrors.

3. To create a storage directory:mkdir -p /var/lib/docker/devicemapper/devicemapper

4. Create your pool: dd if=/dev/zero of=/var/lib/docker/devicemapper/devicemapper/data bs=1G count=0 seek=250 Create a 250G sparse file. If you specify bs=1G count=250 (without using seek an option), it creates a normal file (not a sparse file).

5. Restart the Docker daemon. Tip: By default, Docker will use it if you have AUFS support, so if you want to force Device Mapper plug-ins, you need to add the option to the command that launches Docker -s devicemapper .

6. Use docker info to check Data Space Total if the value is correct.

We need a faster pool.

警告: The following actions will also remove all of your containers and mirrors. Be sure to keep your important images in registry and keep the important data inside your container.

The easiest way to get a faster pool is to use a real device instead of a file-based loop device. Process is almost the same. Suppose you have a completely empty hard disk, /dev/sdb and you want to use it entirely for storage of containers, you can do this:

1. Stop the Docker daemon

2. Removal /var/lib/docker (déjà vu, right?) ）

3. Create a storage directory:mkdir -p /var/lib/docker/devicemapper/devicemapper

4. Create a data soft link in the directory, pointing to the device:ln -s /dev/sdb /var/lib/docker/devicemapper/devicemapper/data

5. Restart Docker

6. Use docker info to check Data Space Total if the value is correct

Using RAID and LVM

If you want to merge multiple similar disks, you can use the RADID10 software, which is implemented by linking to it /dev/md . Another great option is to put your disk (or RAID array) into the physical volume of the LVM and create two logical volumes: one is data and one is metadata. I don't have any special suggestions for the best size of the metadata pool, but 1% of the data pool looks good.

Just like before, stop Docker, remove its data directory, and then create a /dev/mapper symbolic link to the device and restart Docker.

If you need more knowledge about LVM, see the LVM howto here.

Expansion container

By default, if you use Device Mapper's storage plug-in, all images and containers are created from an initial 10G file system. Let's look at how to create a container from a larger file system.

First, we use the Ubuntu image to create our container. We don't need to run anything in this container, only this file (or the associated file system) exists. To demonstrate, we'll run in this container df and look at the size of the root file system.

$ docker run -d ubuntu df -h /4ab0bdde0a0dd663d35993e401055ee0a66c63892ba960680b3386938bda3603

Because of the need to modify some of the volume information in Device Mapper management, we now run some commands as root. All commands that begin with # must be executed as root. As long as you can access the Docker Socket service, you can also execute other commands (starting with $) with the identity of the normal user.

Let's see /dev/mapper , there should be a symbolic link to the file system of the corresponding container to docker-X:Y-Z- begin with:

# ls -l /dev/mapper/docker-*-4ab0bdde0a0dd663d35993e401055ee0a66c63892ba960680b3386938bda3603lrwxrwxrwx 1 root root 7 Jan 31 21:04 /dev/mapper/docker-0:37-1471009-4ab0bdde0a0dd663d35993e401055ee0a66c63892ba960680b3386938bda3603 -> ../dm-8

Remember that full name and we'll use it in the future.

First, let's look at the information table for the current volume:

# dmsetup table docker-0:37-1471009-4ab0bdde0a0dd663d35993e401055ee0a66c63892ba960680b3386938bda36030 20971520 thin 254:0 7

The second number is the size of the device, indicating how many 512-bytes sectors are there. This value is slightly above the size of 10GB.

Let's calculate how many sectors a 42GB volume requires,

$ echo $((42*1024*1024*1024/512))88080384

A magical feature of the thin snapshot target is that it does not limit the size of the volume. When you create it, a thin volume uses 0 blocks, and when you start writing to the block, they are allocated from the pooled pool. You can write 0 blocks, or 1 billion blocks, which is fine with a thin snapshot target. The size of the file system is only related to the Device Mapper table.

Feeling confused? Don't worry. We just need to load a new table, which is exactly the same as before, but there are more sectors. That's all.

The old table is 0 20971520 thin 254:0 7 . We will change the second number and be very careful to keep the other values intact. Your volume may not be 7 , so use the correct value!

This action:

# echo 0 88080384 thin 254:0 7 | dmsetup load docker-0:37-1471009-4ab0bdde0a0dd663d35993e401055ee0a66c63892ba960680b3386938bda3603

Now if we check the table information again, the steps are the same as before. First, activate the new table using the following command:

# dmsetup resume docker-0:37-1471009-4ab0bdde0a0dd663d35993e401055ee0a66c63892ba960680b3386938bda3603

After executing the command, check the table information again and discover that it will use the new number of sectors.

We have adjusted the size of the block device, but we still need to resize the filesystem, which we use resize2fs to manipulate:

# resize2fs /dev/mapper/docker-0:37-1471009-4ab0bdde0a0dd663d35993e401055ee0a66c63892ba960680b3386938bda3603resize2fs 1.42.5 (29-Jul-2012)Filesystem at /dev/mapper/docker-0:37-1471009-4ab0bdde0a0dd663d35993e401055ee0a66c63892ba960680b3386938bda3603 is mounted on /var/lib/docker/devicemapper/mnt/4ab0bdde0a0dd663d35993e401055ee0a66c63892ba960680b3386938bda3603; on-line resizing requiredold_desc_blocks = 1, new_desc_blocks = 3The filesystem on /dev/mapper/docker-0:37-1471009-4ab0bdde0a0dd663d35993e401055ee0a66c63892ba960680b3386938bda3603 is now 11010048 blocks long

As an optional step, we'll restart the container and check that we do have the right size of free space:

$ docker start 4ab0bdde0a0dd663d35993e401055ee0a66c63892ba960680b3386938bda3603$ docker logs 4ab0bdde0a0dd663d35993e401055ee0a66c63892ba960680b3386938bda3603df: Warning: cannot read table of mounted file systems: No such file or directoryFilesystem      Size  Used Avail Use% Mounted on-               9.8G  164M  9.1G   2% /df: Warning: cannot read table of mounted file systems: No such file or directoryFilesystem      Size  Used Avail Use% Mounted on-                42G  172M   40G   1% /

Want to automate the whole process? Sure, no problem.

CID=$(docker run -d ubuntu df -h /)DEV=$(basename $(echo /dev/mapper/docker-*-$CID))dmsetup table $DEV | sed "s/0 [0-9]* thin/0 $((42*1024*1024*1024/512)) thin/" | dmsetup load $DEVdmsetup resume $DEVresize2fs /dev/mapper/$DEVdocker start $CIDdocker logs $CID

Expansion mirror

Unfortunately, the current version of Docker does not allow us to easily expand the image. You can expand the mirrored block device and create a container from it, but the new container will not have the correct size.

Similarly, if you commit a large container, the resulting image will not be very large (this is caused by the way Docker prepares the file system for mirroring).

This means that if a container is really over 10GB, you can't properly submit it as a mirror without using some other tips.

Summarize

Docker will certainly provide some better ways to expand the container in the future, and the code changes required are small. Managing a thin pool and corresponding meta-information is complex (because it requires a lot of different operational processes and a potential data migration.) Given the removal of everything to build a new pool, it is not mentioned in this article, but some of the solutions we have mentioned today are believed to have helped you.

As usual, if you have questions or comments, ping me on IRC immediately (Jpetazzo on freenode) or contact me via Twitter (@jpetazzo).

This article is published by Jérôme Petazzoni, click here to read the original text. Mark Shao translated this article and you can talk to him on GitHub. The article is contributed by Jérôme Petazzoni and click here to read the original publication.

Use Device Mapper to change the size of the Docker container