Ceph Introduction of RBD Implementation principle __ceph

RBD is a block device provided by Ceph, this article will briefly introduce its implementation principle.

Ceph official documentation tells us that Ceph is essentially an object store. It is also understood that ceph block storage is actually handled by several objects at the client. In other words, for ceph clusters, the data is stored in the format of the line object. As the following illustration shows, the information from the mirror (the form of the block storage) shows the composition and attributes of the block storage.

Figure 1 RBD Mirror Properties

After you create a mirror of a storage pool, you have related object data in the storage pool. Two of these objects are associated with the mirror. The following illustration shows a storage pool of two mirrors, and see that the storage pool contains three objects (the mirror of format 2 is different).

Figure 2 Mirroring and Ceph object relationship

Where the Rbd_directory object is a mirrored folder object that contains a list of object names that the storage pool contains. The IMAGENAME.RBD object is the object that corresponds to the mirror name, which contains the property information for the object. Figure 3 is the data exported from the object IMG_TEST.RBD. The comparison between figures 1 and 3 shows the correspondence between the two, and no longer dwell on it.

Figure 3 Folder object Export data

In the relevant section of the strip, three parameter order , Stripe_unit, and Stripe_countabout block storage are mentioned. The method of writing and arrangement of the data determined by the three. The default order is 22, which means that the object size is the same size as the object, and Strip_count is 1 4mb,strip_unit. Therefore, for an operation that writes data sequentially, there is an object to store the data for every 4MB of data written. Analogy As a result, this article carries out the actual test, mounts the mirror Img_test and writes the 4MB data to it in sequence, and you can see that there is an extra object under the storage pool zsn_2.

Figure 4 RBD data and object relationships

Writes the data again, and locates the location where the data was written to offset to 40MB. After you have finished writing the data, look at one more object in the list of storage pool objects, and the last two digits of the object name are 0a, or decimal 10.

Figure 5 RBD data and object relationships

Figure 6

To sum up, you can get the following conclusions:

1 The final storage form of the block device in the Ceph cluster is an object, and the object name is associated with LBA.

2 Block device metadata is relatively simple, stored in a storage pool in an object, its contents include object name prefix, order, stripe_unit and Stripe_count, etc., through these parameters, has been written to the data LBA, can easily calculate the object name and offset. 3 The image under the default configuration (format 1) does not achieve the load balancing of the OSD, and needs to use a mirror of format 2.

For a storage pool with format 2, its metadata is in the object attribute, not in the object content of this article, and its RBD is placed in the same way, which can be further analyzed by the method of this article.