A number of issues arising from storage virtualization

This period of time has been considering a system based on storage virtualization, from the point of view of data reliability has encountered a very difficult problem.

Storage virtualization technology has been applied in a number of areas, for example, as shown in the following illustration, in a VMware virtual machine environment to improve IO efficiency, virtual virtualization technology is needed to better order the IO requests sent by multiple virtual machines to improve IO performance. In the field of SSD storage, virtualization technology is used to solve the performance loss problem of NAND flash, and to balance the life of NAND flash block. In the field of disk arrays, there are currently some manufacturers using virtualization technology to explore the lifetime potential of the disk, the disk data block relocation, so that the most efficient use of disk, and reduce disk refactoring time to the minimum.

One of the core ideas of storage virtualization technology is the granular management of physical disk resources and the dynamic allocation of physical block resources to logical devices. The upper level application uses the logical device, through the virtualization layer can easily know how many physical resources the storage device uses, once the physical resources have the problem, may through the data migration and so on the data redistribution. Because of the fine-grained management of physical resources and logical resources, it is very flexible to predict data security and then do data remapping. Of course, in the storage virtualization technology, the data block allocation strategy is very important, for different application requirements, can adopt different block assignment strategy. The simple distribution diagram between the logical block and the physical block is shown below:

See more highlights of this column: http://www.bianceng.cnhttp://www.bianceng.cn/Servers/virtualization/

In the process of application of storage virtualization technology, data security issues need special consideration. When there is a problem with the physical block, what the virtualization layer often needs to do is to map the data and migrate the data, as shown in the following illustration:

The data remapping problem is not very large, because the data is still in memory, unless there is a problem with memory. However, there is a significant risk that the data migration the virtual layer to fetch data from the source block and write the new block. If the data in the source block is corrupted, the virtualization layer cannot detect the error at all, and it writes the data to the new block. Because virtualization technology often reduces the level of application errors, the invalid data migration method will accumulate errors, which ultimately face the problem of data unrecoverable, in the absence of virtualization technology, the upper application will do more error handling, so that can detect, restore silent bad block. Of course, this problem will slow down if the top application does cyclical verify.

Virtualization technology introduces a new topic for reliable data migrations, so how do you achieve reliable data migrations? This is a problem for storage virtualization to consider. We do not consider these problems when we do the experimental system, but for the actual system, this is the key to design. To solve this problem, I think the SCSI layer of the End-to-end data validation standard is a very good choice, with the support of the hardware, in order to improve the reliability of the data, without too much impact on performance. However, the current standard only supports the SAS disk, so without the standard support, we can improve data reliability by adding additional checksum information, but obviously performance can be a big problem. This performance loss is acceptable for the backup system, but it is not acceptable for primary storage.

Therefore, when storage virtualization technology is heavily applied, we need to consider more data integrity issues in the IO path, and individuals feel that they should speed up the extension and application of end-to-end data integrity standards.