Analysis of the prospect of RAID technology

Since 1988, when the University of Berkeley invented RAID (disk array) technology, RAID has been the core of the storage area, shouldering the responsibility of protecting data. But as disk capacity continues to grow, RAID technology is encountering unprecedented new problems. The biggest problem is that the data reconstruction time is too long, which prolongs the system degraded time, increases the probability of multiple disk failures and reduces the data reliability. And during the raid data reconfiguration, the raid itself rebuild IO has an impact on the application Io, which directly results in a significant reduction in the performance of the application IO. More seriously, the application of IO and rebuild IO constitutes an interactive relationship, and a large number of application IO can also cause rebuild time to be extended further.

At present, in the storage system, 4TB disk is about to be applied to a large area. The application of the 4TB disk must consider the problem of RAID itself. Here, I did a small experiment, a storage vendor's RAID rebuild time was evaluated, without the application of Io, for 500GB disk, need nearly 2 hours of rebuild time, and for 4TB disk, rebuild time increased to more than 14 hours.

Relationship between disk capacity and refactoring time

The RAID algorithm we commonly use today is RAID-6, which can tolerate up to 2 disk failures, and if more than two disks fail, it will cause the entire RAID system to crash. So, within 14 hours, if two disks have read and write errors, the entire RAID system will fail, which is the biggest problem that future raid needs to address.

In the face of the above problems, the industry has put forward some solutions, the basic ideas are as follows:

1, continuation of RAID5 to RAID6 technical thinking. Rebuild time can do its best to optimize, for example, the use of copy to replace the traditional rebuild method (CAs computing patent). In addition, if you can allow RAID tolerance for more disk failures, such as tolerating a 3-speed disk failure, then the long refactoring time is not too much of a problem. Therefore, many people in the industry began to research and development can tolerate three disk failure algorithm, the typical algorithm is reed-solumen algorithm. We call this thinking raid a RAID-7. RAID-7 The biggest problem is the need for stronger computing resources, RAID-7 computational complexity is much higher than RAID-6, therefore, RAID-7 research and development needs of hardware vendors to participate in research and development.

2,de-clustered raid thought. One idea of declustered raid is that, in most cases, the failed disk model is partial failure (local damage) rather than overall damage. To do this, declustered raid divides a disk into multiple zones and builds raid based on those zones. If a read-write error occurs in a zone, then raid only needs to refactor the area without having to refactor the entire disk. This kind of thought can reduce rebuild time, and can spread spare space to different disk in many regions, can avoid spare disk become IO bottleneck point in rebuild process. Declustered raid has studied the large-capacity disk failure model and optimized the existing raid, but it is not considered flexible enough to effectively separate applications IO and rebuild IO.

3, build raid based on low-level virtualization (Low level virtualization). As disk capacity continues to grow, it is necessary to add a virtualization layer to manage physical disks on the lower level of the raid, to manage physical disks through the virtualization layer, and to provide virtual disks for raid. When a read-write error is encountered, the virtualization layer can remap raid requests, making many error messages transparent to raid and reducing the number of RAID data refactorings. With this virtualization layer, the data awareness can be easily reconstructed to further reduce the raid refactoring time. Because the virtualization layer can manage the underlying physical resources, the disk health state can be monitored to allocate the storage resources reasonably and achieve the purpose of reliable data storage. Most importantly, the virtualization layer can detach application IO and Rebuild io, and in many cases Rebuild IO does not have a performance impact on application IO. Of course, there are some problems with the underlying virtualization, the most critical is how to allocate storage resources? Do not waste storage space to the maximum extent possible. At present, I am conducting research and exploration in this field.

4, in many clustered file systems, the traditional RAID technology has been basically abandoned, and the function of data redundancy protection is put on the file system level. For example, the Hadoop file system stores data redundancy in three copies directly at the file system level.

RAID technology has always been the key to data storage, in the face of increasing disk capacity today, we need to think about the future of RAID technology, today 4TB disk will be a large area of application, tomorrow 10TB hard drive will be far?

This column more highlights: http://www.bianceng.cnhttp://www.bianceng.cn/database/storage/