Thoughts on wide stripe Technology

With the development of RAID technology, we have encountered application bottlenecks. The biggest problem is that Data Reconstruction takes too long. During the long data reconstruction process, the probability of multiple disks being damaged is very high. For raid 6, if the third disk is damaged, the data will be completely lost. In the process of Data Reconstruction, application data and reconstruction data compete with each other for a limited I/O bandwidth, resulting in a further increase in Data Reconstruction time and serious challenges to data security.

In the face of problems such as RAID, the industry has been thinking about whether our RAID technology has reached the end of history. I have also analyzed that many storage companies in the industry have put forward their own solutions to this problem. One of the most famous concepts is Declustered RAID. Declustered RAID is only a type of RAID with no specific definition. Some companies simply remove RAID and hand over data redundancy to the file system. The file system provides data protection at the file level. Many people in the industry are included, and the underlying RAID volume is close, the future is a file system. Internet companies simply do not need RAID at the disk level. All data is stored on two or three storage nodes. The master node is broken and there are two other backup nodes. Recently, the dynamic disk technology of NetApp and Dell has been booming, greatly reducing the time for Data Reconstruction and tolerating damages to multiple disks. As we can see, we can feel that this field has already started. It is certain that solutions from different vendors are telling us that with the increase in disk capacity, traditional RAID is already difficult to protect data, and it is bound to exit the stage of history, it's just time.

Recently, I have been studying how to solve the problem of traditional RAID Data Reconstruction. Inadvertently, my ideas coincide with the NetApp solution. I personally think that dynamic disk technology is a good replacement and continuation of traditional RAID. The essence of dynamic disk technology is to separate the data protection layer from the disk management layer. In traditional RAID, the Data Protection Layer and disk management layer are tightly coupled. RAID6 data distribution Array) and Disk Group) are a one-to-one relationship, once a Disk goes bad, it will affect all the strip in an Array. This tightly coupled relationship will lead to the following two performance bottlenecks:

1) spare is a serious write bottleneck. After all the data is reconstructed, the spare disk must be written. This performance bottleneck is easily discovered and can be solved through the distribute spare technology.

 

2) rebuild the read bottleneck. In traditional RAID, because of the one-to-one correspondence between the protective layer and the disk layer, the maximum read performance of reconstruction is the read performance of a single disk. Therefore, refactoring read is a serious bottleneck for data refactoring. In addition, this bottleneck will limit the application of distribute spare technology.

Dynamic Disk technology can solve the above two bottlenecks. Therefore, the Data Reconstruction performance can be greatly improved and can be linearly expanded with the number of disks. The core technology of dynamic disks to solve the above problems lies in the wide stripe technology. After the data protection layer is separated from the disk management layer, the data in the data protection layer can be dynamically and evenly distributed to all disks. In this way, a damaged disk will affect all arrays and call all disks to recover the damaged data. Naturally, the rebuild read and spare single point write problems will disappear. When the number of disks reaches a certain level, the data rebuild performance will be greatly improved.

Some people worry that Wide Stripe will introduce mixed read/write problems, resulting in a decline in the read/write performance of a single disk. Indeed, when the data in the strip is unordered to all disks, the read/write performance of a single disk will decrease. Therefore, when the number of disks is small, dynamic Disk technology brings little value. The more disks, the larger the value. It can be said that the "three stints, better than Zhuge Liang", the performance of a single disk does not matter, the key lies in the overall performance.

650) this. width = 650; "title =" df.jpg "src =" http://www.bkjia.com/uploads/allimg/140112/2059303928-0.jpg "alt =" wkiol1lovr7cav03aadf4yytgbuw.jpg "/>

It is the rebuild time comparison result of the Dell Dynamic Disk array. After dynamic disk technology is used, the reconstruction time is greatly reduced, and the performance is particularly prominent when the two disks are damaged at the same time. Dynamic Disk technology is a good alternative to traditional RAID.

Compared with the file system data redundancy solution, what is the difference between dynamic disk technology from a technical perspective? In fact, in my opinion, dynamic disk technology (DDP), IBM Declustered RAID, Panasas's Object Store, and HP's Virtual RAID are essentially the same. In summary, this is the "separation of data protection layer and disk management layer" mentioned above ". Data protection layer can be implemented at the file system level or block device level. Different implementations produce different system forms and features. The dynamic disk technology provides a data protection layer at the block device level, and Object Store implements a data protection layer at the file system level. The disk management layer allocates resource blocks to provide storage objects for the data protection layer. Therefore, the solutions mentioned above share the same idea, so there are many common problems.

In the face of the traditional RAID problem, the separation of the data protection layer and the disk management layer is the trend of the times. After the separation, the Wide Stripe technology can be used to crack the bottleneck of traditional RAID reconstruction and greatly improve data reliability. Of course, new things must have new problems. For example, how to balance data distribution? How much impact does data migration have on application performance? How can we better optimize the data allocation algorithm? How to optimize sequential read performance? The advantages of new technologies are obvious, and there are many problems. A great era has just begun.

 

This article is from the "Storage path" blog, please be sure to keep this source http://alanwu.blog.51cto.com/3652632/1350266