RAID Technology Introduction and summary (reprint)

Brief introduction

RAID is a noun that we can often see. But because rarely can experience in the actual environment, so it is difficult to have a very clear understanding and grasp of its principle. This article will introduce and summarize the raid technology in order to clarify its concept as much as possible.

RAID is all called Redundant Array of independent disks (redundant array of independent Disks), the basic idea is to combine a number of relatively inexpensive hard drives to become a hard disk array group, so that performance reaches even more expensive, large capacity of the hard disk. RAID is usually used on a server computer and uses the exact same hard drive to form a logical sector, so the operating system will only treat it as a hard disk. RAID is divided into different levels, each of which has a different tradeoff between data reliability and read and write performance. In the actual application, can choose the different RAID scheme according to own actual demand.

Standard raid

RAID 0

RAID0 is called striped (Striping) storage, where data is segmented and stored in separate disks, and read-write can be processed in parallel. As a result, the read-write rate is n times of a single disk (n is the number of disks that make up the RAID0), but without data redundancy, damage to a single disk can cause data to be unrecoverable. Most striping implementations allow managers to define how data is segmented and written to disk by adjusting two key parameters, which have a significant impact on the performance of RAID0.

STRIPE WIDTH

stripe widthRefers to the number of stripe that can be written in parallel, which equals the number of disks in the disk array.

STRIPE SIZE

Also known as block size ( chunk size , stripe length , granularity ), refers to the size of the data block written to each disk. Block-segmented raid usually allows the selection of block sizes ranging from 2KB to 512KB, but also higher, but must be 2 times the number of points. BYTE-segmented (for example, RAID3) stripe size is typically 1 bytes or 512 bytes, and the user cannot adjust. The stripe size impact on performance is difficult to measure easily, and it is best to adjust and observe the impact of your own needs in real-world applications. Typically, fewer stripe size files are divided into smaller chunks, which can transmit data faster, but require more disks to keep, increase positioning performance , and vice versa. It should be said that there is not a theoretically optimal value. Many times, consider the strategy of the disk controller, such as some disk controller will wait until a certain amount of data to start writing to disk.

RAID 1

Mirrored storage (mirroring) with no data validation. Data is written equally to two or more disks, and it is conceivable that the write speed will be slower, but the reading speed will be faster. The read speed can be approximated by the sum of all disk throughput, and the write speed is limited to the slowest disk. RAID1 is also the lowest disk utilization. If you use two different size disks to build RAID1, you can use the smaller space, the larger disk more out of the part can be used for his use, not wasted.

RAID 2

RAID0 's improved version, added Hamming code ( Hanmming Code ) error checking.

Hamming code can detect up to two simultaneous bit errors, and can correct single-bit errors. There is an inequality relationship between the number of bits of Hamming code and the number of bits of data, namely:

1	2^P ≥ P + D +1

P represents the number of Hamming code, D represents the number of data bits, such as 4-bit data requires 3-bit hamming, 7-bit data requires 4-bit hamming codes, 64-bit data need 7-bit Hamming code. RAID2 is divided into 1bit to write the data, and P:D represents the number of data disk and check disk. Therefore, the larger the data bit width, the smaller the scale of the disk used for checking. Because Hamming codes can correct single-bit errors, the Hamming code can correct the data when a single disk is damaged.

RAID 2 because every read and write requires a full set of disk linkage, so in order to maximize its performance, it is best to ensure that each disk spindle synchronization, so that at the same time each disk head of the sector logic number is consistent, coexistence and take, to achieve the best performance. If synchronization is not possible, a wait is generated and the speed is affected.

Compared to RAID0, the RAID2 has a better transfer rate. Because RAID0 generally is stripe size too large for the 1bit of RAID2, it is not guaranteed to be multi-disk parallel each time. RAID2 Each IO can be guaranteed to be multi-disk parallelism, in order to achieve this advantage, the disk seek time must be reduced (seek time than the data transfer time is a few orders of magnitude), so RAID2 suitable for continuous io, large io (such as video streaming service) situation.

RAID 3

Similar to RAID2, data striping (stripe) is stored on different hard disks, with data in bytes, except that RAID3 uses a single disk to store simple parity information, so the final disk count is n+1. When one of the hard drives in this n+1 fails, the data from the other n hard drives can also be restored to the original data, and when a new hard disk is replaced, the system can restore the complete checksum fault-tolerant information again.

Because in a hard disk array, more than one hard disk at the same time the probability of failure rate is very small, so under normal circumstances, the use of RAID3, security can be guaranteed. RAID 3 will spread the data write operations across multiple disks, regardless of which data disk is written to the data, you need to rewrite the relevant information in the check disk. Therefore, for applications that often require a large number of write operations, the load on the check disk will be large enough to meet the speed of the program, resulting in degraded performance across the raid system. For this reason, RAID 3 is more suitable for applications where there are fewer write operations and more read operations, such as databases and Web servers.

RAID 4

Similar to RAID3, but RAID4 is accessed by block (sector). There is no need for a small I/O operation, like RAID3, to involve a whole group of two hard drives (one data disk, one check disk) in the group, thus increasing the small data I/O speed.

RAID 5

Parity (XOR), data is stored in block-segmented striping. The checksum information is stored on all data disks.

RAID5 stores the data and the corresponding parity information on each disk that makes up the RAID5, and the parity information and the corresponding data are stored on separate disks, where the full data is stored on any N-1 block disk, meaning that there is space equivalent to a disk capacity for storing parity information. Therefore, when a disk in RAID5 is damaged, the data integrity is not affected, thus ensuring data security. When the damaged disk is replaced, the raid also automatically rebuilds the data on the disk with the remaining parity information to maintain the RAID5 's high reliability.

RAID 5 can be understood as a compromise between RAID 0 and RAID 1. RAID 5 provides data security for the system, but with a lower level of protection than mirroring and higher disk space utilization than mirroring. RAID 5 has a similar data read speed as RAID 0, except that the speed of writing data is slightly slower than writing to a single hard drive because of the extra parity information.

RAID 6

Similar to RAID5, but added a second independent parity information block, two independent parity systems using different algorithms, the data reliability is very high, even if the two disks fail at the same time will not affect the use of data. However, RAID 6 requires more disk space allocated to parity information and a greater write loss than RAID 5, so "write performance" is very poor.

As is known to the graph, there is an XOR check area for each data block in addition to the peer data Xor check area on each hard disk. Of course, the current disk data block checksum data can not exist on the current disk but interleaved storage. From a mathematical point of view, RAID 5 uses an equation to solve an unknown variable, while RAID 6 can restore two unknown data by forming a set of equations with two independent linear equations.

With the growth of hard disk capacity, RAID6 has become more and more important. Terabytes of hard drives are more likely to cause data loss, and the process of rebuilding data (such as RAID5, allowing only one hard drive to be corrupted) is also getting longer, even to weeks, which is completely unacceptable. RAID6 allows two hard drives to fail at the same time, so people are getting more and more popular.

With the advent of Cd,dvd and Blu-ray discs, the storage media appeared erasure code technology, even if the media surface scratches, can still play, most common erasure code algorithm has evolved into the 60 's MIT Lincoln Laboratory developed Reed-solomon code. In reality, most RAID6 implementations use standard RAID5-bit and Reed-Solomon code. The use of the Pure Erase code algorithm allows the raid 6 array to fail two or more hard drives, stronger protection, some implementations provide multiple levels of protection, and even allow the user (or storage administrator) to specify the level of protection.

Hybrid Raidraid 01

As the name implies, is the combination of RAID0 and RAID1. First make a strip (0), then do the Mirror (1).

RAID 10

Ditto, but do the mirror first (1), then do the strip (0)

RAID01 and RAID10 are very similar, and there is no difference in reading and writing performance. But RAID10 on security is better than RAID01. , assuming that the DISK0 is damaged, in RAID10, in the remaining 3 disks, the entire RAID will fail only if the DISK1 fails. However, in the RAID01, DISK0 damage, the left strip will not be read, in the remaining 3 fast disk, as long as the DISK2 or DISK3 two disk any damage, will cause raid failure.

RAID10 and RAID5 are also two kinds of schemes that are often used for comparison, both of which have been widely applied in production practice. RAID10 security is higher, but space utilization is low. As for read-write performance, it is very relevant to the cache, it is best to test the comparison selection according to actual situation.

Non-standard Raiddrfs

DRFS, i.e. DistributedRaidFileSystem , is a technology that attempts to combine raid with the DFS of Hadoop. The usual HDFs needs to be replication factor set to 3 in practice to ensure data integrity, and if Raid stripe and partity (parity) techniques are used, the data is divided into blocks and the checksum information (XOR or erasure code) is stored for each block. With these measures, the number of copies of a block can be reduced and the same data is guaranteed to be reliable, saving a significant portion of the storage space.

The drfs contains the following components:

• DRFS client: Provides an interface to the application to access the Drfs, fix it when the read file is found to be corrupted, and the entire operation is transparent to the application
• RaidNode: Creation, maintenance of inspection filesdaemon
• BlockFixer: Periodically checks files, recalculates checksums, and repairs files.
• RaidShell: Similar to hadoop shell .
• ErasureCode: The algorithm used by Drfs to generate the checksum, which can be an XOR or an Reed-Solomon algorithm. XOR can only create a check byte, and Reed-Solomon then can create no digits (the more bits, the more data can be recovered), if used Reed-Solomon , replication or even reduced to 1, the disadvantage is to reduce the degree of parallelism of data reading and writing (only from a single machine).

Implement software implementation

Now very many operating systems provide RAID software implementation, mainly by the following aspects:

• The software creates a raid on multiple devices, such as a tool on Linux mdadm . See the examples in the reference links for specific use.
• LVMOr Veritas , the virtual Volume management tool.
• File system implementation: btrfs , ZFS GPFS . These files can directly manage data on multiple devices, enabling similar levels of RAID functionality.
• RAID System (RAID-F) that provides data validation on top of existing file systems

Firmware/Driver Implementation

The software implementation is always compatible with the boot process of the system, the hardware implementation (RAID controller) is always too expensive and is vendor-specific technology, so there is a hybrid implementation: When the system starts, the firmware (firmware) to achieve raid, the system started almost, the drive to manage the raid. Of course, this requires the operating system to provide support for this driver.

Reference links

1. Wikipedia: RAID
2. Stripe Width and Stripe Size
3. RAID6
4. DRFS
5. HDFS and Erasure Codes
6. Linux floppy disk matrix software RAID implementation steps introduction
7. Wiki:mdadm
8. Introduction to the VxFS (Veritas file System) filesystem
9. Linux LVM Logical Volume management Detailed introduction
10. About GPFS
11. Raid-f
12. Basic RAID Introduction

The source of the original: Ya Yu (@ If East is near the sea)

Http://yayua.github.io/storage/raid/

RAID Technology Introduction and summary (reprint)