A detailed explanation of the disk array RAID technology principles

1.RAID Description

RAID is an abbreviation for the English Redundant Array of Inexpensive disks, abbreviated as a redundant array of inexpensive disks.

Simply put, RAID is a combination of multiple separate hard disks (physical hard disks) in different ways to form a hard disk group (logical hard disk), providing higher storage performance than a single hard drive and providing data backup technology.

Different ways of composing a disk array become RAID level (RAID levels).

The function of data backup is that when the user data is damaged, the data can be recovered by using the backup information, thus guaranteeing the security of user data. As the user looks, the group of disks is like a hard disk, which the user can partition, format, and so on. In summary, the operation of the disk array is identical to a single hard drive. The difference is that the disk array is much faster to store than a single drive and can provide automatic data backup.

RAID technology is broadly divided into two types: hardware-based RAID technology and software-based RAID technology.

RAID functionality can be achieved with the software that comes with it under Linux, eliminating the cost of buying expensive hardware RAID controllers and attachments to greatly enhance the IO performance and reliability of the disks. Because it is the software to implement the RAID function, so it is flexible configuration, easy to manage. With software RAID, it is also possible to combine several physical disks into a larger virtual appliance for performance improvement and data redundancy. Of course, hardware-based RAID solutions are more slightly better than software-based RAID technology in terms of performance and service performance, with the ability to detect and repair multiple errors, error disk detection, and array reconstruction.

There are three main benefits of using RAID technology for storage systems:
Provides disk spanning functionality by organizing multiple disks together as a logical volume;
To increase the speed of accessing the disk by dividing the data into multiple chunks (blocks) to write/read multiple disks in parallel;
Provides fault tolerance through mirroring or validation operations;

2.RAID level introduction;

Commonly used RAID classes are RAID 0, RAID1, RAID 2, RAID 3, RAID 4, and RAID 5, plus two-in-one raid 0+1﹝ or RAID 10﹞.

Let's compare the advantages and disadvantages of these RAID levels first:

Relative disadvantage of RAID level relative advantages
RAID 0 Access speed is the fastest without fault tolerance
RAID 1 is fully fault-tolerant and cost-efficient
RAID 2 with sea-check, data redundancy, slow
RAID 3 Write performance is best without multi-tasking capabilities
RAID 4 features multi-tasking and fault tolerance Parity disk drives cause performance bottlenecks
RAID 5 with multi-tasking and fault-tolerant features when writing overhead
RAID 0+1/raid 10 is fast and full fault-tolerant cost is high

2.0 RAID 0 features, principles and applications;

Also known as stripe mode (striped), which is to spread the continuous data across multiple disks, as shown in the figure. When a system has data requests it can be executed in parallel by multiple disks, each of which performs its own portion of the data request. The parallel operation on this data can make full use of the bus bandwidth and significantly improve the overall disk access performance. Because reads and writes are done in parallel on the device, read and write performance increases, which is typically the main reason for running RAID 0. However, RAID 0 does not have data redundancy and if the drive fails, then no data can be recovered.

RAID 0: Stripe group with error-free control

To achieve RAID0 must have more than two hard drives, RAID0 implemented a stripe group, the data is not saved on a hard disk, but divided into blocks of data saved on different drives. Because the data is distributed across different drives, the data throughput rate is greatly increased and the load on the drive is balanced. If the data you need is just as efficient on different drives as possible. It does not need to calculate the check code, the realization is easy. Its disadvantage is that it has no data error control, if the data in one drive is wrong, even if the data on the other disk is correct, it is useless. It should not be used in situations where data stability requirements are high. It is more appropriate to use RAID0 if the user is editing the image (including animation) and other requirements for transmission are relatively large. At the same time, RAID can increase the data transfer rate, such as the required to read the files distributed on two hard disks, the two hard drives can be read simultaneously. Then the time to read the same file was shortened to 1/2. At all levels, the speed of RAID 0 is the fastest. However, RAID 0 does not have redundant functionality, and if one disk (physical) is damaged, all data is not available.
2.1 RAID 1 features, principles and applications;

RAID 1, also known as mirroring (mirroring), has a fully redundant pattern, as shown in the figure. RAID 1 can be used on two or 2xN disks and uses 0 or more spare disks, which are written to the mirror disk each time the data is written. This array is highly reliable, but its effective capacity is reduced to half of the total capacity, and the disks should be equal in size, otherwise the total capacity will only have the smallest disk size.
RAID 1: Mirror image structure

For devices that use this RAID1 structure, the RAID controller must be able to read two disks simultaneously and write to two mirror disks. You can also see that you must have two drives by following the structure diagram. Because it is a mirror structure in the case of a set of disk problems, you can use the mirror image, improve the system's fault-tolerant ability. It is relatively easy to design and implement. The disk can read only one piece of data per read, which means that the data block transfer rate is the same as the read rate of the individual disk. Because the calibration of RAID1 is very complete, so it has a great influence on the processing ability of the system, usually the RAID function is implemented by software, and the implementation method can greatly affect the server efficiency when the server load is heavy. When your system requires very high reliability, such as data statistics, then using RAID1 is more appropriate. and RAID1 technology to support "hot replacement", that is, the case of continuous power to replace the failed disk, replace as long as the data from the mirror disk recovery. When the primary hard drive is damaged, the mirrored hard disk can work instead of the primary hard drive. Mirrored hard disk is the equivalent of a backup disk, it is conceivable that this hard disk mode security is very high, RAID 1 data security at all RAID level is the best. But its disk utilization is only 50%, the lowest of all RAID levels.

2.2 RAID 2 features, principles and applications; RAID 2: With a sea-check

In a conceptual sense, RAID 2 is similar to raid 3, where the data is striped across different hard disks, in bits or bytes. However, RAID 2 uses certain coding techniques to provide error checking and recovery. This coding technique requires multiple disk storage checks and recovery information, making RAID 2 technology more complex to implement. Therefore, it is seldom used in a business environment.

Verify that the number of disks used is too much!!

Each disk on the left side of the graph is the individual bits of the data, and the Hamming check code obtained by a different bit operation of the data can be saved on another set of disks, as shown in the figure below. Because of the characteristics of the sea-light, it can be corrected in the case of errors in the data, in order to ensure the correct output. Its data transfer rate is quite high, if you want to achieve a more ideal speed, it is best to improve the hard disk to save the Code ECC Code, for the controller design, it is more than raid3,4 or 5 to be simpler. No free lunch, also here, to use the sea-codes, you have to pay the cost of data redundancy. The rate of output data is equal to the slowest speed in the drive group.

2.3 RAID 3 features, principles and applications;

RAID 3 is an XOR operation that produces parity data (parity), which is written to the member disk drive in parallel access mode, and therefore has the advantages and disadvantages of parallel access mode. Further, RAID 3 each data transfer, update the entire Stripe﹝ that is, each member of the disk drive relative location to update ﹞, so there will not be a part of the disk drive existing data read out, with the new data for XOR operation, and then write the situation occurs ﹝ this situation in the RAID 4 and RAID 5 will occur, commonly referred to as read, Modify, write process, we can be translated as read, change, write the procedure ﹞. Therefore, in all RAID levels, the write performance of RAID 3 is the best.

RAID 3 Parity data is generally stored in a dedicated Parity disk, but because each data update the entire stripe, therefore, RAID 3 of Parity disk will not be like RAID 4 Parity disk, resulting in access bottlenecks.

The parallel access mode of RAID 3 requires the support of RAID controller special function to achieve disk drive synchronization control, and the advantages of the above write performance can be replaced by current caching technology, so it is generally considered that the application of RAID 3 will gradually fade out of the market.

RAID 3, with its superior write performance, is ideal for applications with large, continuous file writes, such as drawing, video, video editing, multimedia, data warehousing, high-speed data capture, and more.

RAID3 compared to RAID2, the use of the calibration disk data, cost reduction, calibration efficiency improvement, the disadvantage is not to support error correction

RAID3: Parallel transfer with parity check code

This check code and RAID2 different, can only check the wrong can not be corrected. It accesses data one band at a time, which improves read and write speeds, and it stores data in parallel like RAID 0, but at no faster than RAID 0. The checksum is generated and saved on another disk when the data is written. When required, the user must have more than three drives, the write rate and the readout rate are very high, because the parity bit is relatively small, so the calculation time is relatively small. It is very difficult to implement RAID control with software, and the implementation of the controller is not very easy. It is mainly used for graphics (including animation) and other requirements of high throughput rate. Unlike RAID 2,raid 3, a single disk is used to store parity information. If a disk fails, the parity disk and other data disks can regenerate the data. If the parity disk fails, data usage is not affected. RAID 3 provides a good transfer rate for large amounts of continuous data, but for random data, parity disks can be a bottleneck for write operations. The use of a separate check disk to protect the data, although no mirror security is high, but the hard disk utilization has been greatly improved, for n-1.

RAID3 execution of an IO will inevitably affect the use of all disks, at this time, the other queued io must wait, so RAID3 cannot implement concurrent IO.

2.4 RAID 4 features, principles and applications;

Creating a RAID 4 requires three or more disks, it saves the checksum information on one drive, and writes the data to another disk in RAID 0, as shown in the figure. Because a disk is reserved for the checksum information, the size of the array is (N-L) *s, where S is the smallest drive in the array. As in RAID 1, the size of the disks should be equal.

If a drive fails, you can use the checksum information to reconstruct all the data. If two drives fail, all of the data is lost. This level is not often used because the validation information is stored on a drive. Each time you write to another disk, you must update the information. As a result, it is very easy to create a bottleneck for verifying the disk when writing large amounts of data, so currently this level of RAID is rarely used.
RAID 4 is a standalone access mode while storing parity Data with a single, dedicated parity Disk. Each transmission of RAID 4 has a long ﹝strip﹞ profile and can perform overlapped I/O, so its read performance is good.

·

However, due to the use of a single proprietary parity Disk to store parity Data, there is a large bottleneck in writing. Therefore, RAID 4 is not widely used.

RAID4: Stand-alone disk structure with parity check code

RAID4 and RAID3 are very much alike, and the difference is that the data is accessed by data blocks, that is, by disk, each time a disk. As you can see in the picture, RAID3 is a horizontal bar, and RAID4 a vertical bar at a time. Its characteristics of the RAID3 is also quite like, but in the failure of recovery, it can be more difficult than RAID3, the controller design is much more difficult, and access to data is not very efficient.

The RAID4 executes an IO with only one disk, and the other IO can also be performed concurrently with this IO, provided that the other IO target is not the disk to which the IO is to be read or written.

2.5 RAID 5 features, principles and applications;

RAID 5 is probably the most useful raid mode when you want to combine a large number of physical disks and still retain some redundancy. RAID 5 can be used on three or more disks and use 0 or more spare disks. Just like RAID 4, the size of the resulting RAID5 device is (N-1) *s.

The biggest difference between RAID5 and RAID4 is that the calibration information is evenly distributed across the drives.

As shown in Figure 4, this avoids the bottleneck problems in RAID 4. If one of the disks fails, all data remains intact due to the checksum information. If an alternate disk is available, synchronizing the data begins immediately after the device fails. If two disks fail at the same time, all data will be lost. RAID5 can withstand a disk failure, but cannot withstand two or more disk failures.

RAID 5 is also a standalone access mode, but its parity Data is distributed to individual member disk drives, so in addition to having overlapped I/O multitasking (implementing parallel IO), it is also detached from a single dedicated parity such as RAID 4 Disk's write bottleneck. However, RAID 5 is still slightly affected by the "read, change, and write process" when the data is written.

Since RAID 5 can perform overlapped I/O multitasking, the higher the number of member disk drives in a RAID 5, the higher the performance is because one disk drive can only execute one Thread at a time, so the more disk drives you have, the more overlapped The more thread you have, the higher your performance will be. Conversely, the more disk drives, the higher the probability of a disk drive failure in the array, the reliability of the entire array, or the Mtdl (Mean time to Data Loss) is reduced.

Basically, multi-tasking environment, frequent access, data volume is not a great application, are suitable for the use of RAID 5 architecture,

such as Enterprise file server, WEB server, online trading system, e-commerce and other applications, are small data volume, access to frequent applications.

RAID5: Isolated disk structure for distributed parity

It can be seen from its schematic that its parity code exists on all disks, where the p0 represents the parity value of the No. 0 band, and the other meanings are the same. RAID5 's readout efficiency is very high, writing efficiency is general, block-type collective access efficiency is good. Because parity codes are on different disks, reliability is increased and a single disk error is allowed.

RAID 5 is also a data check bit to ensure the security of data, but it is not a separate hard disk to hold the data check bit, but the data segment of the check bit interaction on each hard disk. This way, any one of the hard drives can be damaged to reconstruct the corrupted data based on the parity bits on the other hard drives. The utilization of the hard disk is n-1. But it does not solve the parallelism of data transmission, and the design of controller is very difficult.

The important difference between RAID 3 and RAID 5 is that RAID 3 transfers every time, and involves all array disks. For RAID 5, most data transfers operate on only one disk, which can be done in parallel. In RAID 5 There is a "write loss", that is, each write operation, will produce four actual read/write operations, two reads old data and parity information, two times write new data and parity information. RAID-5, the advantage is to provide redundancy (support a disk off the line is still normal operation), disk space utilization (n-1/n), read and write Faster (N-1 times).

RAID5 The biggest advantage is that in the case of a disk drop, RAID will work as usual, compared to the RAID0 must be normal for each disk to work properly fault-tolerant performance is much better. So RAID5 is the most common type of RAID level.

The RAID5 check bit, or P-bit, is obtained through other stripe data xor (XOR).

P=d1 xor D2 xor D3 ... xor Dn (d1,d2,d3 ... DN is data block, p is checksum, XOR is different or operation)

The verification principle of XOR (Exclusive OR) is as follows:

The A and B values here represent two bits, where A and B are the same when the XOR result is 0,a and B, the XOR result is 1, and knowing the XOR result and any of the values in A and B, you can reverse another value. For example A for the 1,xor result is 1, then B is definitely 0, if the XOR result is 0, then B must be 1. This is the basic principle of XOR encoding and checking.

RAID 5 is a very useful raid mode and is widely used in a variety of environments. How RAID 5 mode works:

RAID 5 uses at least three hard drives to achieve the array, it can realize RAID 0 acceleration function can also realize RAID 1 backup data function, when there are three hard disks in the array, it will be required to store the data according to the user-defined segmentation size into a file fragment stored in two hard drives, at this time, The third hard disk in the array does not receive file fragments, it is used to verify the data stored in the other two hard drives in the part of the data, this part of the verification data is generated through a certain algorithm, you can use this data to recover the data stored on the other two hard disks. In addition, the task of the three hard disk is not immutable, that is, in this storage may be 1th hard disk and 2 good hard disk to store the fragmented file fragments, then the next time the storage may be the 2nd hard disk and the 3rd hard disk to complete this task. It can be said that in each storage operation, the task of each disk is randomly allocated, however, it must be two hard disks used to store the fragmented file fragments of another hard disk to store the checksum information.

This verification information is generally calculated by the RAID controller, which typically requires a single chip on a RAID controller to compute and decide which hard disk storage to send this information to.

RAID 5 also enables high-speed storage reads for RAID 0 and data recovery for RAID 1, which means that RAID 5 can utilize three hard disks to achieve RAID 0 speed doubling capability and RAID 1 data backup functionality, and when raid 5 a piece of hard disk damage, adding a new hard disk can also be used to restore data.

The following is an analysis of how RAID 5 can restore data, for example, using 3 hard disks to form a RAID 5 array with a user-defined split file size of 64k, at which time the file size required to be stored is 128k. First, when the RAID controller received this part of the data after the use of a certain algorithm to obtain the verification information, and then split the 128k file into two size 64k size of the file fragments, and then the two pieces of the files are placed on the 1th hard disk and 2nd hard disk, the final check information is sent to the 3rd hard drive. If one of the hard drives in the array is damaged, the original data can be restored: if the 3rd hard drive used to store the checksum information is damaged, the verification information can be re-generated through the 1th and 2nd hard drives, and if the damaged number is 1th or 2nd, You can regenerate the original file fragments using the checksum information stored on drive 3rd.

RAID 5 mode is not a good thing, if the information on a hard disk in the array has changed, then need to recalculate the fragmentation of the file, and the verification information needs to be recalculated, then, three hard disks need to be recalled.

Similarly, if you want to do a RAID 5 array, it is best to use the same capacity of the same speed of the hard disk, the effective capacity of the RAID 5 mode is the smallest capacity in the array of hard disk capacity in the number of hard drives minus one, the number of hard drives here to subtract one because one of the hard disk to hold the verification information.

RAID 5 Doubles the speed and ensures the security of the data, so this raid pattern is used in many high-end systems. 2.6 RAID 0+1﹝raid 10﹞ features, principles and applications;

RAID 0+1/raid 10, combined with the advantages of RAID 0 and RAID 1, is suitable for high speed demand, but also full fault tolerance, and of course, a lot of funds for the application

raid Strip "striped" access mode;

Among the raid systems that use data strip ﹝data Stripping﹞, there are two ways to access member disk drives:

Parallel Access ﹝paralleled Access﹞
Standalone Access ﹝independent Access﹞

Raid 2 and RAID 3 are taken in parallel access mode.

RAID 0, RAID 4, RAID 5, and RAID 6 are in standalone access mode.

http://blog.163.com/hlz_2599/blog/static/14237847420141610112127/