Detailed explanation of RAID6 Structure

What is RAID]

The concept of RAID is everywhere on the Internet. The simplest principle is to allow the definition of a storage method to not affect all data when some data is missing, similar to error codes in the communication field. Different redundancy modes form different RAID categories.

Single Redundancy Model]

We need to first describe the RAID model with only one Disk redundancy (same idea as RAID3, RAID4, RAID5 ).

Suppose there are three blank pages of paper to record some numbers. In order to clearly record them, we will first draw tables of the same size on each page. Assume that one of the three pages may be lost under certain circumstances. In order to ensure that these numbers are completely and securely recorded in the above settings, we need to design redundant relationships that affect each other.

For example, the sequence of numbers to be recorded is 3, 14, 28, 4, 98, and 88. We can write the numbers to be recorded on pages 1st and 2nd in turn, and write their sum on pages 3rd. As shown in:

650) this. width = 650; "style =" background-image: none; border-right-0px; padding-left: 0px; padding-right: 0px; border-top-width: 0px; border-bottom-width: 0px; border-left-width: 0px; padding-top: 0px "title =" clipboard "border =" 0 "alt =" clipboard "src =" http://www.bkjia.com/uploads/allimg/131228/0232145a3-0.png "width =" 455 "height =" 98 "/>

It is easy to analyze that no matter which page of the three pages is lost, the data of the other two pages can be calculated. Obviously, even if there are more than three pages, the above-mentioned Design Record mode can also support the loss of any page of records.

However, this model will not be applied in practice because of the third row of data: 98 + 88 = 186. From the calculation of this row, in order to record the sum of the entire row of data, the size of the verification page must be greater than or equal to that of any data page. In fact, the total capacity of the checksum should be equal to the total capacity of all data pages. In other words, if a 10-page data is recorded, another 10 pages of space may be used for record verification, this is a completely meaningless solution (instead, it is better to copy two copies of all the data, that is, the RAID1 model)

Therefore, some engineers began to use other algorithms instead of addition to reduce the total capacity of the checksum. However, the Implementation Effect of the algorithm must be exactly the same as that of addition, and the calculation should be simple enough. The best solution is exclusive or.

XOR or bitwise operations, first of all, have excellent computing performance, without the need for more specialized processors. At the same time, the XOR algorithm fully satisfies the full ing between the positive operation and the inverse operation (that is, the result of the positive operation is unique, and the inverse operation result of the positive operation is also unique. What is this in mathematics? Forgive me for having a poor math, which is called for the time being) to satisfy the exchange law and the combination law. And, most importantly, it is abnormal or will not rise.

The storage record after rewriting with an exclusive or algorithm is as follows:

650) this. width = 650; "style =" background-image: none; border-right-0px; padding-left: 0px; padding-right: 0px; border-top-width: 0px; border-bottom-width: 0px; border-left-width: 0px; padding-top: 0px "title =" clipboard [1] "border =" 0 "alt =" clipboard [1] "src =" http://www.bkjia.com/uploads/allimg/131228/02321435Q-1.png "width =" 454 "height =" 98 "/>

We can clearly see that the checksum in the third row is no longer three digits, and no matter how many data members are there, the checksum capacity obtained with an exception or will not be accumulated.

For better generalization, we use "+" to represent this forward verification operation, and "-" to represent its inverse operation. In our initial description, it indicates the addition and subtraction of numbers, and "+" indicates the exclusive OR, and "-" is also the exclusive or (the inverse operation of the exclusive or is also the same or, so the calculator is simple, fast)

Suppose there are n storage members, each storage member is divided into several storage units, N-1 (mathematical subtraction, the following Dn-1 is the same) member disk for data, one member disk is verified. The verification relationships of each horizontal bar are as follows:

D1 + D2 + D3 +... + Dn-1 = P1

Dx = P1-D1-D2-D3-...-Dn-1 (D sequence exclusive Dx)

That is: Dx = P1 + D1 + D2 + D3 +... + Dn-1 (D sequence excludes Dx)

 

Multiple redundancy Models]

The preceding single redundancy only supports the absence of one storage member. In practice, higher redundancy may be required, and further algorithm improvement is required.

If you need to design a storage model, the storage can still perform complete operations without two members, and the best mathematical model may be a binary one-time equation, see the following equations:

AX + bY = c

DX + eY = f. A/d! = B/e

The above equation uses multiplication and division. At the same time, multiplication and division are completely reversible and meet the exchange law, combination law and allocation rate.

It is still difficult in addition. Ordinary mathematical multiplication will lead to the accumulation of verification capacity, so it is not advisable. Is there any multiplication or division that meets our requirements? Yes! --- It is based on the multiplication and division of galonia.

The mathematical concept is very abstract. Taking GF (2 ^ 8) as an example, we design a finite cyclic domain with only numbers 0-in the domain, A complete addition, subtraction, multiplication, division operation is designed between these numbers, and the result is still in these numbers. The operation result is unique and has no ambiguity.

Let's design an algorithm. For 2, if the Npower of 2 is greater than a certain value (primitive polynomial), let it get the remainder of this value (primitive polynomial, make sure that you fold back to the range 0-255. If the value ranges from 2 ^ 0, 2 ^ 1 ^ 2, until 2 ^, perform the operation according to this rule, the obtained values are in the range of 0 to and are not equal. In this way, a one-to-one ing relationship is formed, at the same time, it satisfies the calculation rules of multiplication/division between these power and result (for specific theories, we need to refer to mathematical theories such as group, ring, field, and finite field ).

On GF (2 ^ 8), there are 16 primitive polynomials that meet the conditions, respectively:

1X8 +X7 +X6 +X5 +X4 +X2 + 1 1 1111 0101 = 0x1F5

2X8 +X7 +X6 +X5 +X2 +X+ 1 1 1110 0111 = 0x1E7

3X8 +X7 +X6 +X3 +X2 +X+ 1 1 1100 1111 = 0x1CF

4X8 +X7 +X6 +X+ 1 1 1100 0011 = 0x1C3

5X8 +X7 +X5 +X3 + 1 1 1010 1001 = 0x1A9

6X8 +X7 +X3 +X2 + 1 1 1000 1101 = 0x18D

7X8 +X7 +X2 +X+ 1 1 1000 0111 = 0x187

8X8 +X6 +X5 +X4 + 1 1 0111 0001 = 0x171

9X8 +X6 +X5 +X3 + 1 1 0110 1001 = 0x169

10X8 +X6 +X5 +X2 + 1 1 0110 0101 = 0x165

11X8 +X6 +X5 +X+ 1 1 0110 0011 = 0x163

12X8 +X6 +X4 +X3 +X2 +X+ 1 1 0101 1111 = 0x15F

13X8 +X6 +X3 +X2 + 1 1 0100 1101 = 0x14D

14X8 +X5 +X3 +X2 + 1 1 0010 1101 = 0x12D

15X8 +X5 +X3 +X+ 1 1 0010 1011 = 0x12B

16X8 +X4 +X3 +X2 + 1 1 0001 1101 = 0x11D

0x11D is commonly used as the original polynomial of RAID 6, which means that if the Npower of 2 is greater than 0x11D, the remainder operation of xor is performed to ensure that the result is less than 0x256, in this way, all the values between 2 ^ 0 and 2 ^ 255 can be calculated.

The addition and subtraction on GF (2 ^ 8) is also an exclusive or algorithm (xor ).

Multiplication on GF (2 ^ 8) is polynomial multiplication, but xor is still required for primitive polynomials. There are multiple computational methods in algorithm design, however, on GF (2 ^ 8), the fastest recommended method is the look-up table method. You only need to calculate all the 0 ~ 255x0x0 ~ Generate a table with 255 values. You can directly look up the table when calculating the value. You can also use the logarithm lookup method to convert multiplication into addition operations. You need to use the lookup and addition methods in combination.

Division on GF (2 ^ 8) can be converted to the multiplication of its inverse element, that is, dividing a by d. Assume d corresponds to (m power of x ), then find the corresponding value d ', a divided by d, which is equal to a multiplied by d '.

RAID6]

After the addition, subtraction, multiplication, division, and division are all determined, a system of two yuan equations can be solved. Therefore, a RAID structure generated based on this principle is roughly as follows (taking five disks as an example, P is the first verification, Q is the second verification, and xn is the data ):

650) this. width = 650; "style =" background-image: none; border-right-0px; padding-left: 0px; padding-right: 0px; border-top-width: 0px; border-bottom-width: 0px; border-left-width: 0px; padding-top: 0px "title =" clipboard [2] "border =" 0 "alt =" clipboard [2] "src =" http://www.bkjia.com/uploads/allimg/131228/0232141422-2.png "width =" 367 "height =" 115 "/>

The reason for the spiral distribution of p and Q is to balance the load on all disks. If it is hard to understand, you can put P and Q in a column separately. The meaning of the algorithm is the same.

Again, the +,-, *, And/operations mentioned below refer to addition, subtraction, multiplication, and division based on GF (2 ^ 8 ).

The P value is equal to the sum of all units on the same line (band. It can also be understood as the sum after 1 is multiplied by each unit, such as the P of the first strip:

P = x1 + x2 + x3, that is, P = 1 * x1 + 1 * x2 + 1 * x3

On GF (2 ^ 8), each polynomial corresponds to a 0 ~ The value of 255, that is, d0 corresponds to the 0 power of Polynomial X, and d1 corresponds to the 2 power of Polynomial X. It is expanded according to polynomial, and X is in hexadecimal format. Therefore, d0 = 1, d1 = 2, d2 = 4, d3 = 8, and so on, as shown in the following table:

650) this. width = 650; "style =" background-image: none; border-right-0px; padding-left: 0px; padding-right: 0px; border-top-width: 0px; border-bottom-width: 0px; border-left-width: 0px; padding-top: 0px "title =" clipboard [3] "border =" 0 "alt =" clipboard [3] "src =" http://www.bkjia.com/uploads/allimg/131228/0232141430-3.png "width =" 552 "height =" 314 "/>

Return to the RAID structure, where the Q value equals to the result of multiplying each numeric cell by their corresponding dn and then accumulating it. The dn can be agreed to ensure that no duplicate dn occurs in the operation of the same band, for example, the Q in the first line can be:

Q = d1 * x1 + d2 * x2 + d3 * x3

In this way, for each row (strip), we can ensure that any two units are lost and can be calculated (for clarity, the following calculation directly changes the subtraction to addition ):

Take the first behavior as an example:

A) if both P and Q are lost, the data zone will not be affected. x1, x2, and x3 can be read and written normally.

B) If xn is lost, it can be calculated based on P or Q (in reality, because P is faster, P verification is usually used to calculate the loss of xn)

C) if P and xn are lost, P values are not processed. If x2 is lost, the Q value is defined.

Q = d1 * x1 + d2 * x2 + d3 * x3

=> D2 * x2 = Q + d1 * x1 + d3 * x3

=> X2 = (Q + d1 * x1 + d3 * x3) * x253; // Note: x253 is the reverse element of x253, which can be obtained through the Table query.

D) If two x values are lost, you can solve the problem based on the standard solution of the binary one-time method. Assume that the loss is x1, x3:

P = x1 + x2 + x3

Q = d1 * x1 + d2 * x2 + d3 * x3

=> X1 = P + x2 + x3

=> Q = d1 * (P + x2 + x3) + d2 * x2 + d3 * x3

=> Q = d1 * P + d1 * x2 + d1 * x3 + d2 * x2 + d3 * x3

=> Q = d1 * P + d1 * x2 + d2 * x2 + d1 * x3 + d3 * x3

=> Q + d1 * P + d1 * x2 + d2 * x2 = (d1 + d3) * x3

=> X3 = (Q + d1 * P + d1 * x2 + d2 * x2)/(d1 + d3)

After the lookup table method is used to obtain the reverse dn of (d1 + d3), we can see that

X3 = (Q + d1 * P + d1 * x2 + d2 * x2) * dn

Then obtain x1 Based on P = x1 + x2 + x3 to complete the completion of all data.

This article is from the "Zhang Yu (data recovery)" blog, please be sure to keep this source http://zhangyu.blog.51cto.com/197148/1134820