[Turn]reed Solomon Erasure codehttp://peterylh.blog.163.com/blog/static/12033201371375050233/Erasure code is a common data redundancy technology in storage domain, and the erasure code can obtain higher data reliability with smaller data redundancy compared with multi-copy replication. Reed Solomon Coding is an erasure code commonly used in storage, and its basic principles are as follows: given n data blocks D1, D2,..., dn,n and a positive integer m, RS generates m check blocks based on n data blocks, C1, c2,..., cm. For arbitrary N and M, any n blocks from n raw data blocks and M-check blocks can decode the original data, that is, RS tolerate the maximum of M data block or check block at the same time lost (erasure code can only tolerate data loss, can not tolerate data tampering, erasure code is the name of this).
Coding PrincipleRS encodes word for encoding and decoding units, a large chunk of data is split into word with a length of w (the length of w is typically 8 or 16 bits), and then the word is decoded. So the coding principle of the data block and word encoding principle is not different, for the exposition convenient, after the text of the Variable di, CI will represent a word. First, the input data is treated as vector d= (d1,d2, ..., DN), the encoded data is considered as vectors (D1, D2,..., Dn, C1, C2,.., Cm), and RS encoding can be visualized as a matrix operation of 1. The leftmost is the encoding matrix, the upper part of the matrix is the unit array (n rows n columns), below is the Vandermonde matrix B (m row n column), Vandermode Matrix 2, line I, the original value of column J is j^ (i-1). The reason why the Vandermonde matrix is adopted is that RS data recovery algorithm requires the arbitrary n*n sub-matrix of the encoding matrix to be reversible.Figure 1: Encoding Operation Figure 2:vandermode matrix
principles of data recoveryRS can tolerate a maximum of M delete errors. The process of data recovery principle is as follows: (1) Delete the missing data block and the missing code block corresponding line from the coding matrix.assuming that D1 and C2 are lost, we get the following B ' and equation according to the RS coded operation equation shown in Figure 1. (2) Since B ' is reversible, multiply B ' inverse matrix on both sides. (3) The calculation formula of the original data d is obtained as follows (4) to D re-encode, get lost check code matrix inverse using Gaussian elimination method, need to do real subtraction arithmetic, can not be used for binary data of the word length W.to solve this problem, RS uses the arithmetic law defined in gamma Lowa GF (2^w). GF (2^w) fields have 2^w values, each corresponding to a polynomial less than w, so that the arithmetic on the domain is converted to the operation of the polynomial space [2]. Addition in GF (2^w) domain is XOR, multiplication is special, need to maintain two size of 2^w-1 table: Log table Gflog, anti-log table Gfilog. multiplication formula: A * b = Gfilog (Gflog (a) + Fglog (b))% (2^w-1)
Summaryfeatures of Rs:(1) Low redundancy, high reliability. (2) high cost of data recovery. when the data block or code block is lost, RS needs to read n blocks and check blocks to recover the data, and the efficiency of data recovery also restricts the reliability of Rs to some extent. (3) high cost of data update. Data updates are equivalent to recoding, expensive, and therefore often for read-only data, or cold data. (4) RS encoding relies on two 2^w-1 size log tables, usually only 16-bit or 8-bit word length, can not take full advantage of the computing power of the 64-bit server, the implementation may have to do some optimization. Reference Documents:[1]james S. Plank. Erasure Codes for Storage application. [2]james S. Plank. A Tutorial on Reed-solomon Coding for fault-tolerance in Raid-like Systems
Go Reed Solomon Erasure Code
