Read Redis C language design three: CRC data validation

Redis has problems with data errors when it comes to network transmissions and RDB backups, so it uses data validation. Includes a validation algorithm that uses the CRC64. The CRC test principle is actually the addition of an R-bit binary test code (sequence) after a P-bit binary data sequence, thus constituting a binary sequence with a total length of n=p+r bits; There is a particular relationship between the code attached to the data sequence and the contents of the data series. This particular relationship can be compromised if one or some of the bits in the data series are incorrectly caused by interference, for reasons such as this. Therefore, by examining this relationship, we can test the correctness of the data. The CRC algorithm inside Redis is implemented as follows:

uint64_t CRC64 (uint64_t CRC, Const unsigned char *s, uint64_t l) {    uint64_t J;    for (j = 0; J < L; j + +) {        uint8_t byte = s[j];        CRC = Crc64_tab[(uint8_t) CRC ^ Byte] ^ (CRC >> 8);    }    return CRC;}

One of the crc64_tab is the polynomial in each character: a code word length of 64bit CRC encoding. The commonly used generation polynomial is:

x^64 + x^4 + x^3 + x+ 1

From the design of C language, because the character Uchar is a value of 0-255, there are only 256 discrete values for this polynomial, so you can use a tab to save. So there is a crc64_tab. At the time of verification, we send out an original message and a CRC check code. As long as the receiving party to accept the information and check code to take the remainder can be seen whether there are errors. If the remainder is 0, then the transmission is not wrong. Although CRC is useful in error detection, CRC does not reliably validate data integrity because CRC polynomial is a linear structure that can be easily changed by changing the data to achieve CRC collisions, here is a more popular explanation, assuming a string with CRC checksum code in the transmission, If there is a continuous error, when the number of errors reached a certain number of times, then almost certainly there will be a collision (the value is incorrect but the CRC result is correct), but as the CRC data bits increase, the collision probability will be significantly reduced, such as CRC32 than CRC16 has more reliable verification, CRC64 will be more reliable than CRC32, of course, according to the ITU standard conditions

Read Redis C language design three: CRC data validation