String lookup algorithm (ii)

This is a creation in Article, where the information may have evolved or changed.

Continue with the previous string lookup algorithm. The last article said, the magic number 16777619 , at that time do not know what this is for, now almost know.

String hashes, which are often used with the FNV hashing algorithm. The FNV hashing algorithm is as follows: The string is treated as an integer of the length of the string, and the binary of the number is a prime. After the result is computed, the remainder is calculated according to the hash range, and the result is the hash result.

#define TRUE_HASH_SIZE ((u_int32_t)50000) /* range top plus 1 */#define FNV_32_PRIME ((u_int32_t)16777619)#define FNV1_32_INIT ((u_int32_t)2166136261)#define MAX_32BIT ((u_int32_t)0xffffffff) /* largest 32 bit unsigned value */#define RETRY_LEVEL ((MAX_32BIT / TRUE_HASH_SIZE) * TRUE_HASH_SIZE)u_int32_t hash;void *data;size_t data_len;hash = fnv_32_buf(data, data_len, FNV1_32_INIT);while (hash >= RETRY_LEVEL) {hash = (hash * FNV_32_PRIME) + FNV1_32_INIT;}hash %= TRUE_HASH_SIZE;

The following gives three prime numbers, which are used when the range is 32-bit, 64-bit, 128-bit, and 256-bit hash values, respectively. Of course, these three prime numbers are how to get, I certainly do not know. > Bit fnv_prime = 224 + + 0x93 = 16777619>-bit fnv_prime = + + + 0xb3 = 1099511628211>-bit fnv_ Prime = 288 + 0x3b = 309485009821345068724781371> bit fnv_prime = 2168 + + 0x63 = 3741444191567111470601433 17175368453031918731002211

Continue to see Golang code, string strings match with unsigned 32-bit integers, that is, 32-bit length, naturally, prime numbers need to choose 16777619. The result will be in accordance with the 32-bit largest integer to find the remainder, here, because the result exists in the uint32 inside, so out of range will be discarded, can also be considered to be redundant operation.

const primeRK = 16777619// hashstr returns the hash and the appropriate multiplicative// factor for use in Rabin-Karp algorithm.func hashstr(sep string) (uint32, uint32) {hash := uint32(0)for i := 0; i < len(sep); i++ {hash = hash*primeRK + uint32(sep[i])}var pow, sq uint32 = 1, primeRKfor i := len(sep); i > 0; i >>= 1 {if i&1 != 0 {pow *= sq}// 只有32位，超出范围的会被丢掉sq *= sq}return hash, pow}

The rest is the one mentioned in the previous article, the RK algorithm, based on the value obtained by the FNV hash, can be calculated in the o(1) time range to get the next string hash value. However, the FNV hashing algorithm ensures that in most cases the results of the hashes are evenly distributed within the specified range, but not all. So at the end of the time to determine whether the string is equal, will be added s[:n] == sep to ensure that the exact same, so, the complexity of the RK algorithm is accurate said o(m+n) .

if h == hashsep && s[:n] == sep {return 0}

Why is it not directly used s[:n] == sep to judge whether each string is equal when judged equal? Because of this, the complexity changes back o(m*n) .

###### references + FNV hashing algorithm "learning"-Bai yan+ "about the distribution of FNV hash results"-yasi_xi

Text link: string lookup algorithm (ii), reproduced please indicate the source!