Implementation principle of redis (2) -- dictionary

1. dict

2.1 data structure definition dict. h

// Hash table structure typedef struct dictht {dictentry ** table; // hash table array pointer unsigned long size; // hash table size unsigned long sizemask; // mask, unsigned long used is used for hash; // number of existing nodes} dictht; // hash table node Structure typedef struct dictentry {void * key; Union {void * val; uint64_t u64; int64_t s64;} V; // value, which can be pointer type, uint64, or int64 struct dictentry * Next; // points to the next node to form a single-chain table} dictentry; // define the dictionary typedef struct dict {dicttype * type; void * privdata; dictht HT [2]; int rehashidx; // The redistribution flag-1 indicates that int iterators are being redistributed; // redistribution progress} dict; // dictionary type // each dicttype stores a series of functions used to operate a specific dictionary, dictionary types of different purposes different typedef struct dicttype {// hash function unsigned int (* hashfunction) (const void * Key); // copy void * (* keydup) of key) (void * privdata, const void * Key); // copy void * (* valdup) (void * privdata, const void * OBJ) of value ); // key comparison int (* keycompare) (void * privdata, const void * key1, const void * key2); // key destruction void (* keydestructor) (void * privdata, void * Key); // void (* valdestructor) (void * privdata, void * OBJ) destroyed by value;} dicttype;

 

2.2 hash

Call the hashfunction method in type to calculate the hash value of the key. The hash algorithm used in redis is the murmurhash2 algorithm developed by Austin Appleby.

H = D-> type-> hashfunction (key)

Calculate the index value of the key. Search for the key in HT [0]. If the key is not found in the rehash process, search for it in HT [1 ].

Idx = H & D-> HT [Table]. sizemask

To solve hash conflicts, redis adopts the zipper method. keys with the same index value are stored in a single-chain table. Therefore, after determining the index value, you need to search in the corresponding single-chain table.

while(he) {            if (dictCompareKeys(d, key, he->key))                return -1;            he = he->next;        }

 

2.3 rehash

To ensure dictionary usage efficiency, redis adopts a regular rehash Mechanism for the dictionary structure, because rehash is a CPU-heavy operation, in order to avoid external non-response in the process, here we have made an incremental rehash optimization.

The rehash process is as follows:

1) Create a New empty hash table. The size is the first integer greater than or equal to 2N.

unsigned long i = DICT_HT_INITIAL_SIZE;if (size >= LONG_MAX) return LONG_MAX;while(1) {    if (i >= size)       return i;    i *= 2;}

 

2) re-calculate the hash and index values of keys in HT [0], index them to HT [1], and set the corresponding index values in HT [0] to null.

3) after all the data in HT [0] Is rehash to HT [1], set HT [1] to HT [0] and create a new HT [1]

Incremental rehash is the optimization of step 2. Only one index key is rehash at a time, and read/write operations are restricted during the rehash process:

1) read: Check HT [0] first. If not, search for it on HT [1 ].

2) Write: During the rehash process, the new key is only written to HT [1], and all keys corresponding to the index are re-hashed to HT [1.

At a certain time point, all keys in HT [0] will be re-hashed to HT [1.