Redis Storage Architecture Design

Redis is a large dictionary that contains a lot of key-value pairs, and this dictionary supports a very rich set of value that can be used for strings, hash tables, lists, collections, and ordered sets, based on these types of rich value, extending powerful operations such as Hmset, Lpush, SADD, etc.


1. Dictionaries
A dictionary is the most basic data structure of Redis, and a dictionary is a db,redis that supports multiple db
The Redis dictionary is implemented with a hash table, and the method used for the collision problem is "chain address Method", which concatenates multiple nodes with identical hash values to resolve the conflict problem.
The problem with the "chain address method" is that when the collision is severe, the performance degradation is serious, for example: when there are n data, M slots, if m=1, then the entire hash table degenerate into a linked list, query complexity O (n)
To avoid hash collision attacks, Redis randomly hashes hash table seeds

The Redis scheme is "double buffer", where the normal process uses a buffer, when the collision is detected violently (judging by the current number of slots and the number of keys), assigning a larger buffer, and then gradually migrating the data from the old buffer to the new buffer.

2. hash table in Redis

As mentioned earlier, Redis is a key/value storage system, and people who have studied data structures know that the simplest data result of key/value is a hash table (and of course, other ways, such as B-tree, binary balance tree, etc.), The performance of a hash table depends on two factors: the size of the hash table and the method of resolving the conflict. These two are contradictory: the hash table is large, then the conflict is few, but the memory is too large, and the hash table is small, the memory uses less, but the conflict is many, the performance is low. A good hash table will weigh these two factors to make memory usage and performance as low as possible. In Redis, a hash table is the basis for all other data structures, and for all other structure, such as String,set,sortedset, which is stored in value in the hash table, this can easily be done by setting the type of value to void*. In this paper, the design idea and implementation method of the hash table in Redis are introduced in detail.

3. Design idea of redis hash table

The image below is from the Taobao "Redis Memory storage Structure Analysis" of the image, the main description of the Redis hash table organization way.

In Redis, the hash table is called a Dictionary (dictionary), which uses a typical chained conflict resolution method, namely: when there are multiple Key/value key mappings (each pair of key/value is saved, it is preceded by a hash-like (key) MOD The N method calculates a value to determine the location of its corresponding hash table, saves the value in the form of a single-linked list, and in order to control the memory size of the hash table, Redis uses a dual hash table (Ht[2]) structure, and gradually expand the hash table capacity (bucket size) of the strategy, namely: just beginning, hash table ht[0] bucket size is 4, hash table ht[1] bucket size is 0, to be serious conflict (Redis has a certain condition), ht[1] in the size of the bucket ht[0] twice times, and gradually (note the word: " Gradually ") migrate the elements in the hash table ht[0] (called" Hash Again ") to ht[1], until all elements in ht[0] are migrated to Ht[1], then ht[1] to Ht[0] (here is just the C-language address Exchange), then repeat the process above.

The Redis dictionary structure is as follows:
[CPP]   View plain copy typedef struct dict {       dicttype *type ;        void *privdata;         dictht ht[2]; //Double buffer       int rehashidx;        int iterators;  } dict;      typedef struct  dictht {       dictentry **table; //hash chain List         unsigned long size;       unsigned long  sizemask;       unsigned long used;  } dictht;      //Data nodes <K,V>   typedef struct dictentry {        void *key;       union {            void *val;           uint64_t  u64;           int64_t s64;        } v;       struct dictEntry *next;  }  dictEntry;   Redisobject is a real storage of Redis's various types of structure, with the following content:
[CPP]View plain copy typedef struct REDISOBJECT {unsigned type:4;//logic type unsigned notused:2; /* Not used */unsigned encoding:4;        Physical storage type unsigned lru:22;       /* LRU time (relative to Server.lruclock) */int refcount;  void *ptr;   Specific data} RobJ; Where type is the type of logic supported by Redis, including:
[CPP]View Plain copy #define REDIS_STRING 0 #define REDIS_LIST 1 #define REDIS_SET 2 #define REDIS_ZSET 3 #define REDIS _hash 4 enconding is physical storage, a logical type can use different storage methods, including: