Redis Learning Note Two

Double linked list

As a common data structure, the double-ended linked list is used very much inside Redis: It is both the bottom of the Redis list structure
Layer implementation, which is also used by a large number of Redis modules to build other features of Redis.

Application

• Implement a Redis list type;

A double-ended linked list is also one of the underlying implementations of the Redis list type, when the key to the list type is manipulated-such as executing
Rpush, Lpop, or Llen commands, it is possible for the program to operate at the bottom of a double-ended linked list.

The Redis list uses two types of data structures as the underlying implementation:

1. Double-ended linked list

2. Compression list

Because the double-ended list uses more memory than the compression list, when you create a new list key, it takes precedence over the pressure
The indented list is implemented as the underlying, and is converted from the compression list implementation to the double-ended linked list when necessary.

• The construction of Redis's own function

In addition to implementing the list type, the doubly linked list is also used by many Redis internal modules:

1. The transaction module uses a double-ended linked list to sequentially save the input commands;
2. The server module uses a double-ended linked list to hold multiple clients;
3. The subscription/Send module uses a double-ended linked list to hold multiple clients of the subscription pattern;
4. The event module uses a double-ended list to hold the time event;

• The realization of double-end linked list

The implementation of the double-ended list consists of the ListNode and list two data structures, which shows the two structures that comprise a
Example of a double-ended list:

Iterators

Redis implements an iterator for a double-ended list that iterates over a doubly-linked list in two directions:
• Proceed along the next pointer of the node, iterating from the table head to the end of the table;
• Move along the prev pointer of the node, iterating from the end of the table to the table head;

typedef struct LISTITER {
Next node
ListNode *next;
Iteration Direction
int direction;

} Listiter;

The direction record iteration should start there:
• If the value is Adlist.h/al_start_head, then the iterator executes an iteration from the header to the footer;
• If the value is Adlist.h/al_start_tail, then the iterator executes an iteration from the end of the table to the table header;

Dictionary

It is an abstract data structure, consisting of a set of key-value pairs (key-value pairs), the keys of each key-value pair are different, the program can add new key-value pairs to the dictionary, or based on the key to find, update or delete operations.

The application of dictionaries

1. Implement the database key space (key spaces);

Redis is a key-value pair database, and the key-value pairs in the database are saved by a dictionary: each database has a relative
The dictionary, which is called the key space.

When a user adds a key-value pair to the database (regardless of the type of key-value pair), the program adds the key-value pair to the key-null
When a user deletes a key-value pair from the database, the program removes the key-value pair from the key space, and so on.

2. One of the underlying implementations used as a hash type key;

The hash type key for Redis uses the following two data structures as the underlying implementation:
1. Dictionaries;
2. Compression list;

Because the compression list is more memory-efficient than a dictionary, the program uses a compressed list as the underlying when creating a new hash key.
Implementation, the program will convert the underlying implementation from a compressed list to a dictionary when necessary.

Redis chooses an efficient and easy-to-implement hash table as the underlying implementation of the dictionary.

The implementation of the dictionary:

typedef struct DICTHT {
Hash table node Pointer array (commonly known as buckets, buckets)
Dictentry **table;
Size of the pointer array
unsigned long size;
The length mask of the pointer array, used to calculate the index value
unsigned long sizemask;
Hash shows the number of nodes
unsigned long used;
} dictht;

The table property is an array, and each element of the array is a pointer to the DICTENTRY structure.

Each dictentry holds a key-value pair, and a pointer to another dictentry structure:

/*
* Hash Table node
*/
typedef struct DICTENTRY {
Key
void *key;
1.3. Dictionaries 15
Redis design and implementation, first edition
Value
Union {
void *val;
uint64_t U64;
int64_t S64;
} V;
Chain Backward Node
struct Dictentry *next;
} dictentry;

In the dictionary example, although the dictionary has created two hash tables, it is using only the No. 0 hash table, which indicates that the dictionary is not in the rehash state.

How to deal with collisions:

Suppose there are now three borrowing points for the hash table as follows:

For a new key-value pair Key4 and value4, if the hash value of the Key4 is the same as the hash value of key1, then they
A collision will occur on the No. 0 index of the hash table. By connecting the Key4-value4 and key1-value1 two key-value pairs with a linked list, you can solve the collision problem:

The rehash action is triggered when a new key-value pair is added:

For hash table dictht, which uses the chain address method to solve the collision problem, the performance of the hash table depends on its size (size
property) and the ratio of the number of nodes it holds (the Used property):
• The performance of the hash table is best when the ratio is 1:1;
• If the number of nodes is much larger than the size of the hash table, then the hash table will degenerate into multiple linked lists, hash tables
The performance advantage of itself no longer exists;

In order to maintain good performance in the case of dictionary key-value pairs, the dictionary needs to perform rehash operations on the hash table (Ht[0]) used: To expand the hash table without modifying any key-value pairs, and to maintain the ratio at 1:1

Around.

Dictadd hash Table ht[0] is checked each time a new key value pair is added to the dictionary, for Ht[0]
The size and used properties, if the ratio between them ratio = Used/size satisfies any one of the following conditions,
The rehash process will be activated:
1. Natural Rehash:ratio >= 1, and the variable dict_can_resize is true.
2. Force Rehash:ratio is greater than the variable dict_force_resize_ratio (in the current version,
The value of Dict_force_resize_ratio is 5).

Note: When will the Dict_can_resize be false?

A database is a dictionary, the hash type key in the database
is also a dictionary when Redis uses a child process to perform a background persistence task on a database, such as executing a bgsave
or bgrewriteaof), in order to maximize the use of the system's copy on write mechanism, the program temporarily
Dict_can_resize is set to false to avoid natural rehash, which reduces the memory touch (touch) of the program.
When the persistence task is complete, the dict_can_resize will be set to true again.

The rehash operation of a dictionary is essentially performing the following tasks:
1. Create a larger ht[1]->table than the ht[0]->table;
2. Migrate all key-value pairs in the ht[0]->table to ht[1]->table;
3. Empty the data of the original ht[0] and replace the ht[1] with the new ht[0];
After the above steps, the program will not change the original key value on the basis of data, increase the size of the hash table.

Progressive type Rehash

Progressive rehash is mainly performed by the _dictrehashstep and dictrehashmilliseconds two functions:
_dictrehashstep is used for passive rehash of database dictionaries and dictionary of hash keys;
Dictrehashmilliseconds is performed by the Redis Server Common task program (server cron Job), with
Active rehash of the database dictionary;

Shrinking of the Dictionary

The contraction rehash is almost identical to the operation shown above for the extended rehash, which performs the following steps:
1. Create a ht[1]->table smaller than the ht[0]->table;
2. Migrate all key-value pairs in the ht[0]->table to ht[1]->table;
3. Empty the data of the original ht[0] and replace the ht[1] with the new ht[0];
Extending rehash and shrinking rehash perform exactly the same process, a rehash is to extend or shrink the dictionary, the key is
The size of the newly allocated ht[1]->table:

The shrink rule of a dictionary is defined by the Redis.c/htneedsresize function:
/*
* Check if the dictionary usage is lower than the minimum allowable rate for the system
**
If yes, return 1, otherwise return 0.
*/
int htneedsresize (Dict *dict) {
Long long size, used;
Hash table number of nodes used
Size = Dictslots (dict);
Hash Table Size
used = Dictsize (dict);
When the hash table size is greater than dict_ht_initial_size
and the dictionary fill rate is lower than redis_ht_minfill
Returns 1
Return (size && used && size > Dict_ht_initial_size &&
(Used*100/size < Redis_ht_minfill));
}

By default,the value of Redis_ht_minfill is 10, that is, when the fill rate of the dictionary is less than 10%, the process
The dictionary can be shrunk by the sequence.

The other operations of the dictionary and the iteration of the dictionary ..... Slightly

Jumping table (very cool kind of table) .... Slightly

Operation has Zadd Zrange and so on.

Redis Learning Note Two