In-depth introduction to the Redis-redis underlying data structure (below), simple introduction to redis-redis

In-depth introduction to the Redis-redis underlying data structure (below), simple introduction to redis-redis
Overview:

　　　　Learning to use Redis doesn't actually need to study the implementation of its underlying data. We only need to know which common data types he has, and then use them skillfully, so that we can have a good grasp of the Redis tool. However, this learning method is only suitable for getting started with Redis. "To do this, you must first sharpen the tool." to make good use of Redis, you must have a deep understanding of how Redis is implemented at the underlying layer, we can make the right choice only when selecting the data structure.

In the previous blog "go deep into Redis-redis underlying data structure (I)", we have explained the dynamic strings, linked lists, and dictionaries in Redis.

Here we will briefly review their features:

1. Dynamic string SDS: different from C language strings, it has good scalability. It has better performance than C language strings in obtaining string lengths and modifying strings to prevent cache overflow.

2. Linked List: stores object information in sequence and has attributes used to cache the length of linked list. This function has good performance in inserting and deleting objects to avoid loops.

3. dictionary: the key-value storage method. The key storage is determined by hash value calculation. When the storage capacity is too large, the dictionary size will be re-allocated through rehash.

 

 

5. Skip table 5.1 Overview

　　　　A skip table is an ordered data structure. It maintains multiple pointers to other nodes in each node to quickly access the node. A skip table is a type of randomized data. A skip table stores elements in a hierarchical linked list in an orderly manner, efficiency is comparable to that of the Balance Tree-operations such as search, deletion, and addition can all be completed within the expected logarithm time. Compared with the Balance Tree, the implementation of skip tables is much simpler and more intuitive.

Redis only uses the jump table in two places. One is the implementationSorted set key, The other is inCluster nodeUsedInternal Data Structure.

 

　　

5.2 skip table definition

Let's take a look at the complete structure of the entire skip table:

Redis jump table consists of two parts: zskiplist and zskiplistNode)

　　　　

5.2.1 zskiplistNode data structure:

Typedef struct zskiplistNode {
// Layer struct zskiplistLevel {
// Forward pointer struct zskiplistNode * forward;
// Span unsigned int span;} level [];
// Return pointer struct zskiplistNode * backward;
// Double score; // member object robj * obj ;}

 

1. layer: The level array can contain multiple elements. Each element contains a pointer to another node.

2. Forward pointer: The forward pointer pointing to the end of the table.

3. Span: used to record the distance between two nodes

4. Backward pointer: used to access a node from the end of the table to the header.

5. values and members: All nodes in the skip table are sorted by values from small to large. The member object points to a string, which stores an SDS value.

 

5.2. zskiplist data structure:

Typedef struct zskiplist {// header node and table tail node structz skiplistNode * header, * tail; // Number of table nodes unsigned long length; // int level for the node with the maximum number of layers in the table;} zskiplist;

　　　　

From the structure diagram, we can clearly see that the header and tail point to the head and end nodes of the jump table respectively. Level is used to record the maximum number of layers, and length is used to record the number of nodes.

 

5.3 conclusion

• 　A skip table is one of the underlying implementations of an ordered set.
  
• It consists of two structures: zskiplist and zskiplistNode.
• The height of each hop table node is a random number between 1 and 32.
• In the same skip table, multiple nodes can contain the same score, but the objects of each node must be unique.
• Nodes are sorted by the value size from large to small. If the value is the same, nodes are sorted by the size of member objects.

 

 

 

6. Intset)

 

6.1 Overview

Redis Design and ImplementationDefine the integer set as follows: "The Integer Set isSet CreationOne of the underlying implementations of, when a set contains only integers, and the number of elements in this set is small, redis uses the Integer Set intset as the underlying implementation of the set ."

We can understand the Integer Set in this way. It is actually a special set. The data stored in it can only be an integer, and the data volume cannot be too large.

 

6.2 Integer Set implementation

　　　　

Typedef struct intset {// encoding method uint32_t enconding; // number of elements contained in the Set uint32_t length; // The int8_t contents [];}

 

We observe the structure of a complete Integer Set:

　　

1. encoding: used to define the encoding method of the Integer Set

2. length: used to record the number of variables in an Integer Set

3. contents: array used to save elements. Although we can see in the data structure diagram that intset defines the array as int8_t, the type of elements saved in the array depends on encoding.

 

 

6.3 integer collection upgrade

In the preceding data structure, we can see that intset sets the encoding method in the Integer Set by default. However, when the stored integer does not conform to the encoding format in the Integer Set, you need to use the upgrade policy in Redis to solve the problem.

Update the Integer Set in Intset and add new elements in three steps:

1. Expand the space of the underlying array of the Integer Set Based on the type of the new element and allocate space for the new element.

2. Convert all existing elements of the underlying array into a new encoding format and allocate space.

3. Add new elements to the layer Group

For example, we now have the following Integer Set:

 

 

We need to insert a 32-bit integer, which obviously does not match the Integer Set. We will convert the encoding format and allocate space for the new elements:

Step 2: Convert the original data type to the same type as the new data: (data after Space reallocation)

　　　　

Part 3: Add new data to the array:

　　

 

6.3.1 benefits of integer collection upgrade

1. Improved flexibility

2. Memory saving

 

6.4 conclusion

An integer set is one of the underlying implementations of set creation.

The underlying implementation of an integer set is an array. This array stores the set elements in an ordered, non-repeating paradigm. When necessary, the program will change the array type based on the newly added element type.

The upgrade operation provides operation flexibility for integer sets and saves as much memory as possible.

The Integer Set only supports upgrade and does not support downgrade.

7. compress the list

 

7.1 Overview

Compressing a list is one of the underlying implementations of the list key and hash key. When a list key only sweats a small number of list items, and each list item is either a small integer or a short string, redis uses the compressed list as the underlying implementation of the list key.

 

　　

7.2 composition of the compressed list

A compression list consists of the following:

　　

1. zlbytes: used to record the memory bytes occupied by the entire compression list

2. zltail: records the number of bytes between the end node and the start address of the compressed list.

3. zllen: records the number of nodes in the compression list.

4. entryX: nodes included in the List

5. zlend: used to mark the end of the compressed list

 

 

 

 

 

7.3 conclusion

Compressed list is a sequential data structure developed to save memory

The compressed list is used as one of the underlying implementations of the list key and hash key.

The compressed list can contain multiple nodes. Each node can save an array of bytes or an integer.

Adding a new node to the compression list may cause a chained update operation.