Redis Bottom Quest (v): Redis Object

In the previous articles, we learned about all the major data structures used by Redis, such as simple dynamic strings (SDS), double-ended lists, dictionaries, compressed list, integer collections, and so on.

Redis does not directly use these data structures to implement key-value pairs of databases, but instead creates an object system based on these data structures, which contains five types of objects, such as String objects, list objects, hash objects, collection objects, and ordered collection objects. Each of these objects uses at least one of the data structures we described earlier.

With these five of these five different types of objects, Redis can determine whether an object can execute a given command, depending on the type of object, before executing the command. Another benefit of using objects is that we can set up a number of different data structure implementations for the objects for different usage scenarios, thus optimizing the efficiency of objects for use in different scenarios.

In addition, the Redis object system implements a memory recovery mechanism based on the reference counting technique, and when the program no longer uses an object, the memory used by the object is automatically freed, and Redis implements the object sharing mechanism by reference counting technology, which can, under appropriate conditions, Save memory by having multiple database keys share the same object.

Finally, the Redis object has access time-logging information that can be used to calculate the idle time of the database keys, and when the server has the Maxmenory feature enabled, those keys that are larger when idling may be removed by the server first.

Next we will learn about the Redis objects and features mentioned above.

Types and encodings of objects

Redis uses objects to represent keys and values in a database, and each time we create a new key-value pair in a Redis database, we create at least two objects, one object as the key for the key-value pair, and the other for the value of the key-value pair.

Each object in Reids is represented by a redisobject structure with the following three properties related to saving data

struct redisobject{    // type    unsigned type:4;     // Coding    Unsigned encoding:4;     // Pointer to the underlying implementation data structure    void *ptr;    ... ..}robj

Type

The Type property of the object records the types of the object, and the value of this property is one of the following tables.

Key-value pair keys for a Redis database The key is always a string object, and the value can be either a string object, a list object, a hash object, a collection object, or one of the ordered collection objects.

So when we execute the type command, the command returns the result of the database key corresponding to the type of the value object, not the type of the key object.

Set msg "Hello"

Type msg//string

The output corresponding to the different value types is as follows

Coding and the underlying implementation

The object PTR pointer points to the underlying implementation data structure of the object, which is determined by the object's Encoding property.

The Encoding property records the encoding used by the object, that is, what data structures the object uses as the underlying implementation of the object, which can be one of the constants listed below

Each type of object uses at least two different encodings, and each type of object can use the following encoding

The object Encoding command can view the encoding of a database value object

Object encoding msg//"Embstr"

Lists the object encoding output for different encoded objects

Using the Encoding property to set the encoding used by an object rather than associating a fixed encoding for a particular type of object object greatly improves the flexibility and efficiency of redis, because Redis can set different encodings for an object based on different usage scenarios. This optimizes the efficiency of the object in a given scenario.

For example, when a list object contains fewer elements, Redis uses a compressed list as the underlying implementation:

1 because the compression list is more memory-efficient than the double-ended lists, and when the number of elements is low, a compressed list saved in memory in contiguous chunks is Zhaoju to the cache more quickly than a double-ended linked table.

2 as the list object contains more and more elements, the advantage of using a compressed list to save elements fades away, and the object moves the underlying implementation from the compression list to the more powerful, double-ended linked lists that are more suitable for storing a large number of elements.

Other types of objects are also optimized for type by using a number of different encodings.

Next, we'll learn about Redis's five different types of objects, the encoding they use, the conditions of conversion, and how the same command is implemented on a variety of different encodings.

String Object

The encoding of a string object can make Int,raw or embstr.

If a string object holds an integer value and the integer value can be represented by a long type, the string object saves the integer value inside the PTR attribute of the string object structure (converts void * to long) and sets the encoding of the string object to int.

For example, execute the following command

Set Number 10086

Object encoding number//"int"

Structure such as

If the string object holds a string value, and the string value is longer than 32 bytes, the string object will use a simple dynamic string (SDS) to hold the string value and set the object's encoding to Raw.

If the string object holds a string value and the string value is less than or equal to 32 bytes, the string object will save the string value using EMBSTR encoding.

EMBSTR encoding is an optimized encoding that is designed to hold short strings, which, like raw encoding, uses the REDISOBJECT structure and SDSHDR structure to represent string objects. However, the raw encoding calls two memory allocation functions to create the redisobject structure and the SDSHDR structure, and EMBSTR encoding allocates a contiguous space by calling a memory allocation function once, with the redisobject and SDSHDR structures once in the space. Such as

EMBSTR encoded string objects produce the same effect as raw-encoded string objects when executing commands, but using EMBSTR-encoded string objects to hold short string values has the following benefits:

1 EMBSTR encoding the number of memory allocations required to create a string object is reduced from two raw-encoded to one-time.

2 Releasing a EMBSTR encoded string object requires only one call to the memory-release function, and the release of the raw-encoded string object requires the two-time memory-release function to be called.

3 because all the data of a EMBSTR encoded string object is stored in contiguous memory, this encoded string object makes better use of the benefits of caching than raw-coded string objects.

Finally, floating-point numbers that can be represented by a long double type are also stored as string values in Redis. If we were to save a floating-point number into a string object, the program would first convert the floating-point to a string value, and then save the converted string value.

The INT-encoded string object and the EMBSTR encoded string object are converted to raw-encoded string objects in the case that the condition is met.

Implementation of STRING commands

List objects

The encoding of the list object can make Ziplist or LinkedList.

Ziplist encoded list objects use a compressed list as the underlying implementation, and each compressed list node (entry) holds a list element. is the Ziplist encoded list object, the red box is stored in the data.

On the other hand, the LinkedList encoded list object uses a double-ended chain table as the underlying implementation, each of which holds a string object, and each string object holds a list element, such as

Note that the LinkedList encoded list object contains multiple string objects in the underlying double-ended linked table structure, and the behavior of such nested string objects will appear in the hash objects, collection objects, and ordered collection objects that are described later. The string object is the only one of the Redis five types of objects that will be nested by other four-type objects.

To simplify the representation of a string object, we use the Stringobject typeface to represent the string object in the complete format as follows

Encoding Conversion

The list object uses Ziplist encoding when the list object can meet the following two conditions:

1 The length of all string elements saved by the list object is less than 64 bytes

2 The number of elements saved by the list object is less than 512;

(the upper limit value of the above two conditions can be modified)

Implementation of the LIST command

Hash object

The encoding of a hash object can be either Ziplist or Hashtable.

The Ziplist encoded hash object is implemented as the bottom of the compression list, and whenever a new key-value pair is added to the hash object, the program saves the key to the compressed list node, pushes it into the compressed list footer, and then pushes the compressed list node that holds the value into the compressed list footer, so:

1 The two nodes that saved the same key-value pair are always close together, the node that holds the key is in front, and the node that holds the value is behind

2 The key-value pairs that are added to the hash object are placed in the header direction of the compressed list, and the key-value pairs that are later added to the hash object are placed in the footer direction of the compressed list.

As an example, execute the following command

Hset Profile Name "Tom"

Hset Profile Age 25

Hset profile Career "Programmer"

Well, he's coding back to ziplist, corresponding to the hash object such as

On the other hand, the Hashtable encoded hash object uses a dictionary as the underlying implementation, and each key-value pair in the hash object uses a dictionary key-value pair to hold

1 each key of the dictionary is a string object that holds the key of the key-value pair

2 each value of the dictionary is a string object that holds the value of a key-value pair

In the example above, the corresponding Hashtable encoded hash object, such as

Encoding Conversion

When a hash object can satisfy two conditions at a time, the hash object uses Ziplist encoding

1 The key and value string lengths of all key-value pairs saved by a hash object are less than 64 bytes.

2 The number of key-value pairs saved by a hash object is less than 512

Hash objects that do not meet these two conditions need to use Hashtable encoding (the upper values of these two conditions can be modified in the Redis configuration.) ）

Implementation of the hash command

Because the hash key value is a hashed object, all commands for the hash key are built on the hash object, and the following table lists some of the hash keys commands and how they are implemented under different encoding objects.

Here alone the Hget method, although Ziplist encoding, Ziplistfind method complexity is O (N), but the total number of key-value pairs is less than 256, execution speed is also very fast. When the key value pair is many, the implementation of Hashtable, the complexity is O (1), the speed of execution is also very fast.

Collection Object

The encoding of the collection object can be either Intset or Hashtable.

The Intset encoded collection object uses the certificate collection as the underlying implementation, and all the elements contained in the collection object are stored in the integer collection.

On the other hand, the Hashtable encoded collection object uses a dictionary as the underlying implementation, each key of the dictionary is a string object, each string object contains a collection element, and the value of the dictionary is all set to NULL.

Encoding Conversion

When a collection object can satisfy two conditions at a time, the object uses Intset encoding:

1 All elements saved by the collection object are integer values

2 The number of elements saved by the collection object does not exceed 512

A collection object that cannot satisfy both conditions uses Hashtable encoding.

Implementation of the collection command

Ordered collection objects

The encoding of an ordered set can be either Ziplist or skiplist.

Ziplist encoded Compression list objects use a compressed list as the underlying implementation, each set element is saved with two gold-loving compressed list nodes, the first node holds the element's members, and the second element holds the element's score.

The collection elements in the compression list are sorted by the score from small to large Venus, the elements with smaller scores are prevented from approaching the heading of the table, while the elements with larger scores are blamed to prevent the direction near the end of the table, such as.

The Skiplist encoded ordered collection object uses the ZSET structure as the underlying implementation, and a ZSET structure contains both a dictionary and a skip table:

struct zset{    *ZSL;     *dict;} Zset

ZSL jumping table in Zset structure saves all collection elements from small to large, and each skip table node holds a collection element: The object property of the Skip table node holds the members of the element, and the score attribute of the Jump table node preserves the element's score. Through this jumping table, the program can do the scope of the ordered set of operations, such as Zrank, Zrange and other commands are based on the Jumping table API to achieve.

In addition, the Dict dictionary in the ZSET structure creates a mapping from member to score for an ordered set, and each key-value pair in the dictionary holds a collection element: The dictionary key holds the member of the element, and the value of the dictionary preserves the element's score. Through this dictionary, the program can use O (1) complexity to find the score of a given member, the Zscore command is based on this feature, and many other ordered collection commands in the implementation of the internal use of this feature.

The members of each element of an ordered collection are a string object, and the score for each element is a double type of floating-point number. It is worth mentioning that although the Zset structure uses both a skip table and a dictionary to hold an ordered set element, both data structures share the same members and scores through pointers, so using a skip table and a dictionary to save the collection element does not result in any duplicate members or scores, nor does it waste additional memory.

Encoded conversions

When the party ordered collection object can meet the following two conditions, the object uses Ziplist encoding:

1 The number of elements in an ordered set that remains at the same time is less than 128

2 The length of all element members saved by an ordered collection is less than 64 bytes

Implementation of ordered SET command

Redis Bottom Quest (v): Redis Object