The object of the Redis implementation (i)

Object

In the previous chapter, we introduced the main data structures of Redis, such as simple dynamic string SDS, double-ended linked list, dictionaries, compression lists, integer collections, etc. Redis does not directly use these data structures to implement key-value pairs of databases, but instead creates an object system based on these data structures, which contains five types of objects, such as String objects, list objects, hash objects, collection objects, and ordered collection objects. Each object uses at least one of the data structures we introduced earlier

With these five different types of objects, Redis can determine whether an object can execute a given command, depending on the type of object, before executing the command. Another benefit of using objects is that we can set different data structure implementations for objects based on different usage scenarios, thus optimizing the efficiency of objects for use in different scenarios

In addition, the Redis object system implements a reference counting-based memory recovery mechanism, when the program no longer uses an object, the memory occupied by the object is automatically freed, and Redis also implements the object sharing mechanism by reference counting, which can be under the appropriate conditions, Save memory by having multiple database keys share an object

Finally, the Redis object has access time logging information that can be used to calculate the idling time of the database keys, and those keys with a large idling time may be removed by the server if the server has the MaxMemory function enabled.

Type encoding of the object

Redis uses objects to represent keys and values in a database, and each time we create a new key-value pair in a Redis database, we create at least two objects, one object as the key for the key-value pair (the Key object), and the other object as the value of the key-value pair (value object)

For a chestnut, the following set command creates a new key-value pair in the database, where the key of a key-value pair is an object that contains the string "MSG", and the value of the key-value pair is an object that contains the string "Hello World"

127.0.0.1:6379> SET msg "Hello World" OK

　　

Each object in Redis is represented by a redisobject structure that holds three properties related to the data: type, encoding, PTR

Redis.h

typedef struct REDISOBJECT {//type    unsigned type:4;    unsigned notused:2;  Coding    unsigned encoding:4;    unsigned lru:22;  Reference count    int refcount;//Pointer to the underlying implementation data structure,    void *ptr;} robj;

　　

Type

The Type property of the object records the types of the object, and the value of this property can be one of the constants listed in table 1-1.

Table 1-1 Types of objects

Type constants	The name of the object
Redis_string	String Object
Redis_list	List objects
Redis_hash	Hash object
Redis_set	Collection Object
Redis_zset	Ordered collection objects

For a key-value pair saved by a Redis database, the key is always a string object, and the value can be either a string object, a list object, a hash object, a collection object, or one of the ordered collection objects, so:

• When we call a database key "string key", we mean "This database key corresponds to a value of string object"
• When we call a database key "list key", we mean "This database key corresponds to the value of the list object"

The type command is implemented similarly, and when we execute the type command on a database key, the command returns the value object type corresponding to the database key, not the key object type:

# key is a string object, the value is a string object 127.0.0.1:6379> SET msg "Hello World" ok127.0.0.1:6379> TYPE msgstring# key is a string object, the value is a list object 127.0.0.1 :6379> Rpush numbers 1 3 5 (integer) 3127.0.0.1:6379> TYPE numberslist# key is a string object, value is hash object 127.0.0.1:6379> Hmset ProFi Le name Tome age career programmerok127.0.0.1:6379> TYPE profilehash# key to String object, value for collection object 127.0.0.1:6379> sadd fruits A pple Banana Cherry (integer) 3127.0.0.1:6379> TYPE fruitsset# key is a string object and the value is an ordered collection object 127.0.0.1:6379> zadd price 8.5 Apple 5 .0 Banana 6.0 Cherry (integer) 3127.0.0.1:6379> TYPE pricezset

　　

Table 1-2 lists the output that the type command produces when it faces different types of value objects

table 1-2 type command output for different types of value objects

Object	The value of the object Type property	Output of the type command
String Object	Redis_string	String
List objects	Redis_list	List
Hash object	Redis_hash	Hash
Collection Object	Redis_set	Set
Ordered collection objects	Redis_zset	Zset

Coding and the underlying implementation

The PTR pointer to the object points to the underlying implementation data structure of the object, which is determined by the object's Encoding property. The Encoding property records the encoding used by the object, that is, what data structures the object uses as the underlying implementation of the object, and the value of this property can be one of the constants listed in table 1-3.

table 1-3 encoding of Objects

Encoding constants	The underlying data structure for the encoding
Redis_encoding_int	An integer of type long
Redis_encoding_embstr	EMBSTR encoded Simple Dynamic string
Redis_encoding_raw	Simple dynamic string
Redis_encoding_ht	Dictionary
Redis_encoding_linkedlist	Double-ended linked list
Redis_encoding_ziplist	Compression list
Redis_encoding_intset	Integer collection
Redis_encoding_skiplist	Jumping Tables and dictionaries

Each type of object uses at least two different encodings, and table 1-4 lists the encodings that can be used for each type of object

table 1-4 objects of different types and encodings

Type	Coding	Object
Redis_string	Redis_encoding_int	String objects that are implemented using integer values
Redis_string	Redis_encoding_embstr	A string object implemented using EMBSTR encoded simple dynamic string
Redis_string	Redis_encoding_raw	String objects implemented using a simple dynamic string
Redis_list	Redis_encoding_ziplist	List objects implemented using a compressed list
Redis_list	Redis_encoding_linkedlist	List objects implemented using a double-ended chain table
Redis_hash	Redis_encoding_ziplist	Hash object implemented using a compressed list
Redis_hash	Redis_encoding_ht	Hash object implemented using a dictionary
Redis_set	Redis_encoding_intset	Collection objects implemented using an integer collection
Redis_set	Redis_encoding_ht	Collection objects implemented using a dictionary
Redis_zset	Redis_encoding_ziplist	An ordered collection object implemented using a compressed list
Redis_zset	Redis_encoding_skiplist	An ordered collection object implemented using a skip table and a dictionary

Use the object Encoding command to view the encoding of a database Key's value object:

127.0.0.1:6379> SET msg "Hello wrold" ok127.0.0.1:6379> OBJECT ENCODING msg "EMBSTR" 127.0.0.1:6379> sadd numbers 1 3 5 (integer) 3127.0.0.1:6379> OBJECT ENCODING Numbers "Intset" 127.0.0.1:6379> sadd Numbers "seven" (integer) 1127.0.0.1:6379> OBJECT ENCODING Numbers "Hashtable"

　　

Table 1-5 lists the object encoding command output for the different encoded objects:

table 1-5 OBJECT encoding output for different encodings

The underlying data structure used by the object	Encoding constants	OBJECT encoding Command output
Integer	Redis_encoding_int	Int
Embstr coded simple dynamic string (SDS)	Redis_encoding_embstr	Embstr
Simple dynamic string	Redis_encoding_raw	Raw
Dictionary	Redis_encoding_ht	Hashtable
Double-ended linked list	Redis_encoding_linkedlist	LinkedList
Double-ended linked list	Redis_encoding_ziplist	Ziplist
Integer collection	Redis_encoding_intset	Intset
Jumping Tables and dictionaries	Redis_encoding_skiplist	Skiplist

Using the Encoding property to set the encoding used by an object rather than associating a fixed encoding for a particular type of object greatly improves the flexibility and efficiency of Redis because Redis can set different encodings for an object based on different usage scenarios. This optimizes the efficiency of the object in a given scenario. Give me a chestnut. When the list object contains fewer elements, Redis uses a compressed list as the underlying implementation of the List object:

• Because the compression list is more memory-efficient than the double-ended lists, and when the elements are relatively young, the compressed list saved in memory in contiguous chunks can be loaded into the cache more quickly than the double-ended chain table.
• As list objects contain more and more elements, and the advantages of using a compressed list to preserve elements fade away, the object will convert the underlying implementation from the compression list to a more successful, more suitable, double-ended chain of elements

Other types of objects will also perform similar optimizations by using a number of different encodings, and in the next section we will describe five different types of objects in Redis, explaining the encoding used at the bottom of these objects, and listing the conditions required for the object to be converted from one encoding to another. and how the same command is implemented on many different encodings

String Object

The encoding of a string object can be int, raw, or embstr. If a string object holds an integer value, and the integer value can be represented by a long type, the string object saves the integer value in the PTR attribute of the string structure (converts void * to long) and sets the encoding of the string object to int

For a chestnut, if we execute the following set command, then the server will create a 1-1 int encoded string object as the value of the number key:

127.0.0.1:6379> SET number 10086ok127.0.0.1:6379> OBJECT ENCODING number "int"

　　

Figure 1-1 INT-encoded string object

If the string object holds a string value, and the string value is longer than 32 bytes, the string object will use a simple dynamic string (SDS) to hold the string value and set the object's encoding to Raw. Give me a chestnut, if we execute the following command, then the server will create a 1-2 raw encoded string as the value of the store key

127.0.0.1:6379> SET Story "Long, long, long, long, long ago there lived a king ..." ok127.0.0.1:6379> STRLEN stories (in Teger) 55127.0.0.1:6379> OBJECT ENCODING Story "Raw"

　　

Figure 1-2 Raw encoded string Object

If the string object holds a string, and the string is less than or equal to 44 bytes, the string is encoded using EMBSTR, as shown in the following example

127.0.0.1:6379> SET Story "Long, long, long, long, long ago there lived" Ok127.0.0.1:6379> STRLEN Story (integer) 4412 7.0.0.1:6379> OBJECT ENCODING Story "Embstr" 127.0.0.1:6379> SET Story "Long, long, long, a long ago there lived" ok127.0.0.1:6379> STRLEN Story (integer) 45127.0.0.1:6379> OBJECT ENCODING Story "Raw"

　　

EMBSTR encoding is an optimized encoding for storing short strings, which, like raw encoding, uses the REDISOBJECT structure and SDSHDR structure to represent string objects. However, the raw encoding calls two memory allocation functions to create the redisobject structure and the SDSHDR structure, and EMBSTR encoding allocates a contiguous space by invoking a memory allocation function, which in turn contains redisobject and SDSHDR structures. 1-3 is shown

Figure 1-3 memory block structure created by EMBSTR encoding

EMBSTR encoded string objects produce the same effect as raw-encoded string objects when executing commands, but using EMBSTR encoded strings to hold short string values has the following benefits:

• EMBSTR encoding reduces the number of memory allocations required to create a string object from raw encoded two times to one
• Releasing a EMBSTR encoded string object requires only one call to the memory-free function, and the release of the raw-encoded string object calls two memory-free functions
• Because all the data for a EMBSTR encoded string object is kept in a contiguous amount of memory, this encoded string object makes better use of the benefits of caching than raw-encoded string objects

As an example, the following command creates a EMBSTR encoded string object as the value of the MSG key, and the value object looks like 1-4

127.0.0.1:6379> SET msg "Hello World" ok127.0.0.1:6379> OBJECT ENCODING msg "EMBSTR"

　

　

Figure 1-4 EMBSTR encoded string Object

Finally, floating-point numbers that can be represented by a long double type are also stored as string values in Redis. If we want to save a floating-point number into a string object, the program first converts the float to a string value and then saves it. Give me a chestnut. Execute the following code to create a string object that contains 3.14

127.0.0.1:6379> SET pi 3.14ok127.0.0.1:6379> OBJECT ENCODING pi "Embstr"

　　

When needed, the program converts the string value saved in a String object back to a floating point value, performs some action, and then converts the floating-point number that was executed to the string value back to the character, and continues to be stored in the string object. To give a chestnut, we execute the following code:

127.0.0.1:6379> SET pi 3.14ok127.0.0.1:6379> incrbyfloat pi 3.0 "6.14" 127.0.0.1:6379> OBJECT ENCODING PI "embstr

　　

The program first removes the string value "3.14" from the string object, converts it back to the floating-point value of 3.14, and then adds 3.14 and 2.0 to get 5.14 after converting back to the string and saving the string "5.14" to the string object. Table 1-6 summarizes and lists the encoding used by string objects to hold different types of values

Table 1-6 String objects save encoding for each type of value

Value	Coding
Integers that can be saved with a long type	Int
Floating-point numbers that can be saved with a long double type	Embstr or Raw
A string value, or an integer that is too large to be represented by a long type, or a floating-point number that is too large to be represented by a long double type	Embstr or Raw

Encoded conversions

The INT-encoded string object and the EMBSTR encoded string object are converted to raw-encoded string objects in the case that the condition is met. For an INT-encoded string object, if we have executed some commands on the object so that the object is not saved as an integer value, but rather as a string value, the string object will change from int to Raw

In the following example, we append a string value to a string that holds an integer value through the append command, because the append operation can only be performed on a string value, so the program converts the previously saved integer value to a string and then performs the append operation, and the result of the operation is a raw encoded, A string object that holds a string value

127.0.0.1:6379> SET number 10086ok127.0.0.1:6379> OBJECT ENCODING number "int" 127.0.0.1:6379> APPEND number "is A good number! " (integer) 23127.0.0.1:6379> GET number "10086 is a good number!" 127.0.0.1:6379> OBJECT ENCODING Number "raw"

　　

In addition, because Redis does not write any corresponding modifiers for EMBSTR encoded string objects (only int encoded string objects and raw encoded string objects have these programs), the EMBSTR encoded string object is actually read-only. When we execute any modification command on a EMBSTR encoded string object, the program first converts the object's encoding from EMBSTR to raw before executing the Modify command. For this reason, the EMBSTR encoded string object will always become a raw encoded string object after executing the Modify command.

The following code shows an example of an EMBSTR encoded string object that, after executing the append command, changed the encoding of the object from Embstr to Raw:

127.0.0.1:6379> SET msg "Hello World" ok127.0.0.1:6379> OBJECT ENCODING msg "EMBSTR" 127.0.0.1:6379> APPEND msg "a gain! " (integer) 18127.0.0.1:6379> OBJECT ENCODING msg "Raw"

　　

Implementation of STRING commands

Because the value of the string key is a string object, all commands used for the string key are constructed for the string object, table 1-7 gives some of the string commands, and the implementation of these commands under different encoded string objects

table 1-7 Implementation of STRING commands

Command	An implementation method of int encoding	Implementation method of EMBSTR coding	How to implement RAW code
SET	saving values with INT encoding	Saving values using EMBSTR encoding	Saving values with raw encoding
GET	Copies the integer value saved by the object, converts the copy to a string value, and returns the string value to the client	Returns a string value directly to the client	Returns a string value directly to the client
APPEND	Convert the object to raw encoding, and then do this as raw encoding	Convert the object to raw encoding, and then do this as raw encoding	Call the Sdscatlen function to append the given string to the end of an existing string
Incrbyfloat	Takes the integer value and converts it to a long double type of floating-point number, adds the float, and then saves the resulting floating-point numbers.	Remove the string value and attempt to convert it to a long double type of floating-point number, add the float, and then save the resulting floating-point numbers. If the string value cannot be converted to a floating-point number, an error is returned to the client	Remove the string value and attempt to convert it to a long double type of floating-point number, add the float, and then save the resulting floating-point numbers. If the string value cannot be converted to a floating-point number, an error is returned to the client
Incrby	The calculation results are saved as integers for the addition calculation of integer values.	EMBSTR encoding cannot execute this command, returning an error to the client	Raw encoding cannot execute this command, returning an error to the client
Decrby	By subtracting the integer value, the resulting calculation is saved as an integer.	EMBSTR encoding cannot execute this command, returning an error to the client	Raw encoding cannot execute this command, returning an error to the client
STRLEN	Copies the integer value saved by the object, converts the copy to a string value, calculates and returns the length of the string value	Call the Sdslen function to return the length of the string	Call the Sdslen function to return the length of the string
SETRANGE	Convert the object to raw encoding, and then execute the command as raw code	Convert the object to raw encoding, and then execute the command as raw code	Sets the value on a string-specific index to the given character
GETRANGE	Copies the integer value saved by the object, converts the copy to a string value, and then takes out and returns the character at the specified index of the string	Directly takes out and returns the character of the string at the specified index	Directly takes out and returns the character of the string at the specified index

The object of the Redis implementation (i)