Compression list in Redis

compression list in Redis

The compression list (ziplist) is one of the underlying implementations of the list key and hash keys. When a list key contains only a few list items, and each list item is either a small integer value or a shorter string, Redis uses a compression list to do the underlying implementation of the list key.

For example, execute the following command to create a list key that compresses the list implementation:

127.0.0.1:6379> Rpush 1st 1 3 5 10086 "Hello" "World"
(integer) 6
127.0.0.1:6379> OBJECT ENCODING 1st
"Z IPList "

The list keys contain small integer values such as 1, 3, 5, 10086, and short strings such as Hello and world.

In addition, when a hash key contains only a small number of keys and values, and the keys and values of each key pair are either small integer values or short-length strings, Redis uses a compressed list as the underlying implementation of the hash keys.

For example, execute the following command to create a hash key that is implemented by a compression list:

127.0.0.1:6379> Hmset Profile Name Jack Age Job Programmer
OK
127.0.0.1:6379> OBJECT ENCODING profile
   "Ziplist"

All keys and values contained in the hash key are small integer values or short strings. composition of the compression list

The compression list, developed by Redis to conserve memory, is a sequential (sequential) data structure consisting of a series of specially coded contiguous blocks of memory. A compressed list can contain any number of nodes (entry), each of which can hold a byte array or an integer value.

The following diagram shows the various components of the compression list:

The following table records the types, lengths, and uses of each component:

Observe the example of a compressed column shown below:

The value of the list Zlbytes property is 0x50 (decimal 80), which indicates that the total length of the compressed list is 80 bytes. The value of the list Zltail property is 0x3c (decimal 60), which means that if we have a pointer p to the starting address of the compressed list, you can calculate the address of the footer node Entry3 by using the pointer p plus the offset 60. The value of the list Zllen property is 0x3 (decimal 3), which indicates that the compression list contains three nodes.

Another example of a compressed column is observed:

The value of the list Zlbytes property is 0xd2 (decimal 210), which indicates that the total length of the compressed list is 210 bytes. The value of the list Zltail property is 0xb3 (decimal 179), which means that if we have a pointer p to the starting address of the compressed list, you can calculate the address of the footer node Entry5 by using the pointer p plus the offset 179. The value of the list Zllen property is 0x5 (decimal 5), which indicates that the compression list contains five nodes. the composition of a compressed list node

Each compressed list node can hold a byte array or an integer value, where the byte array can be one of the following lengths: A byte array of length less than or equal to 63 (2^6-1) bytes, and a byte array with a length of less than or equal to 16383 (2^14-1) bytes; byte array with a length of less than or equal to 4294967295 (2^32-1) bytes;

The integer value can be one of the following six lengths: 4-bit long, unsigned integer between 0 and 12, 1-byte long signed, 3-byte long signed integer, int16_t type Integer, int32_t type Integer, int64_t type Integer.

Each compressed list node consists of previous_entry_length, encoding, and content three parts:

Previous_entry_length

The Previous_entry_length property of the node, in bytes, records the length of the previous node in the compressed list. The length of the Previous_entry_length property can be 1 bytes or 5 bytes: If the length of the previous node is less than 254 bytes, the length of the Previous_entry_length property is 1 bytes: The length of the previous node is stored in this byte. If the length of the previous node is greater than or equal to 254 bytes, the length of the Previous_entry_length property is 5 bytes: where the first byte of the property is set to 0xFE (decimal 254), and the subsequent four bytes are used to hold the length of the previous node.

The following illustration shows a compressed list node with a one-byte long previous_entry_length attribute with a property value of 0x05 that indicates that the previous node is 5 bytes in length.

The following figure shows a compression node that contains a five-byte long Previous_entry_length property with a value of 0xfe00002766, where the highest byte 0xFE of the value indicates that this is a five-byte long Previous_entry_length property. The subsequent four-byte 0x00002766 (decimal 10086) is the actual length of the previous byte.

Because the Previous_entry_length property of the node records the length of the previous node, the program can calculate the starting address of the previous node based on the starting address of the current node, using the pointer operation.

For example, if we have a pointer to the starting address of the current node C, then we simply subtract the value of the current node Previous_entry_length property with the pointer C, we can draw a pointer to the start address of the previous node p, as shown in the following figure:

The table-footer traversal of a compressed list is done using this principle, as long as we have a pointer to the starting address of a node, then through this pointer and the node's Previous_entry_length property, the program can go forward a node backwards, The table header node that eventually reaches the compression list. encoding

The encoding property of a node records the type and length of data held by the content property of a node: one byte, two bytes, or five bytes long, the highest bit of value is 00, 01, or 10 is a byte array encoding: This encoding means that the content property of a node holds a byte array. The length of the array is removed by encoding the other bit records after the maximum of two bits; one byte long, the highest bit of the value begins with an integer encoding of 11: This encoding indicates that the content property of the node holds the integer value, the type and length of the integer value are removed from the other bits recorded by the code up to two bits;

The following table records all available byte array encodings, the underscore in the table is empty, and the B, x variables represent the actual binary data, and for readability, multiple bytes are separated by spaces:

The following table records all the available integer encodings:

content

The content property of a node is responsible for saving the node's value, which can be a byte array or an integer, and the type and length of the value is determined by the node's Encoding property.

The following illustration shows an example of a node that holds a byte array:

The maximum encoding of two bits 00 means that the node holds a byte array, the last six bits of the encoded 001011 record the length of the byte array, and the content property holds the value of the node: Hello World.

The following illustration shows an example of a node that holds an integer value:

Encoding 11000000 means that the node holds an integer value of type int16_t, and the content property holds the value of the node 10086.