Redisbook notes--redis memory-mapped data structures

Although the internal data structures are very powerful, creating a complete set of data structures itself is a very memory-intensive task, and using expensive internal data structures is not the best approach when an object contains a small number of elements, or the elements themselves are not large in size.

To solve this problem, Redis uses memory-mapped data structures instead of internal data structures, where conditions permit.

Memory-mapped data structures can save a lot of memory for users. However, because memory-mapped data structures are much more complex to encode and manipulate than internal data structures, memory-mapped data structures consume more CPU time than similar internal data structures.

This section describes the two types of memory-mapped data structures that Redis is currently using.

Integer collection

An integer set (Intset) is used to hold multiple integer values in an orderly, non-repeatable way, and it automatically chooses what length of integer type to hold the element, based on the value of the element.

Intset is one of the underlying implementations of the collection key, if a collection:

1. Only the integer element is preserved;

2. The number of elements is not many;

Redis then uses Intset to save the collection elements.

Depending on your needs, Intset can be automatically upgraded from int16_t to int32_t or int64_t, or from int32_t to int64_t.

typedefstructIntset {

// Save the length of the type used by the element

uint32_tencoding;

// number of elements

uint32_tlength;

// save an array of elements

int8_tcontents[];

} Intset;

Instance

Now, to add a value of int32_t of length 65535 to the collection, Intset needs to perform the following steps:

1. Set the Encoding property to Intset_enc_int32.

2. The contents array is memory redistributed based on the value of the Encoding property.

3. Since the original 3 int16_t values are also "squeezed" in the 48 bits in front of contents, the program needs to move and type them to adapt them to the new encoding of the collection.

4. Finally, add the new value 65535 to the array:

Intset is used to store multiple integer values in an orderly, non-repeatable way, and it automatically chooses what length of integer type to hold the element, based on the value of the element.

• When a longer integer value is added to Intset, the Intset needs to be upgraded, and the bit length of each element in the new intset is equal to the bit length of the newly added value, but the value of the original element does not change.

• The upgrade causes the entire intset to reallocate memory and move all the elements in the collection, which is an O (N) complex operation.

Intset only supports upgrades and does not support demotion.

The Intset is ordered, and the program uses a binary lookup algorithm to implement the find operation with an O (LgN) complexity.

Compression list

Ziplist is a list of specially coded blocks of memory, a ziplist can contain multiple nodes (entry), and each node can hold a limited-length character array (a char array that does not end with a) or an integer.

• Character array

– The length is less than or equal to 1) byte array of characters

– length is less than or equal to 16383 (214 ??? 1) byte array of characters

– length is less than or equal to 4294967295 (232 ??? 1) byte array of characters

• Integer

–4 bits long, unsigned integer between 0 and 12

–1-byte long, signed integer

–3-byte long, signed integer

–int16_t type integer

–int32_t type integer

–int64_t type integer

Ziplist is a list of specially coded blocks of memory that can hold character arrays or integer values, and is one of the underlying implementations of hash keys, list keys, and an ordered set key.

Redisbook notes--redis memory-mapped data structures