In-depth understanding: REDIS hash Structure memory model anatomy

This article focuses on the most frequently used data types in Redis: hashing (or hashing), and how it is stored inside redis.

The brain map of this article is as follows:

Hash type internal encoding details

For the 5 commonly used data types of Redis (String, Hash, List, set, sorted set), each data type provides a minimum of two internal encoding formats, and the selection of internal encodings for each data type is completely transparent to the user. Redis Adaptively chooses a more optimized internal encoding format based on the amount of data.

If you want to see the internal encoding format for a key, you can use the OBJECT ENCODING keyname directive, such as:

There are two possible internal encodings for the most frequently used hash types:

• Obj_encoding_ziplist (compression list)
• OBJ_ENCODING_HT (hash table)

Redis chooses the best of both encodings based on the amount of data available, all of which are completely transparent to the user.

Redis is stored in a compressed list (obj_encoding_ziplist) encoding when there are fewer data entries and the data values are small. Here the members are "less" and the criteria for member values "smaller" can be configured with the following configuration:

hash-max-ziplist-entries 512hash-max-ziplist-value 64

Redis defaults to default values, although users can configure their own according to the actual situation.

Redis uses obj_encoding_ when the number of fields for the hash type key is < hash-max-ziplist-entries and the length of each field name and field value < Hash-max-ziplist-value Ziplist to store the key, and vice versa, it is converted to OBJ_ENCODING_HT encoding.

Accreditations, we might as well start with an experiment and feel it:

Obviously, this experiment verifies that when the field value is longer than 64 o'clock, the encoding format is switched from Ziplist mode to hashtable mode.

Source code before, there is no secret, we look at the Redis on this part of the switching source code implementation, then understand more clearly:

The following is a detailed description of the internal storage models of the two encoding formats, obj_encoding_ziplist and OBJ_ENCODING_HT, knowing their respective features and advantages and disadvantages, and naturally understanding the intent of using them within Redis.

Obj_encoding_ziplist encoding

Ziplist compression list is a compact encoding format, the overall idea is time-to-space, that is, at the cost of partial read and write performance, in exchange for a very high memory space utilization, so only for a small number of fields, and the field value is smaller scene.

The reason for the high memory utilization of the compressed list is inseparable from the characteristics of its contiguous memory, and its typical memory structure can be displayed visually:

So if you use Ziplist to store a redis hash type, the arrangement of the elements becomes the image shown: The key and value are logical contiguous memory:

OBJ_ENCODING_HT encoding

Obj_encoding_ht This encoding is the real hash table structure, or the dictionary structure, which can achieve O (1) complexity of the read and write operations, so the efficiency is very high.

Within Redis, the real hash data structure from the OBJ_ENCODING_HT type to the bottom layer is nested in layers, with the following relationships:

This relationship can be seen from the source code of the Redis hash table Definition section:

Here's a look at each section:

• About hash nodes (Dictentry)

• About hash tables (DICTHT) and dictionaries (dict)

• About Dicttype

• How Redis calculates hash values

The source code for Redis compute hash is as follows:

This is a C language macro definition, in fact, the real responsibility behind the hash value calculation is described above in the DICTTYPE structure of the function pointer hashfunction.

The Hashfunction function pointer, when initialized, corresponds to the actual function that is assigned a real hash value, as follows:

• How Redis calculates the index value for access indexes

The calculation of the index value depends on the hash value calculated above, the code is as follows:

Here, there is also a very interesting detail: The dictionary dict is always saved with two hash table structure ht[2], and its highly relevant rehash operations, which is explained in the next article.

Original link: https://my.oschina.net/hansonwang99/blog/1934354

Because of limited capacity, if there are errors or inappropriate, but also please criticize correct, learn to communicate together!

By the way, we recommend a Java Architecture Communication Learning Group:698581634, which will share some of the video materials recorded by senior architects : Spring , MyBatis , Netty source analysis, high concurrency, performance, distributed, micro-service architecture principles, JVM performance Optimization These become the necessary knowledge systems for architects, mainly for Java developers to improve themselves, break through the bottleneck, I believe you come to learn, there will be ascension and harvest.

In-depth understanding: REDIS hash Structure memory model anatomy