Redis learning Manual (memory optimization)

I. Special encoding:

Since redis 2.2, many data types can be optimized by special encoding. Here, the storage structure can be optimized by hash, list, and sets composed of Integers to consume less space. In some cases, you can save 9/10 of the space.
These special codes are completely transparent for redis usage. In fact, they are only a transaction between CPU and memory. If the memory usage is higher, the CPU consumed during data operations will naturally increase, and vice versa. Redis provides a set of configuration parameters for setting various thresholds related to special encoding, such:
# If the number of fields in the hash is smaller than the parameter value, redis uses special encoding for the hash value of the key.
Hash-max-zipmap-entries 64
# If the maximum length of each field in hash is no more than 512 bytes, redis uses a special encoding method for the hash value of the key.
Hash-max-zipmap-value 512
# The meanings of the following two parameters are basically the same as those of the above two hash-related parameters, but the object type is list.
List-max-ziplist-entries 512
List-max-ziplist-value 64
# If the number of integer elements in the set cannot exceed 512, redis uses this special encoding.
Set-max-intset-entries 512
If a value that has been encoded exceeds the maximum limit in the configuration information after modification, redis automatically converts it to a normal encoding format, which is very fast, however, if you perform this operation in turn, convert a normal encoding of the Greater half to a special encoding. redis recommends that you test the conversion efficiency before the formal implementation, this conversion is often very inefficient.

2. Bit and byte operations:

Starting from redis 2.2, redis provides the getrange/setrange/getbit/setbit command for the string type key/value. With these commands, we can access the value data of the string type just like an array operation. For example, the ID that uniquely identifies a user may be a substring of a string value. In this way, you can use the getrange/setrange command to conveniently extract data. In addition, bitmap can be used to indicate the user's gender information, for example, 1 indicates male and 0 indicates female. In this way, when the Gender Information of 100,000,000 users is expressed, it only occupies 12 Mb of storage space. At the same time, it is very efficient to traverse data through the setbit/getbit command.

3. Use hash as much as possible:

because small hash data occupies a relatively small amount of space, we should consider using hash data, such as user registration information, as much as possible in actual applications, this includes the name, gender, email, age, and password fields. Of course, we can store the information in the form of a key, and the information entered by the user is stored in the form of a string value. However, redis is more recommended to store data in the form of hash, and the above information is expressed in the form of field/value.
now we will further prove this statement by studying the redis storage mechanism. At the beginning of this blog, we have mentioned the special encoding mechanism. There are two configuration parameters related to the hash type: hash-max-zipmap-entries and hash-max-zipmap-value. As for the scope of their functions, we will not repeat them too much here. Now let's assume that the number of fields stored in hash value is smaller than hash-max-zipmap-entries, and the length of each element is smaller than hash-max-zipmap-value. In this way, whenever a new hash key/value storage is available, redis creates a fixed-length space for the hash value. The maximum number of pre-allocated bytes is:
total_bytes = hash-max-zipmap-entries * hash-max-zipmap-value
As a result, all fields in the hash are reserved, in addition, the field/value can be randomly accessed as the access array, and the step interval between them is hash-max-zipmap-value. Only when the number of fields in the hash value or the length of a new element exceeds the preceding two parameter values, redis will consider re-storing them as hash tables, otherwise, this efficient storage and access method will always be maintained. Besides, each key stores related system information, such as the expiration time and LRU. Therefore, compared with the string-type key/value, the hash type greatly reduces the number of keys (most keys are expressed and stored in the form of hash fields), further optimizing the efficiency of storage space usage.