Memory Usage Analysis of sorted sets and lists in Redis

Memory Usage Analysis of sorted sets and lists in Redis

Before getting started, you need to understand several definitions: dictionary, compressed list, and skip table.

Dictionary: a common data structure, the key-value structure.

Common implementations include the red/black tree (map in stl) and the hash table (unordered_map in stl ). The search operations on the red/black tree have O (logN) time complexity. The time complexity of the hash table search operation is O (1. The dictionary in redis uses a hash table as the underlying implementation.

Compressed list: a sequential data structure consisting of some columns of specially encoded continuous memory blocks.

The compressed list can contain multiple nodes (only one of which is called an array ). The advantage of compressing the list is to save memory. All the downsides of the sequential structure are compressed.

Skip table: an ordered data structure. It maintains multiple pointers to other nodes in each node to quickly access the node.

The Skip table supports searching nodes with average O (logN) and Worst O (N) time complexity.

The jump table in redis consists of two structures: zskiplist and zskiplistNode. zskiplist is used to save the information of the jump table (such as the header node, table tail node, and length), and zskiplistNode is used to represent the Skip table node. Nodes in the skip table are sorted by branch size. When the scores are the same, nodes are sorted by the size of member objects. In the same hop table, multiple nodes can contain the same score, but the Member objects of each node must be unique.

Why is the memory occupied by the sorted set in redis larger than that in the list?

Let's talk about the implementation of the list in redis. The list in redis is implemented by compressing the list or linked list at the underlying layer. The redis list has two different encodings (implementation methods): ziplist and pull list. The encoding formats can be converted to each other under specific conditions. When the length of all string elements stored in the list object is less than 64 bytes and the number of elements in the list object is less than 512, the list object uses ziplist. Otherwise, use the upload list encoding.

The implementation of Ordered Sets is more complex in redis, including ziplist and skiplist.

Ziplist-encoded ordered collection objects use the compressed list as the underlying implementation. Each set element is saved by two compressed list nodes that are next to each other. The first node stores the member (member) of the element, and the second element stores the score of the element ).

An ordered set object encoded by skiplist uses the zset result as the underlying implementation. A zset structure contains both a dictionary and a skip table. The Skip table in the zset structure stores all the set elements according to the scores from small to large. Through this skip table, range operations such as zrank and zrange can be performed in an orderly manner. The dictionary in the zset structure stores the ing between members and values in the ordered set. With this dictionary, you can use the time complexity of O (1) to find the values of members. Although zset uses two data structures to store data, the two data structures use pointers to share members and scores of the same element, so no duplicate members or scores are generated.

Ziplist encoding is used when the number of elements saved in an ordered set is less than 128 and the length of all element members is less than 64 bytes. Instead, use skiplist encoding.

Why should we use skip tables and dictionaries for ordered sets?

When using dictionaries separately, searching is fast and requires only the time complexity of O (1). However, the range operation requires sorting of dictionary elements. To do this sort, at least O (NlogN) is required) time Complexity and extra O (N) memory space.

When a skip table is used separately, the range operation of the Skip table will be retained, but the query efficiency will decrease, and the time complexity of the query will change from O (1) to O (logN ).

Through the above analysis, we can see that the implementation of List objects is much simpler than that of Ordered Set objects, and there are not so many messy things. Therefore, an ordered set occupies more memory than a list.

Install and test Redis in Ubuntu 14.04

Redis cluster details

Install Redis in Ubuntu 12.10 (graphic explanation) + Jedis to connect to Redis

Redis series-installation, deployment, and maintenance

Install Redis in CentOS 6.3

Learning notes on Redis installation and deployment

Redis. conf

Redis details: click here
Redis: click here

This article permanently updates the link address: