Redis Bottom Principle

Introduction to Redis Object types

Redis is a key/value database in which each key and value are represented using objects. For example, we execute the following code:

[plain] view plain copy print? Redis>set message "Hello Redis"
The key is the message, which is an object that contains the string "message". And value is an object that contains the "Hello Redis".

Redis A total of five types of objects, respectively:

Type Constants	the name of the object
Redis_string	String Object
Redis_list	List objects
Redis_hash	Hash object
Redis_set	Collection objects
Redis_zset	Ordered collection objects

The structure body of an object in Redis is represented as follows:

[CPP] View Plain copy print? /*   * Redis  objects    */   typedef struct redisobject {           //  type        unsigned type:4;                   //  not using ( Align bit)        unsigned notused:2;           //  Coding mode        unsigned encoding:4;           // LRU  time (relative to  server.lruclock)        unsigned  lru:22;          //  reference count         int refcount;          //  value to object         void *ptr;     } robj;  

Type represents the object type of the object, one of the top five. However, in order to improve storage efficiency and program execution efficiency, the underlying data structure of each object can be implemented in more than one way. Encoding represents the encoding used at the bottom of the object. The following is an introduction to each of the underlying data structure implementations, and then to the underlying structure of each object type and analysis of their relationships. Redis Object underlying data structure

There are eight types of underlying data structures, as shown in the following table:

Encoding Constants	the underlying data structure corresponding to the encoding
Redis_encoding_int	Integer of type Long
Redis_encoding_embstr	EMBSTR encoded simple dynamic strings
Redis_encoding_raw	Simple dynamic string
Redis_encoding_ht	Dictionary
Redis_encoding_linkedlist	Double-end Linked list
Redis_encoding_ziplist	Compress list
Redis_encoding_intset	Integer collection
Redis_encoding_skiplist	Jumping Tables and dictionaries

String Object

The encoding of a string object can be int, raw, or embstr.

If the contents of a string can be converted to long, the string is converted to a long type, and the object's PTR points to that long and the object type is represented by an int type.

There are two types of ordinary strings, embstr and Raw. EMBSTR should be the new data structure for Redis 3.0, which is not in 2.8. If the length of the string object is less than 39 bytes, the Embstr object is used. Otherwise, use a traditional raw object. You can see from the following code: [CPP] view plain copy print? #define REDIS_ENCODING_EMBSTR_SIZE_LIMIT robj *createstringobject (char *ptr, size_t len) {if (Len <= redis_       Encoding_embstr_size_limit) return Createembeddedstringobject (Ptr,len);   else return Createrawstringobject (Ptr,len); The benefits of EMBSTR are as follows: Embstr is created by allocating only one memory, while Raw is two times (one for the SDS, another for objet, Embstr for the first time). In contrast, the amount of memory freed is changed from two to one time. Embstr's objet and SDS are put together to better exploit the benefits of caching.

It should be noted that Redis does not provide any way to modify the Embstr, that is, EMBSTR is a read-only form. Changes to EMBSTR are actually converted to raw before modification.

The difference between raw and EMBSTR can be shown in the following two images:

List Objects

The encoding of a list object can be either Ziplist or LinkedList.

Ziplist is a compact list, and its advantage is that it saves more memory space because it stores content in contiguous memory areas. When the list object element is small and each element is small, the Ziplist store is used. But when the amount of data is too large, ziplist is not so useful. Because in order to ensure the continuity of his storage content in memory, the insertion complexity is O (N), that is, each insertion will be realloc again. As shown in the following illustration, PTR in the object structure is pointing to a ziplist. The entire ziplist only needs to be malloc once, and they are a contiguous area in memory.

LinkedList is a two-way linked list. Its structure is relatively simple, the node holds the pre and next two pointers, as well as node-related information. When each node is added, it is necessary to malloc a piece of memory again.

Hash Object

The underlying implementation of a hash object can be either Ziplist or Hashtable.

Hash objects in the ziplist are stored in the order in which they are key1,value1,key2,value2. This approach is highly efficient when the number of objects is small and the content is not large.

Hashtable is implemented by the DICT structure.[CPP]View Plain copy print?       typedef struct DICT {Dicttype *type;       void *privdata;       Dictht ht[2]; Long Rehashidx; /* rehashing not in progress if rehashidx = = 1 */int iterators;   /* Number of iterators currently running * * DICT; Dict is a dictionary in which the pointer Dicht Ht[2] points to two hash tables
[CPP]View Plain copy print?       typedef struct DICTHT {dictentry **table;       unsigned long size;       unsigned long sizemask;   unsigned long used;   } dictht; DICHT[0] is used to actually store data, dicht[1] is typically used to relay data when the hash table element is too rehash.

The table term in dictht really holds the element, each key/value pair is represented by a dictentry and placed in the Dictentry array.

Collection Objects

The encoding of the collection object can be either Intset or Hashtable.

Intset is an integer set, which is stored as an integer of the same type, supporting integers of three lengths: [CPP] view plain copy print? #define INTSET_ENC_INT16 (sizeof (int16_t)) #define INTSET_ENC_INT32 (sizeof (int32_t)) #define INTSET_ENC_INT64 (sizeof   (int64_t)) Intset is an ordered collection that finds the complexity of an element O (Logn), but not necessarily O (Logn), because it may involve an upgrade operation. For example, when the collection is full of int16_t-type integers, then to insert a int32_t, so in order to maintain the consistency of the data type in the collection, then all the data will be converted to the int32_t type, involving the reallocation of memory, when the insertion complexity is O (N). Intset is not supported for degraded operations.
ordered collection objects

The encoding of ordered sets may be two kinds, one is ziplist, the other is the combination of skiplist and dict.

Ziplist as a collection and as a hash object, the member and the score are stored sequentially. According to score from small to large order. Its structure is no longer repeated.

Skiplist is a jumping table that implements a fast lookup in an ordered set, and in most cases it can be as fast as a balanced tree. But its implementation is relatively simple, can be used as a substitute for the balance tree. Its structure is quite special. The following are the structure of the jump table skiplist and its internal node Skiplistnode:

[CPP] View Plain copy print? /*   *  Jump table    */   typedef struct zskiplist {        //  head node, tail node        struct zskiplistnode *header,  *tail;       //  Node Quantity        unsigned  long length;       //  Current maximum number of nodes in the table         int level;  } zskiplist;  /* zsets use a specialized  version of skiplists */  /*   *  Jumping table node    */   typedef  struct zskiplistNode {       // member  Objects        robj *obj;       //  score         double score;       //  back Pointer  &nBsp     struct zskiplistNode *backward;       //  Layer        struct zskiplistLevel {            //  Forward Pointers            struct  zskiplistnode *forward;           //  number of nodes across this layer            unsigned int span;        } level[];  } zskiplistnode;   Head and tail respectively point to the header and tail nodes, Then the structure within each skiplistnode is layered (i.e. level array)

In the figure, this is probably the following:

Each column represents a node that preserves member and score, sorted by score from small to large. Each node has a different number of layers, the number that is randomly generated when the node is generated. Each layer is a pointer to a later node. This structure allows the jump table to be accessed quickly across many nodes.

As mentioned before, the ordered set Zset are formed by jumping table and hashtable together. [CPP] view plain copy print?       typedef struct ZSET {//dictionary dict *dict;   Jumping table zskiplist *ZSL;   } Zset; Why use this structure? Just imagine that if you use Hashtable, you can quickly find, add, and delete elements, but you can't keep the order of the set. If the single use of skiplist, ordering can be guaranteed, but the speed of the search is too slow O (Logn). End

The five object types of Redis and their underlying implementations are briefly introduced. In fact, the efficiency and flexibility of redis benefit from the use of different underlying structures for the same object type and the conversion of the two when necessary, as well as the reasonable utilization of memory by various underlying structures.