Redis Learning Note (7)---compression dictionary zipmap

Zipmap

In the Hashtable implementation, Redis introduces a ZIPMAP data structure that is guaranteed to be stored with less memory when the Hashtable is created and the elements are small, while the efficiency of the query is not greatly affected.
Zipmap uses strings to implement a simple hash table to store a small number of key-value pairs.

Memory layout

The Zipmap memory layout is as follows:
　　
1) zmlen:1 bytes to record the number of key-value pairs in the current zipmap. Since Zmlen has only 1 bytes, it is stated that the number can only be 0~254, and when zmlen>254, it is necessary to traverse the entire zipmap to get the number of key-value pairs.
2) Len: Used to record the length of key or value, there are two cases when Len's first byte is 0~253, then Len occupies only one byte. The first byte of the Len is 254, then Len is represented by the 4 bytes that follow. So len either takes 1 bytes or consumes 5 bytes.
3) free:1 bytes, which represents the number of free bytes after the subsequent value, is mainly caused by changing the value of key, such as "foo" + "bar" into "foo" and "HI", which results in 1 bytes of free space. When the number of bytes of free is too large to be represented by 1 bytes, the Zipmap will reallocate memory to ensure that the string is as compact as possible.
4) end:1 bytes, 0xFF, used to mark the end of Zipmap
A simple example is as follows:
"\x02\x03foo\x03\x00bar\x05hello\x05\x00world\xff"
Can be found: zmlen=2 key1_len=3 key1= "foo" value1_len=3 free_len=0 value1= "Bar"
key2_len=5 key2= "Hello" value2_len=5 free_len=0 value2= "World" end=0xff
That is, the current zipmap has two key-value pairs, respectively "foo" = "bar" and "Hello" and "World"

Create Zipmap

unsignedchar *zipmapNew(void) {    unsignedchar *zm = zmalloc(2);    zm[00/* Length */    zm[1] = ZIPMAP_END;    return zm;}

For an empty Zipmap only 2 bytes, 1 bytes of zmlen=0,1 a byte of End=0xff
　　

Find

//Find key in Zm, when Totlen=null, indicates no need to get the total number of bytes occupied by ZipmapStatic unsigned Char*zipmaplookupraw (unsigned Char*ZM,unsigned Char*key,unsigned intKlen,unsigned int*totlen) {unsigned Char*p = zm+1, *k =NULL;unsigned intL,llen; while(*p! = zipmap_end) {unsigned CharFree L = zipmapdecodelength (p);//Get Key_lenLlen = Zipmapencodelength (NULL, l);//Get the number of bytes occupied by Key_len        if(Key! =NULL&& k = =NULL&& L = = Klen &&!memcmp (p+llen,key,l)) {if(Totlen! =NULL) {k = p; }Else{///Zipmap bytes not required, direct return                returnP }        }//Skip current Key-value pair, compare nextp + = Llen+l;        L = zipmapdecodelength (p); p + = Zipmapencodelength (NULL, l); Free = p[0]; P + + +1+free;/ * +1 to skip the free byte * /}//Otherwise use Totlen to record the number of bytes occupied by Zipmap    if(Totlen! =NULL) *totlen = (unsigned int) (P-ZM) +1;returnK }

Set operation

unsigned Char*zipmapset (unsigned Char*ZM,unsigned Char*key,unsigned intKlen,unsigned Char*val,unsigned intVlenint*update) {unsigned intZmlen, offset;unsigned intFreelen, Reqlen = Zipmaprequiredlength (Klen,vlen);unsigned intEmpty, Vempty;unsigned Char*p; Freelen = Reqlen;if(update) *update =0; p = Zipmaplookupraw (Zm,key,klen,&zmlen);//First find key in Zipmap    if(p = = NULL) {//When a key does not exist in the Zipmap, the extended memoryZM = Zipmapresize (ZM, Zmlen+reqlen); p = zm+zmlen-1; Zmlen = Zmlen+reqlen;if(zm[0] < Zipmap_biglen) zm[0]++; }Else{key already in//zipmap, you need to update its value to Val        if(update) *update =1; Freelen = Zipmaprawentrylength (p);if(Freelen < Reqlen) {//If free space is insufficient, the memory needs to be extendedoffset = P-ZM;            ZM = Zipmapresize (ZM, Zmlen-freelen+reqlen);            p = zm+offset; Memmove (P+reqlen, P+freelen, zmlen-(offset+freelen+1));            Zmlen = Zmlen-freelen+reqlen;        Freelen = Reqlen; }    }//Move the current key-value to the back of the content and reserve the spaceempty = Freelen-reqlen;if(Empty >= Zipmap_value_max_free)        {offset = P-ZM; Memmove (P+reqlen, P+freelen, zmlen-(offset+freelen+1));        Zmlen-= empty;        ZM = Zipmapresize (ZM, Zmlen);        p = zm+offset; Vempty =0; }Else{vempty = empty; }//write key-value to Zipmapp + = Zipmapencodelength (P,klen);memcpy(P,key,klen);    p + = Klen;    p + = Zipmapencodelength (P,vlen); *p++ = Vempty;memcpy(P,val,vlen);returnZM;}

Get operation

intZipmapget (unsigned Char*ZM,unsigned Char*key,unsigned intKlen,unsigned Char**value,unsigned int*vlen) {unsigned Char*p;if(P = Zipmaplookupraw (Zm,key,klen,NULL)) ==NULL)return 0;    p + = Zipmaprawkeylength (p);    *vlen = Zipmapdecodelength (p); *value = p + zipmap_len_bytes (*vlen) +1;return 1;}

Returns 1 indicates a successful lookup. When the lookup succeeds, the address and value_len of value are saved separately in value and Vlen.

The source code referenced in this article is all from the Redis3.0.7 version

Redis Learning Resources:
https://github.com/huangz1990/redis-3.0-annotated
Redis Design and Implementation (second edition)
http://blog.csdn.net/xiejingfa/article/details/51111230

Redis Learning Note (7)---compression dictionary zipmap