SDS for the underlying data structure of Redis

Recently, I wanted to learn Redis by using Redis's source code. Although the usual work is not used much, but the Redis is still more interested in, after all, its performance is good. Redis is an open source project that we can use to understand Redis through source code. I will later through their own study, write some about the Redis source code posts. The main content of the post is the analysis of the code design, and does not explain the source code in detail. If there is a wrong place, please correct me. Source code is Reids 3.0.3 version.

Sds

A. Redis string SDS:

The C language string is char *,redis, which defines the internal string SDS itself, and also provides a set of SDS-related operational functions that Redis does to make it easier to use strings. First look at the definition of SDS:

typedef char *SDS;STRUCT SDSHDR {unsigned int len;    string length unsigned int free;        Remaining space size char buf[]; string storage address};

First look at the struct SDSHDR, which is the header information for the Redis string, and the fields are commented on. Such a string design is associated with a string-related operation. To get a better understanding of SDS, let's take a look at how SDS stores "Hello World" in the first instance. Here's an example:

Len = 11free = 5buf[] = "Hello World" + ' + ' + 5 byte space

Where len=11 refers to the length of the string, free=5 refers to the remaining storage space of SDS, the actual size of BUF is Len + 1 + free, where 1 refers to the space size of '% '.

From the data structure definition and the above example can be seen:

1. Len is the length of the string, which is O (1) for the length of the string by recording the length, but this also introduces Len's maintenance problem, and the Len field needs to be updated when the string length is updated.

2. Free is the remaining space and the remaining space is used to extend the length of the string. But the remaining space is limited, in order to ensure that the length of the string can grow freely, need some method to increase the remaining space.

3. BUF the address of the string store, declaring the BUF as a 0-length char array, is a commonly used content space allocation design method. In general, such data structures are dynamic application space. This can be used in the dynamic application space, the size of the data structure itself +buf size of space, to move applications, now also can be re-applied to expand the space. The implementation of Sdsnewlen and SDSMAKEROOMFOR functions can be seen concretely.

4. ' Buf ', the end of the string is always ' s ', the purpose is to allow SDS to align with the C language string, can be directly output through functions such as printf. However, it is important to note that ' + ' does not represent the end of the SDS string, because the SDS string may also contain the ' \ S ' character. The end of the string can only be specified by Len. This design string, not only can support the C language string, but also can be extended to the binary string, just like std::string.

5. Binary storage is supported. Since the length of the SDS string is specified by Len, each byte in the string can overwrite all values in the binary, so binary storage can be supported. Why should Redis support the storage of binary strings? I think it's because Redis needs to indirectly support the storage of complex data structures. Because Redis itself only provides the storage of common data structures such as String,list,set,zset,hash, Redis does not directly support its storage for the user's own defined structure. Redis provides binary storage that allows users to serialize complex data structures into binary strings and then store them. This will require serialization at the time of storage, deserialization of the fetch, CPU consumption, but the commonality of the store.

6. typedef char * SDS. SDS is defined as char* instead of sdshdr*. The main reason is that to make the SDS look more like char*, there are some functions that can be used with char* without distinction, such as printf. This also comes at a cost, which is the increased complexity of SDS-related operations. Almost every SDS-related operation is done by SDS (char*) to locate the sdshdr*.

SDS also has some drawbacks:

Sdshdr occupy a certain space
There is space left in the string, although it can be used for expansion, but it can also be a waste.
It is necessary for the code itself to provide a set of related operational functions for better use.
When using SDS, it is necessary to understand the characteristics of SDS and the usage specification of its operation function, which is very high for programmers.

Second, SDS related operation function:

To make it easier for users to use Sds,redis, a series of operational functions for SDS are provided. Such as:

Sdslen string length Sdsavail string available space length (free) sdsnew Create string sdsdup copy string Sdsfree release string Sdscat string concatenation ...

Not listed here.
SDS and its manipulation functions, together provide a handy string for Redis, but also increase the complexity of string manipulation. When using SDS and its operational functions, it is important to understand the behavior characteristics of SDS to be used correctly.

To give a function to analyze, sdsmakeroomfor:

/* enlarge the free space at the end of the sds string  so that the caller * is sure that after calling this  function can overwrite up to addlen * bytes after the  end of the string, plus one more byte for nul term. *  * note: this does not change the *length* of the sds  string as returned * by sdslen (), but only the free  buffer space we have. *//*  increases the amount of Addlen in the SDS space, in order to increase the string length.  *  If the remaining space of the string itself is greater than Addlen, there is no need to increase the SDS space, otherwise increase the space.  */sds sdsmakeroomfor (Sds s, size_t addlen)  {    struct  sdshdr *sh, *newsh;    size_t free = sdsavail (s);     size_t len, newlen;    /*  If the remaining space is sufficient to store addlen, there is no need to add space  */     if  (Free >= addlen)  return s;    len =  sdslen (s);    sh =  (void*)   (s (sizeof (STRUCT&NBSP;SDSHDR)));     /*  Calculate SDSHDR Address  */    newlen =  (Len+addlen);  /*   The total length of string added after  */    /*  pre-allocated space strategy if the new length is less than  sds_max_prealloc (1M),       pre-allocates more than one length of space, otherwise pre-allocates more Sds_max_prealloc size  */    if  (newlen  < sds_max_prealloc)         newlen *= 2;     else        newlen += SDS_MAX_PREALLOC;     /*  application for Extended space  */    newsh = zrealloc (sh,  sizeof (STRUCT&NBSP;SDSHDR) +newlen+1);    if  (newsh == null)  return NULL;     newsh->free = newlen - len;    return newsh->buf;}

The above function sdsmakeroomfor there will be a pre-allocated space strategy, it should be noted that the pre-allocated space can be reserved space for the string increase, but also cause waste. Not all strings will have pre-allocated space, and only sdsgrowzero,sdscatlen,sdscpylen,sdscatfmt functions will call sdsmakeroomfor.

This article is from the "Chhquan" blog, make sure to keep this source http://chhquan.blog.51cto.com/1346841/1769854

SDS for the underlying data structure of Redis