Redis source code-SDS string of Data Structure

In addition to accompanying Lu You on National Day, it took almost two days to get a preliminary understanding of redis. In the next period of time, I will thoroughly read the redis source code and make a record to read the source code, I think we should start with the basic modules. For example, the data structure and redis have implemented a lot of data structures. Of course, many open-source projects also implement various data structures to meet customization requirements, I first read the underlying string module-SDS

SDS definition:

typedef char *sds;

To put it bluntly, it is the alias of the string. This is used for the entire redis string, but a structure is used internally to maintain this SDS.

struct sdshdr {                     unsigned int len;                           unsigned int free;                                         char buf[];                                                                          }; 

The reason for this design is to meet some characteristics of redis. In general, SDS has the following advantages over C strings:

1. The complexity of obtaining the string length is O (1), because an internal Len is maintained to save the string length

static inline size_t sdslen(const sds s) {    struct sdshdr *sh = (void*)(s-(sizeof(struct sdshdr)));    return sh->len;}static inline size_t sdsavail(const sds s) {    struct sdshdr *sh = (void*)(s-(sizeof(struct sdshdr)));    return sh->free;}

If you want to access the Len and free fields, you also need to set the pointer until the struct starts to appear. At that time, I hesitated to see it here. The basic knowledge is not good --!

2. Binary Security

Here we need to explain what is binary security, and the strings in the C language are represented by ASCII code, but here we use a character array, how to store it, and what to take out, for example: test \ 0test \ 0, then Len will be 9, and the read result will also be test \ 0test \ 0, instead of saying that reading the first Terminator will end.

    sds s = sdsnewlen("test", 5);     printf("%s\n", s);     size_t len = sdslen(s);    printf("%zu\n", len);    size_t free = sdsavail(s);    printf("%zu\n", free);

We can obtain Len = 5.

Source code:

/* Create a new sds string with the content specified by the 'init' pointer * and 'initlen'. * If NULL is used for 'init' the string is initialized with zero bytes. * * The string is always null-termined (all the sds strings are, always) so * even if you create an sds string with: * * mystring = sdsnewlen("abc",3"); * * You can print the string with printf() as there is an implicit \0 at the * end of the string. However the string is binary safe and can contain * \0 characters in the middle, as the length is stored in the sds header. */sds sdsnewlen(const void *init, size_t initlen) {    struct sdshdr *sh;     if (init) {        sh = malloc(sizeof(struct sdshdr)+initlen+1);    } else {        sh = calloc(1,sizeof(struct sdshdr)+initlen+1);    }        if (sh == NULL) return NULL;    sh->len = initlen;    sh->free = 0;     if (initlen && init)        memcpy(sh->buf, init, initlen);    sh->buf[initlen] = '\0';    return (char*)sh->buf;}

3. There will be no buffer overflow. The free field can reduce the memory reallocation caused by modifying the string length.

We know that the two functions in C language will cause buffer overflow if they are improperly used.

char *strcat(char *dest, const char *src);char *strncat(char *dest, const char *src, size_t n);The  strcat()  function  appends the src string to the dest string, overwriting the null byte ('\0') at the end of dest, and then adds a terminating null byte.  The strings may not overlap, and the dest string must have enough space for the result.The strncat() function is similar, except that*  it will use at most n characters from src; and*  src does not need to be null terminated if it contains n or more characters.As with strcat(), the resulting string in dest is always null terminated.

The CAT implementation in SDS is as follows:

/* Append the specified binary-safe string pointed by 't' of 'len' bytes to the * end of the specified sds string 's'. * * After the call, the passed sds string is no longer valid and all the * references must be substituted with the new pointer returned by the call. */sds sdscatlen(sds s, const void *t, size_t len) {    struct sdshdr *sh;     size_t curlen = sdslen(s);    s = sdsMakeRoomFor(s,len);    if (s == NULL) return NULL;    sh = (void*) (s-(sizeof(struct sdshdr)));    memcpy(s+curlen, t, len);    sh->len = curlen+len;    sh->free = sh->free-len;    s[curlen+len] = '\0';    return s;}

How is sdsmakeroomfor implemented?

/* Enlarge the free space at the end of the sds string so that the caller * is sure that after calling this function can overwrite up to addlen * bytes after the end of the string, plus one more byte for nul term. * * Note: this does not change the *length* of the sds string as returned * by sdslen(), but only the free buffer space we have. */sds sdsMakeRoomFor(sds s, size_t addlen) {    struct sdshdr *sh, *newsh;    size_t free = sdsavail(s);    size_t len, newlen;    if (free >= addlen) return s;    len = sdslen(s);    sh = (void*) (s-(sizeof(struct sdshdr)));    newlen = (len+addlen);    if (newlen < SDS_MAX_PREALLOC)        newlen *= 2;    else         newlen += SDS_MAX_PREALLOC;    newsh = realloc(sh, sizeof(struct sdshdr)+newlen+1);    if (newsh == NULL) return NULL;    newsh->free = newlen - len;     return newsh->buf;}

We can see that SDS has such a rule for modifying the string length. If

Addlen <free use free

Addlen> Free & addlen <1 m free = Len

Addlen> Free & addlen> 1 m free = 1 m

redis source code-SDS string of Data Structure