A tutorial on dynamic string learning in Redis _redis

Use of SDS
The main functions of Sds in Redis are as follows: two.

Implements a String object (Stringobject);
Used as a char* type of substitute within the Redis program;
The following two subsections describe each of these two uses.

Implementing a String Object

Redis is a key-value pair database (Key-value DB), where the value of a database can be a string, a collection, a list, and many other types of objects, while the database's keys are always string objects.

For string objects that contain string values, each string object contains a SDS value.

"String object containing String value", this may sound a bit strange at first, but in Redis, a string object can hold a Long value in addition to the string value, so for the sake of rigor, here's the thing to emphasize: When a string object holds a string, it packs Contains the SDS value, otherwise, it is a long type value.
For example, the following command creates a new database key-value pair whose keys and values are string objects that contain a SDS value:

redis> SET Book "Mastering C + +"
OK redis> get book
"Mastering C + + in"

The following command creates another key-value pair whose key is a string object, while the value is a collection object:

Redis> sadd NoSQL "Redis" "MongoDB" "neo4j"
(integer) 3

redis> smembers nosql
1) "neo4j"
2) " Redis "
3" "MongoDB"

Use SDS instead of C default char* type

Because the char* type has a single function, a low level of abstraction, and does not efficiently support some redis commonly used operations (such as append operations and length calculation operations), the REDIS program uses SDS instead of char* to represent strings in most cases.

Performance issues when you introduce the SDS definition later, you'll say, since we have not yet understood the other functional modules of Redis, there is no way to say that SDS is used in detail, but in later chapters we will often see that other modules (almost every one) use the SDS type value.

For now, just remember this fact: in Redis, the protocol content of the client incoming server, the AOF cache, the reply returned to the client, and so on, these important contents are saved by the SDS type.

String in the Redis
in C, a string can be represented by a char array at the end of a.

For example, Hello World can be expressed as "Hello World\0" in the C language.

This simple string representation, in most cases, satisfies the requirement, but it does not effectively support both the length calculation and the append (append) operation:

The complexity of each computed string length (strlen (s)) is θ (N).
N-time append to a string, it is necessary to do N-time memory redistribution (realloc) of the string.
In Redis, strings are often appended and calculated in length, and APPEND and STRLEN are the two operations that are directly mapped in the Redis command, and these two simple operations should not be a performance bottleneck.

In addition, Redis in addition to handling C strings, but also to deal with simple byte arrays, as well as server protocols, and so on, so for convenience, the Redis string representation should also be binary security: The program should not be stored in the string of data to make any assumptions, data can be the end of the C A string, or it can be a simple byte array, or another format of data.

For these two reasons, Redis uses the SDS type to replace the default string representation of the C language: SDS can efficiently implement both append and length calculations, and is binary safe.

The implementation of SDS

In the previous section, we have been describing SDS as an abstract data structure, in fact, its implementation consists of the following two parts:

typedef char *SDS;


struct SDSHDR {

  //buf occupied length
  int len;

  BUF remaining available length
  int free;

  Where the string data is actually stored
  char buf[];

Where the type SDS is the alias (alias) of char *, and the struct SDSHDR holds the three attributes of Len, free, and BUF.

As an example, the following is a newly created SDSHDR structure that also holds the Hello World string:

struct SDSHDR {
  len = one;
  free = 0;
  BUF = "Hello world\0"; The actual length of the BUF is Len + 1
};

With the Len attribute, SDSHDR can implement a length calculation operation with a complexity of θ (1).

On the other hand, by allocating some extra space to the BUF and using free to record the amount of unused space, SDSHDR can significantly reduce the amount of memory that is required to perform the append operation, which we will discuss in detail in the next section.

Of course, SDS also requires a proper implementation of the operation-all functions that handle SDSHDR must correctly update the Len and free attributes, or they can cause bugs.

Data type definition
There are two data types related to SDS implementation, one is SDS:

  Alias of String type 
  typedef char *SDS; 

The other is SDSHDR:

  Structure holding SDS 
  struct SDSHDR { 
    //buf number of string spaces already in use 
    int len; 
    The amount of space reserved for strings in buf 
    int free; 
    The place where the string is actually stored 
    char buf[]; 
   

Where SDS is just the alias of the string array type char*, and SDSHDR is used to hold and preserve the SDS information

For example, Sdshdr.len can be used to get the actual length of the string stored in the sdshdr.buf in the O (1) complexity, while Sdshdr.free is used to save the amount of space reserved in the SDSHDR.BUF

(Here SDSHDR should be the SDS handler abbreviation)

Using SDSHDR as SDS
The SDS module uses a little trick with the SDSHDR structure: It enables the SDSHDR structure to be passed and processed as the SDS type, and revert to the SDSHDR type when needed.

This technique is understood by the following function definition

The Sdsnewlen function returns a new SDS value, which in fact creates a SDSHDR structure:

  SDS Sdsnewlen (const void *init, size_t initlen) 
  {struct SDSHDR 
    ; 
   
    if (init) {/ 
      /create 
      sh = malloc (sizeof (struct SDSHDR) + Initlen + 1); 
    } else { 
      //redistribute 
      sh = calloc (1, siz EOF (struct SDSHDR) + Initlen + 1); 
    } 
   
    if (sh = = null) return null; 
   
    Sh->len = Initlen; 
    Sh->free = 0;  Just start free for 0 
   
    if (initlen && init) { 
      memcpy (sh->buf, Init, Initlen); 
    } 
    Sh->buf[initlen] = ' the '; 
   
    Returns only sh->buf this string part return 
    (char *) sh->buf; 
  } 

By using variables to hold the value of a SDS, you can pass the SDS directly to those functions that deal only with the SDS value itself. For example, the Sdstoupper function is one example:

  Static inline size_t Sdslen (const SDS s) 
  { 
    //The corresponding SDSHDR structure is calculated from SDS struct SDSHDR 
    = (void *) (S-*sh (str UCT sdshdr)); 
   
    Return sh->len; 
  } 
   
   
  void Sdstoupper (SDS s) 
  { 
    int len = Sdslen (s), J; 
   
    for (j = 0; J < Len; j + +) 
      s[j] = ToUpper (S[j]); 
   

Here is a trick, through the pointer operation, you can calculate the corresponding SDSHDR structure from the SDS value:

SDS is a buf to char * (PS: And the empty array does not occupy memory space, the array name is the memory address), but the allocation is allocated sizeof (struct SDSHDR) + Initlen + 1, through sds-sizeof (struct SDSHDR) can calculate the first address of the struct SDSHDR, so that the information of Len and free can be obtained

The Sdsavail function is an example of using this technique:

  Static inline size_t sdsavail (const SDS s) 
  { 
    struct sdshdr *sh = (void *) (S-(sizeof (struct SDSHDR))); 
   
    Return sh->free; 
  } 

Memory allocation function implementation
the functions associated with Reids implementation decisions are sdsmakeroomfor:

  SDS Sdsmakeroomfor (SDS s, size_t Addlen) 
  { 
    struct sdshdr *sh, *newsh; 
    size_t free = Sdsavail (s); 
    size_t Len, Newlen; 
   
    The reserved space can satisfy the local stitching  
    if (free >= addlen) return s; 
   
    Len = Sdslen (s); 
    SH = (void *) (S-(sizeof (struct SDSHDR)); 
   
    Set the string length of the new SDS 
    //This length is larger than the actual length required to complete the stitching 
    (Newlen = (len + addlen) to optimize the next splicing operation through reserving space; 
    if (Newlen < 1024 * 1024) 
      Newlen *= 2; 
    else 
      Newlen + = 1024; 
   
    Reassign Sdshdr 
    newsh = realloc (sh, sizeof (struct SDSHDR) + Newlen + 1); 
    if (newsh = null) return null; 
   
    Newsh->free = Newlen-len; 
   
    Returns only the string part return 
    newsh->buf; 
  } 

This memory allocation strategy shows that when the SDS value is extended (expand), additional space is always reserved, reducing the number of times that the memory is reallocated (reallocate) by spending more memory, and optimizing the processing speed of the next extended operation

Then the Redis if the implementation of the SDS string extension of the method to paste a good idea:

  /** 
   * Expand SDS by Len length and splice t to the end of SDS 
   /SDS 
  Sdscatlen (SDS s, const void *t, size_t len) 
  { 
    struct SDSHDR * SH; 
   
    size_t Curlen = Sdslen (s); 
   
    O (N) 
    s = sdsmakeroomfor (s, Len); 
    if (s = = null) return null; 
   
    Copy 
    memcpy (s + curlen, T, Len); 
   
    Update Len and free properties 
    sh = (void *) (S-(sizeof (struct SDSHDR))); 
    Sh->len = Curlen + len; 
    Sh->free = sh->free-len; 
   
    Non-terminal 
    S[curlen + len] = ' the "; 
   
    return s; 
  } 
   
  /** 
   * Spell a char array at the end of SDS/SDS 
  Sdscat (SDS s, const char *t) 
  {return 
    Sdscatlen (S, T, strlen (t) ); 
  }