Memcached learning notes-storage command source code analysis part 2: memcached learning notes

Memcached learning notes-storage command source code analysis part 2: memcached learning notes

Last Review: memcached Study Notes-storage command source code analysis part I analyzes the process of memcached's storage command source code and describes how memcached parses text commands and the memory management mechanism of mencached.

This article continues the previous article to analyze the source code of the storage command. After successful memory allocation in the previous article, this article mainly explains: 1. memcached storage mode; 2. Differences between add and set commands.

Memcached Storage Method

Hash table (HashTable)

Hash Tables are widely used in practice. For example, the compiler usually maintains a symbol table to store tags. Many advanced languages also explicitly support hash tables. Hash Tables generally provide Search, Insert, and Delete operations. These operations have the same performance as the linked list in the worst case ). However, it is usually not so bad. A hash algorithm with proper design can effectively avoid such situations. Generally, the time complexity of these operations in a hash table is O (1 ). This is also why it is loved.

Memcached stores all items through a HashTable (note: the slab model of memcached manages memory and HashTable stores data ), the solution to HashTable hash conflicts in memcached is the link method. The efficient query of memcached can be attributed to HashTable.

The HashTable of memcached is the declaration and operation in the assoc. c file. Let's take a look at the HashTable Declaration and related operation definitions of memcached.

1/* primary_hashtable is the main HashTable */2 static item ** primary_hashtable = 0; 3 4/* If memcached extends HashTable, the old data will be stored in old_hashtable */5 static item ** old_hashtable = 0; 6 7/* Number of items saved in HashTable */8 static unsigned int hash_items = 0; 9 10/* expanding indicates whether HashTable has been extended, the query will be performed in primary_hashtable and old_hashtable. Otherwise, the query will only be performed in primary_hashtable */11 static bool expanding = false; 12 13/* HashTable initialization function */14 void assoc_init (const int hashpower_init); 15/* HashTable query operation */16 item * assoc_find (const char * key, const size_t nkey, const uint32_t hv); 17/* HashTable insert operation */18 int assoc_insert (item * item, const uint32_t hv ); 19/* HashTable delete operation */20 void assoc_delete (const char * key, const size_t nkey, const uint32_t hv );

After memcached memory is allocated successfully, a new item is returned, and the item is saved to HashTable. complete_nread_ascii> store_item> do_store_item

In complete_nread_ascii (memcached. c), store_item (thread. c) returns the final result of this command to the client based on the returned result.

1 static void complete_nread_ascii (conn * c) {2 assert (c! = NULL); 3 4 item * it = c-> item; 5 int comm = c-> cmd; 6 enum store_item_type ret; 7 8 pthread_mutex_lock (& c-> thread-> stats. mutex); 9 c-> thread-> stats. slab_stats [it-> slabs_clsid]. set_cmds ++; 10 pthread_mutex_unlock (& c-> thread-> stats. mutex); 11 12 if (strncmp (ITEM_data (it) + it-> nbytes-2, "\ r \ n", 2 )! = 0) {13 out_string (c, "CLIENT_ERROR bad data chunk"); 14} else {15 ret = store_item (it, comm, c ); // memcached stores the item operation 16 17 18 //........ 19 20 21 switch (ret) {22 case STORED: 23 out_string (c, "STORED"); // After the storage is successful, the client returns 24 break; 25 case EXISTS: 26 out_string (c, "EXISTS"); 27 break; 28 case NOT_FOUND: 29 out_string (c, "NOT_FOUND"); 30 break; 31 case NOT_STORED: 32 out_string (c, "NOT_STORED"); // when we store an existing key through add, we will get the result 33 break; 34 default: 35 out_string (c, "SERVER_ERROR Unhandled storage type. "); 36} 37 38} 39 40 //......... 41}

Store_item (thread. c)

1 enum store_item_type store_item (item * item, int comm, conn * c) {2 enum store_item_type ret; 3 uint32_t hv; 4 5 // memcached calculate the hash value of the current item key based on the hash algorithm. The hash value is hv 6 hv = hash (ITEM_key (item), item-> nkey, 0 ); 7 8 item_lock (hv); 9 // core operation of memcached storage item 10 ret = do_store_item (item, comm, c, hv); 11 item_unlock (hv); 12 ret return; 13}

Do_store_item (memcached. c)

1 enum store_item_type do_store_item (item * it, int comm, conn * c, const uint32_t hv) {// comm is the command 2 char * key = ITEM_key (it ); 3 4 // query whether the corresponding item 5 item * old_it = do_item_get (key, it-> nkey, hv); 6 7 // storage result. The initial value is NOT_STORED 8 enum store_item_type stored = NOT_STORED; 9 10 item * new_it = NULL; 11 int flags; 12 13 if (old_it! = NULL & comm = NREAD_ADD) {14/* add only adds a nonexistent item, but promote to head of LRU */15 // The data item is not empty, update time 16 // If the add command is called and the corresponding key already exists, then the usage time is modified. The stored value is still NOT_STORED, 17 // Therefore, call add to add an existing item. The NOT_STORED result is displayed. The value corresponding to the key is not changed. In complete_nread, the information 18 do_item_update (old_it) is output ); 19 20} else if (! Old_it & (comm = NREAD_REPLACE21 | comm = NREAD_APPEND | comm = NREAD_PREPEND) 22 {23 24 //.......... 25 26} else if (comm = NREAD_CAS) {27 28 //............ 29 30} else {31 // old_it is empty, and comm is add, set, replace, append; or old_it is not empty, and comm is set, replace, append32/* 33 * Append-combine new and old record into single one. here it's 34 * atomic and thread-safe.35 */36 if (comm = NREAD_APPEND | co Mm = NREAD_PREPEND) {37 38 //.......... 39 40} 41 42 // storage result. The initial value is NOT_STORED43 if (stored = NOT_STORED) {44 if (old_it! = NULL) 45 item_replace (old_it, it, hv); // old_it is not empty, and the command is set: replace the old item old_it46 else47 do_item_link (it, hv) with the new item it in HashTable; // The old_it is empty, and the command is add, set: insert item it into HashTable 48 49 c-> cas = ITEM_get_cas (it); 50 51 stored = STORED; // modify the storage Result 52} 53} // end if54 55 56 //......... 57 58 return stored; 59}

Finally, let's take a look at the query operations of the assoc_find function HashTable.

1 item * assoc_find (const char * key, const size_t nkey, const uint32_t hv) {2 item * it; 3 unsigned int oldbucket; 4 5 // as above: expanding indicates whether HashTable has been extended. If extended, it will be queried in primary_hashtable and old_hashtable, otherwise, only the hash values of key and hv are queried in primary_hashtable, first, locate the linked list of item 7 if (expanding & 8 (oldbucket = (hv & hashmask (hashpower-1)> = expand_bucket) 9 {10 it = old_hashtable [oldbucket]; 11} else {12 it = primary_hashtable [hv & hashmask (hashpower)]; 13} 14 15 item * ret = NULL; 16 int depth = 0; 17 // traverse the linked list it found above and query item18 corresponding to the key from it // return the found item ret; otherwise, NULL19 while (it) is returned) {20 if (nkey = it-> nkey) & (memcmp (key, ITEM_key (it), nkey) = 0) {21 ret = it; 22 break; 23} 24 it = it-> h_next; 25 ++ depth; 26} 27 MEMCACHED_ASSOC_FIND (key, nkey, depth); 28 return ret; 29}

Differences between add and set commands

As can be seen from the do_store_item function, 1) if the add command is used but the value corresponding to the key already exists, it is only the latest time used to update the key, and the value is not overwritten by the new value, returns the NOT_STORED result. 2) If it is the add command and STORED for the first time, the value is added to HashTable and the STORED result is returned. 3) if it is the set command, no matter whether the value corresponding to the key already exists, the new value is inserted into HashTable, And the STORED result is returned.

Finally, I would like to thank you for reading the greetings from your younger brother. Thank you!

(End)

For more information, see www.hcoding.com.