Redis Memory Management (i)

The files for the memory management functions of the Redis database are: Zmalloc.h and ZMALLOC.C.

Redis author in the memory management module to consider the view system is installed in the Tcmalloc or Jemalloc module, these two are already a long time memory management module, code stability, excellent performance, if installed, then use, Finally check whether it is a Mac system, if it is the Mac system loaded file different, the amount, I did not have a Mac programming, this piece is not considered. The corresponding source code is:

1 //Check if Tcmalloc,tcmalloc (thread-caching malloc) is defined with the same functionality as the malloc implementation of the standard GLIBC library, but Tcmalloc is much higher in efficiency and speed than standard malloc2 #ifDefined (USE_TCMALLOC)3     #defineZmalloc_lib ("tcmalloc-" __xstr (tc_version_major) "." __xstr (Tc_version_minor))4#include <google/tcmalloc.h>5 6     #if(Tc_version_major = = 1 && tc_version_minor >= 6) | | (Tc_version_major > 1)7         #defineHave_malloc_size 18         #defineZmalloc_size (P) tc_malloc_size (p)9     #elseTen         #error"Newer version of Tcmalloc required" One     #endif A //Check if the Jemalloc library is defined - #elifDefined (USE_JEMALLOC) -     #defineZmalloc_lib ("jemalloc-" __xstr (Jemalloc_version_major) ". __xstr (Jemalloc_version_minor)". "__xstr (JEMALLOC_ Version_bugfix)) the#include <jemalloc/jemalloc.h> -     #if(Jemalloc_version_major = = 2 && jemalloc_version_minor >= 1) | | (Jemalloc_version_major > 2) -         #defineHave_malloc_size 1 -         #defineZmalloc_size (P) je_malloc_usable_size (p) +     #else -         #error"Newer version of Jemalloc required" +     #endif A //Check if it's an Apple system at #elifDefined (__apple__) -#include <malloc/malloc.h> -     #defineHave_malloc_size 1/* tag has found a usable library of ready-made functions * * -     #defineZmalloc_size (P) malloc_size (p)/* Because each library is implemented differently, the function of the test size is different */ - #endif

First, the whole memory management of all the interface functions, and then slowly analyze, all the interface functions are:

void *zmalloc (size_t size);//override malloc function, request memory void *zcalloc (size_t size);// Overriding the Calloc function, which no longer supports multiply requests by block, internally calls Zmalloc 
 void *zrealloc (void *ptr, size_t size),//overriding memory extension function 
 void zfree (void *ptr); /overrides the memory deallocation function, which updates the value of the used memory when released, and a concurrency error may occur if thread-safe mode is not turned on in multithreading. 
 Char *zstrdup (const char *s);//string persistence Store function function to allocate memory for strings within the heap. 
 size_t zmalloc_used_memory (void);//Gets the memory size function already used. 
 void zmalloc_enable_thread_safeness (void);//Set memory management to multithreaded Safe mode, which is then mutually exclusive with the mutex when the memory size is updated. 
 void Zmalloc_set_oom_handler (void (*oom_handler) (size_t));//function called when a memory exception is set. 
 Float zmalloc_get_fragmentation_ratio (size_t RSS);//
 size_t zmalloc_get_rss (void);//gets all memory sizes that the process can use. 
 size_t zmalloc_get_private_dirty (void);//
 size_t Zmalloc_get_smap_bytes_by_field (char *field); 
 void Zlibc_free (void *ptr);//Releases the pointer to a memory function, which frees the memory by using the value of the memory variable. 
 #ifndef have_malloc_size 
 size_t zmalloc_size (void *ptr);//Gets the memory block total problem size function. 
 #endif 

Before the detailed implementation of the analytic function, first analyze the. c file to begin processing. Three variables are defined in the. c file, one to record the size of the memory already in use, one to record whether the multithreaded security mode is turned on, and one to be a mutex.

static size_t used_memory = 0;/* record the size of the RAM already used */static int zmalloc_thread_safe = 0;/* Whether thread safety is turned on */pthread_mutex_t used_ Memory_mutex = pthread_mutex_initializer;/* Mutex, if multithreading security is turned on, and the compiler does not support atomic operation functions, then a mutex is required to complete the mutex operation of the code. */

Next to determine whether the memory block before the need to add a head, because it is not possible to determine exactly what the memory allocation function, Tcmalloc or Jemalloc Library in the application of memory block before the addition of a small block of memory to record the use of the memory block, because I did not analyze the two libraries, Specific do not do analysis, the specific code is:

#ifdef have_malloc_size//will define this macro when it finds an existing memory management library, if it is already defined, it no longer increases the head size and, if not defined, determines the length of the head size to be increased based on the specific system. #define PREFIX_SIZE (0)//Head length 0#else #if defined (__sun) | | Defined (__SPARC) | | Defined (__sparc__) #define PREFIX_SIZE (sizeof (long))//The size of the head length is Longlong #else #define PREFIX_SIZE (sizeof (size _t)//The size of the head length is size_tde #endif #endif

The next step is to define the two operating functions Update_zmalloc_stat_add and update_zmalloc_stat_sub that update the memory consumption variables, which support multithreading.

1 /*first check whether the compiler supports atomic operation functions, if supported, there is no mutex lock, after all, the efficiency of the lock is very low. */2 #ifDefined (__atomic_relaxed)//this seems to be the C++11 standard began to support, the specific is not very clear3     #defineUpdate_zmalloc_stat_add (__n) __atomic_add_fetch (&used_memory, (__n), __atomic_relaxed)4     #defineUpdate_zmalloc_stat_sub (__n) __atomic_sub_fetch (&used_memory, (__n), __atomic_relaxed)5 #elifDefined (have_atomic)//this is supported by GCC from a certain version, not all versions of GCC support the Sync series function. 6     #defineUpdate_zmalloc_stat_add (__n) __sync_add_and_fetch (&used_memory, (__n))7     #defineUpdate_zmalloc_stat_sub (__n) __sync_sub_and_fetch (&used_memory, (__n))8 #else//a mutex can only be used in cases where atomic operations are not supported. 9     #defineUpdate_zmalloc_stat_add (__n) do {TenPthread_mutex_lock (&Used_memory_mutex);  OneUsed_memory + =(__n); APthread_mutex_unlock (&Used_memory_mutex);  -} while(0) -  the     #defineUpdate_zmalloc_stat_sub (__n) do { -Pthread_mutex_lock (&Used_memory_mutex);  -Used_memory-=(__n); -Pthread_mutex_unlock (&Used_memory_mutex);  +} while(0) -  + #endif A //Update Memory use variable function to determine if multi-thread security is required in the function at #defineUpdate_zmalloc_stat_alloc (__n) do { -size_t _n =(__n); -     if(_n& (sizeof(Long)-1)) _n + =sizeof(Long)-(_n& (sizeof(Long)-1));  -     if(zmalloc_thread_safe) { - Update_zmalloc_stat_add (_n); -}Else {  inUsed_memory + =_n; -     }  to} while(0) + //Update Memory use variable function to determine if multi-thread security is required in the function - #defineUpdate_zmalloc_stat_free (__n) do the { \  *size_t _n =(__n); \ $ 　　 if(_n& (sizeof(Long)-1)) _n + =sizeof(Long)-(_n& (sizeof(Long)-1)); \ Panax Notoginseng  　　 if(Zmalloc_thread_safe) - {\ the update_zmalloc_stat_sub (_n); \ + } A  　　 Else  the {\ +Used_memory-=_n; \ - } \ $} while(0)

The next article then writes the specific function implementation.

Redis Memory Management (i)