Comparison of Different memory distributor fragmentation rates in Redis

We knowRedisThere is no self-implemented memory pool, and nothing else is added to the standard system memory distributor. Therefore, the performance and fragment rate of the system memory distributor may affect Redis performance.

In Redis's zmalloc. c source code, we can see the following code:

 48 /* Explicitly override malloc/free etc when using tcmalloc. */ 49 #if defined(USE_TCMALLOC) 50 #define malloc(size) tc_malloc(size) 51 #define calloc(count,size) tc_calloc(count,size) 52 #define realloc(ptr,size) tc_realloc(ptr,size) 53 #define free(ptr) tc_free(ptr) 54 #elif defined(USE_JEMALLOC) 55 #define malloc(size) je_malloc(size) 56 #define calloc(count,size) je_calloc(count,size) 57 #define realloc(ptr,size) je_realloc(ptr,size) 58 #define free(ptr) je_free(ptr) 59 #endif

From the code above, we can see that Redis will first determine whether to useTcmallocIf yes, the function implementation in the standard libc will be replaced with the function implementation corresponding to tcmalloc. Second, we will judgeJemallocWhether or not to use the memory management function in the standard libc.

In the latest version 2.4.4, jemalloc has been included in the source code package as part of the source code package, so it can be directly used. If you want to use tcmalloc, you need to install it yourself.

The following describes how to install the tcmalloc package. tcmalloc is part of google-proftools. Therefore, we need to install google-proftools. If you are installing on a 64-bit machine, you must first install the libunwind library on which it depends.

wget http://download.savannah.gnu.org/releases/libunwind/libunwind-0.99-alpha.tar.gztar zxvf libunwind-0.99-alpha.tar.gzcd libunwind-0.99-alpha/CFLAGS=-fPIC ./configuremake CFLAGS=-fPICmake CFLAGS=-fPIC install

Then install google-preftools:

Wget http://google-perftools.googlecode.com/files/google-perftools-1.8.1.tar.gztar zxvf google-perftools-1.8.1.tar.gzcd google-perftools-1.8.1 /. /configure -- disable-cpu-profiler -- disable-heap-checker -- disable-debugalloc -- enable-minimalmake & make install sudo echo "/usr/local/ lib ">/etc/ld. so. conf. d/usr_local_lib.conf # if this file is not available, create a sudo/sbin/ldconfig

Then install Redis and specify the relevant parameters during make to enable tcmalloc.

$ curl -O http://redis.googlecode.com/files/redis-2.4.4.tar.gz$ tar xzvf redis-2.4.4.tar.gz$ cd redis-2.4.4$ make USE_TCMALLOC=yes FORCE_LIBC_MALLOC=yes$ sudo make install

Start Redis and run the info command to view the memory distributor used.

Next we will return to the topic of this Article. For tcmalloc, jemalloc, and libc, we will discuss the three memory splitters. What is its performance and fragmentation rate? The following is a simple test result. The Redis-benchmark provided by redis is used to write the same amount of data for testing. The data is extracted from the Redis info information when different splitters are used. We can see that when tcmalloc is used, the fragmentation rate is the lowest, which is 1.01, jemalloc is 1.02, and libc's distributor fragmentation rate is 1.31, as shown below:

used_memory:708391440used_menory_human:675.57Mused_memory_rss:715169792used_memory_peak:708814040used_memory_peak_human:675.98Mmem_fragmentation_ratio:1.01mem_allocator:tcmalloc-1.7

used_memory:708381168used_menory_human:675.56Mused_memory_rss:723587072used_memory_peak:708803768used_memory_peak_human:675.97Mmem_fragmentation_ratio:1.02mem_allocator:jemalloc-2.2.1

used_memory:869000400used_menory_human:828.74Mused_memory_rss:1136689152used_memory_peak:868992208used_memory_peak_human:828.74Mmem_fragmentation_ratio:1.31mem_allocator:libc

The above test data is small data, that is to say, a single piece of data is not big. Next we try to set the-d parameter of benchmark and adjust the value to 1 K. The test results have changed:

used_memory:830573680used_memory_human:792.10Mused_memory_rss:849068032used_memory_peak:831436048used_memory_peak_human:792.92Mmem_fragmentation_ratio:1.02mem_allocator:tcmalloc-1.7

used_memory:915911024used_memory_human:873.48Mused_memory_rss:927047680used_memory_peak:916773392used_memory_peak_human:874.30Mmem_fragmentation_ratio:1.01mem_allocator:jemalloc-2.2.1

used_memory:771963304used_memory_human:736.20Mused_memory_rss:800583680used_memory_peak:772784056used_memory_peak_human:736.98Mmem_fragmentation_ratio:1.04mem_allocator:libc

We can see that the split rate of several sub-splitters is still relatively large in allocating large blocks of memory and small blocks of memory. When we use Redis, try to use your own real data for testing to select the most suitable data distributor.