Discussion on Redis using different memory allocator Tcmalloc and Jemalloc_redis

We know that Redis does not implement the memory pool on its own, and does not add its own things to the standard system memory allocator. Therefore, the performance and fragmentation rate of the system memory allocator will have some performance effects on the Redis.

In the Redis zmalloc.c source, we can see the following code:

/* Double expansion needed for stringification of macro values. */ 
#define __xstr(s) __str(s) 
#define __str(s) #s 
#if defined(USE_TCMALLOC) 
#define ZMALLOC_LIB ("tcmalloc-" __xstr(TC_VERSION_MAJOR) "." __xstr(TC_VERSION_MINOR)) 
#include <google/tcmalloc.h> 
#if (TC_VERSION_MAJOR == 1 && TC_VERSION_MINOR >= 6) || (TC_VERSION_MAJOR > 1) 
#define HAVE_MALLOC_SIZE 1 
#define zmalloc_size(p) tc_malloc_size(p) 
#else 
#error "Newer version of tcmalloc required" 
#endif 
#elif defined(USE_JEMALLOC) 
#define ZMALLOC_LIB ("jemalloc-" __xstr(JEMALLOC_VERSION_MAJOR) "." __xstr(JEMALLOC_VERSION_MINOR) "." __xstr(JEMALLOC_VERSION_BUGFIX)) 
#include <jemalloc/jemalloc.h> 
#if (JEMALLOC_VERSION_MAJOR == 2 && JEMALLOC_VERSION_MINOR >= 1) || (JEMALLOC_VERSION_MAJOR > 2) 
#define HAVE_MALLOC_SIZE 1 
#define zmalloc_size(p) je_malloc_usable_size(p) 
#else 
#error "Newer version of jemalloc required" 
#endif 
#elif defined(__APPLE__) 
#include <malloc/malloc.h> 
#define HAVE_MALLOC_SIZE 1 
#define zmalloc_size(p) malloc_size(p) 
#endif 
#ifndef ZMALLOC_LIB 
#define ZMALLOC_LIB "libc" 
#endif 

As we can see from the above code, REDIS will first judge whether to use Tcmalloc when compiling, and if so, replace the function implementations in the standard libc with tcmalloc corresponding functions. The second will be to determine whether jemalloc makes it possible to use the standard LIBC memory management function in the end if it is not used.

In the latest version 2.4.4, Jemalloc has been included in the source package as part of the source package, so it can be used directly. And if you want to use Tcmalloc, you need to install it yourself.

Here's a quick look at how to install the Tcmalloc package, Tcmalloc is part of the google-proftools, so we actually need to install Google-proftools. If you are installing on a 64-bit machine, you need to first install the Libunwind library that it relies on.

wget http://download.savannah.gnu.org/releases/libunwind/libunwind-0.99-alpha.tar.gz Tar zxvf libunwind-0.99-alpha.tar.gz CD libunwind-0.99-alpha/cflags=-fpic./configure make Cflags=-fpic make CFLAGS=-fPIC Install

And then do the Google-preftools installation:

wget http://google-perftools.googlecode.com/files/google-perftools-1.8.1.tar.gz Tar zxvf google-perftools-1.8.1.tar.gz CD google-perftools-1.8.1/./configure--disable-cpu-profiler--disable-heap-profiler --disable-heap-checker--disable-debugalloc--enable-minimal make && make install sudo echo "/usr/local/lib" >/etc/ld.so.conf.d/usr_local_lib.conf #如果没有这个文件, Build a Sudo/sbin/ldconfig

Then install the Redis and specify the appropriate parameters at make to enable Tcmalloc

$ Curl-o http://redis.googlecode.com/files/redis-2.4.4.tar.gz $ tar xzvf redis-2.4.4.tar.gz $ cd redis-2.4.4 $ make use_t Cmalloc=yes Force_libc_malloc=yes $ sudo make install

After you start Redis, you can see the memory allocator using the info command.

The following is a return to the subject of this article, which corresponds to the three memory allocator for Tcmalloc,jemalloc and libc. What about its performance and fragmentation rate?

The following is a simple test result that uses the Redis Redis-benchmark to write the isometric data for testing, which is excerpted from the Redis info information when using a different allocator.

We can see that the tcmalloc rate is the lowest, the 1.01,jemalloc is 1.02, and the libc splitter fragment rate is 1.31, as follows:

used_memory:708391440 used_menory_human:675.57m used_memory_rss:715169792 used_memory_peak:708814040 used_memory_ peak_human:675.98m mem_fragmentation_ratio:1.01mem_allocator:tcmalloc-1.7

used_memory:708381168 used_menory_human:675.56m used_memory_rss:723587072 used_memory_peak:708803768 used_memory_ peak_human:675.97m mem_fragmentation_ratio:1.02mem_allocator:jemalloc-2.2.1

used_memory:869000400 used_menory_human:828.74m used_memory_rss:1136689152 used_memory_peak:868992208 used_memory_ peak_human:828.74m MEM_FRAGMENTATION_RATIO:1.31MEM_ALLOCATOR:LIBC

The test data above are small data, that is, a single piece of data is not large, we try to set the benchmark-d parameter, adjust the value to 1k size, the test results have changed some:

used_memory:830573680 used_memory_human:792.10m used_memory_rss:849068032 used_memory_peak:831436048 used_memory_ peak_human:792.92m mem_fragmentation_ratio:1.02mem_allocator:tcmalloc-1.7

used_memory:915911024 used_memory_human:873.48m used_memory_rss:927047680 used_memory_peak:916773392 used_memory_ peak_human:874.30m mem_fragmentation_ratio:1.01mem_allocator:jemalloc-2.2.1

used_memory:771963304 used_memory_human:736.20m used_memory_rss:800583680 used_memory_peak:772784056 used_memory_ peak_human:736.98m MEM_FRAGMENTATION_RATIO:1.04MEM_ALLOCATOR:LIBC

Can be seen, in allocating large chunks of memory and small block of memory, the fragmentation rate of several distributors is still relatively large, when we use Redis, or try to use their own real data to do the test, to choose the most suitable for their data distributor.

The above is small series for everyone to bring the Redis using different memory allocator Tcmalloc and jemalloc all content, I hope that we support cloud-Habitat Community ~