Comparison of Different memory distributor fragmentation rates in redis

We know that redis does not implement its own memory pool and does not add its own stuff to the standard system memory distributor. Therefore, the performance and fragment rate of the system memory distributor may affect redis performance.
In redis's zmalloc. C source code, we can see the following code:
48/* explicitly override malloc/free etc when using tcmalloc .*/
49 # If defined (use_tcmalloc)
50 # define malloc (size) tc_malloc (size)
51 # define calloc (count, size) tc_calloc (count, size)
52 # define realloc (PTR, size) tc_realloc (PTR, size)
53 # define free (PTR) tc_free (PTR)
54 # Elif defined (use_jemalloc)
55 # define malloc (size) je_malloc (size)
56 # define calloc (count, size) je_calloc (count, size)
57 # define realloc (PTR, size) je_realloc (PTR, size)
58 # define free (PTR) je_free (PTR)
59 # endif
From the code above, we can see that redis will first determine whether to use tcmalloc during compilation. If yes, it will replace the function implementation in the standard libc with the function implementation corresponding to tcmalloc. The second step is to determine whether jemalloc is enabled. If it is not used, the memory management function in the standard libc will be used.
In the latest version 2.4.4, jemalloc has been included in the source code package as part of the source code package, so it can be directly used. If you want to use tcmalloc, you need to install it yourself.
The following describes how to install the tcmalloc package. tcmalloc is part of Google-proftools. Therefore, we need to install Google-proftools. If you are installing on a 64-bit machine, you must first install the libunwind library on which it depends.
Wget http://download.savannah.gnu.org/releases/libunwind/libunwind-0.99-alpha.tar.gz
Tar zxvf libunwind-0.99-alpha.tar.gz
CD libunwind-0.99-alpha/
Cflags =-FPIC./configure
Make cflags =-FPIC
Make cflags =-FPIC install
Then install Google-preftools:
Wget http://google-perftools.googlecode.com/files/google-perftools-1.8.1.tar.gz
Tar zxvf google-perftools-1.8.1.tar.gz
CD google-perftools-1.8.1/
./Configure -- disable-CPU-Profiler -- disable-heap-checker -- disable-debugalloc -- enable-Minimal
Make & make install

Sudo echo "/usr/local/lib">/etc/lD. So. conf. d/usr_local_lib.conf # if this file is not available, create one by yourself.
Sudo/sbin/ldconfig
Then install redis and specify the relevant parameters during make to enable tcmalloc.
$ Curl-O http://redis.googlecode.com/files/redis-2.4.4.tar.gz
$ Tar xzvf redis-2.4.4.tar.gz
$ Redis-2.4.4 CD
$ Make use_tcmalloc = Yes force_libc_malloc = Yes
$ Sudo make install
Start redis and run the info command to view the memory distributor used.
Next we will return to the topic of this Article. For tcmalloc, jemalloc, and libc, we will discuss the three memory splitters. What is its performance and fragmentation rate? The following is a simple test result. The redis-benchmark provided by redis is used to write the same amount of data for testing. The data is extracted from the redis info information when different splitters are used. We can see that when tcmalloc is used, the fragmentation rate is the lowest, which is 1.01, jemalloc is 1.02, and libc's distributor fragmentation rate is 1.31, as shown below:
Used_memory: 708391440
Used_menory_human: 675.57 m
Used_memory_ RSS: 715169792
Used_memory_peak: 708814040
Used_memory_peak_human: 675.98 m
Mem_fragmentation_ratio: 1.01
Mem_allocator: tcmalloc-1.7
Used_memory: 708381168
Used_menory_human: 675.56 m
Used_memory_ RSS: 723587072
Used_memory_peak: 708803768
Used_memory_peak_human: 675.97 m
Mem_fragmentation_ratio: 1.02
Mem_allocator: jemalloc-2.2.1
Used_memory: 869000400
Used_menory_human: 828.74 m
Used_memory_ RSS: 1136689152
Used_memory_peak: 868992208
Used_memory_peak_human: 828.74 m
Mem_fragmentation_ratio: 1.31
Mem_allocator: libc
The above test data is small data, that is to say, a single piece of data is not big. Next we try to set the-D parameter of benchmark and adjust the value to 1 K. The test results have changed:
Used_memory: 830573680
Used_memory_human: 792.10 m
Used_memory_ RSS: 849068032
Used_memory_peak: 831436048
Used_memory_peak_human: 792.92 m
Mem_fragmentation_ratio: 1.02
Mem_allocator: tcmalloc-1.7
Used_memory: 915911024
Used_memory_human: 873.48 m
Used_memory_ RSS: 927047680
Used_memory_peak: 916773392
Used_memory_peak_human: 874.30 m
Mem_fragmentation_ratio: 1.01
Mem_allocator: jemalloc-2.2.1
Used_memory: 771963304
Used_memory_human: 736.20 m
Used_memory_ RSS: 800583680
Used_memory_peak: 772784056
Used_memory_peak_human: 736.98 m
Mem_fragmentation_ratio: 1.04
Mem_allocator: libc
We can see that the split rate of several sub-splitters is still relatively large in allocating large blocks of memory and small blocks of memory. When we use redis, try to use your own real data for testing to select the most suitable data distributor.