TCMalloc optimizes memory management for MySQL, Nginx, and Redis

Tcmalloc Overview: TCMalloc (Thread-CachingMalloc) implements the same functions as the standard glibc library's malloc, but TCMalloc is much more efficient and faster than standard malloc. TCMalloc is one of the google-perftools tools (four gperftools are: TCMalloc, heap-checker, heap-profiler, and

Tcmalloc Overview: TCMalloc (Thread-Caching Malloc) implements the same functions as the standard glibc library's malloc, but TCMalloc is much more efficient and faster than standard malloc. TCMalloc is one of the google-perftools tools (four gperftools are: TCMalloc, heap-checker, heap-profiler, and

Tcmalloc Overview:

TCMalloc (Thread-Caching Malloc) implements the same functions as the standard glibc library's malloc, but TCMalloc is much more efficient and faster than standard malloc. TCMalloc is one of the google-perftools tools (the four gperftools are TCMalloc, heap-checker, heap-profiler, and cpu-profiler). This tool is open-source, released as source code. If you think it is difficult to maintain a memory distributor, you can consider connecting the TCMalloc static library to your program. The method of calling malloc in glibc is the same. All you need to do is connect the dynamic library or static library of TCMalloc to your program, and you can get an efficient, fast, and secure memory distributor.

Compared with the standard glibc library's malloc, TCMalloc features high memory allocation efficiency and speed, which can control memory usage and improve server performance in high concurrency, reduce the load. For the implementation principle and test report of TCMalloc, see TCMalloc: Malloc of thread cache.

Tcmalloc divides memory requests into two types: Large Object Requests and small object requests. large objects are> = 32 K objects. |
Tcmalloc allocates a local buffer for each thread.
Small object requests can be obtained directly from the local buffer of the thread. If there is no idle memory in the local buffer of the thread, a series of small objects are obtained from the central heap at a time.
For small memory, tcmalloc is allocated by an integer multiple of 8, and for large memory, it is allocated by an integer multiple of 4 K.
There are two advantages to doing so. One is faster allocation. Second, the short-term benefits are relatively large. The allocated small memory wastes up to 7 bytes, while the large memory is 4 kb.
When the total size of all objects in the cache in a thread cache exceeds 2 MB, garbage collection is performed on it. The garbage collection threshold is automatically reduced based on the increase in the number of threads, so that the memory will not be excessively wasted because the program has a large number of threads.

Memory leakage detection
The Tcmalloc program cannot detect memory leakage with valgrind. You can use the heap checker provided by google-perftools.
Usage:
Export HEAPCHECK = TYPE
The TYPE can be minimal, normal, strict, or draconian.

As an option, tcmalloc has been added to the one-click installation package of the latest lnmp source code.

Install the libunwind Library:
If the system is 64-bit, you need to install the libunwind Library first, and the 32-bit system does not need to be installed.

The libunwind Library provides the basic stack trigger function for 64-bit CPU and operating system-based programs, the APIs used to output stack tracing are APIs that are used to program stack decoding and APIs that support the C ++ exception handling mechanism.

wget http://download.savannah.gnu.org/releases/libunwind/libunwind-1.1.tar.gztar zxf libunwind-1.1.tar.gzcd libunwind-1.1CFLAGS=-fPIC ./configuremake CFLAGS=-fPICmake CFLAGS=-fPIC installcd ../

Install gperftools:

wget http://gperftools.googlecode.com/files/gperftools-2.1.tar.gztar xzf gperftools-2.1.tar.gzcd gperftools-2.1

You can add parameters to compile only tcmalloc (. /configure-enable-minimal,-disable-cpu-profiler,-disable-heap-profiler,-disable-heap-checker,-disable-debugalloc ), if the 64-bit operating system does not install libunwind, no error is reported. Note that the generated library file is libtcmalloc_minimal. *
64-bit operating system, as shown below

./configure 

You do not need to install libunwind for a 32-bit system, but you must add the-enable-frame-pointers parameter as follows:

./configure --enable-frame-pointers 

make && make install

After compilation and installation, enter the following command:

Echo '/usr/local/lib'>/etc/ld. so. conf. d/local. confldconfig # Must be executed

Use TCMalloc to optimize MySQL
MySQL 5.1 Static compilation method,./configure precompilation assumes the following parameters

--with-mysqld-ldflags=-ltcmalloc

MySQL 5.5 static compilation method. The following parameters are added during cmake pre-compilation.

-DCMAKE_EXE_LINKER_FLAGS="-ltcmalloc"-DWITH_SAFEMALLOC=OFF

Dynamic Loading

sed -i 's@executing mysqld_safe@executing mysqld_safe\nexport LD_PRELOAD=/usr/local/lib/libtcmalloc.so@'/usr/local/mysql/bin/mysqld_safeservice mysqld restart

Verify whether loading tcmalloc takes effect in MySQL as follows:

lsof -n | grep tcmalloc

Use TCMalloc to optimize Nginx
To enable nginx to support google-perftools, you need to add the "-with-google_perftools_module" option during installation to recompile nginx. The installation is as follows:

cd lnmp/src/nginx-1.4.2make clean./configure --prefix=/usr/local/nginx --user=www --group=www \--with-http_stub_status_module --with-http_ssl_module --with-http_flv_module \--with-http_gzip_static_module --with-google_perftools_modulemake && make install

To add a thread directory:

Mkdir/tmp/tcmallocchown-R www. www/tmp/tcmallocvi/usr/local/nginx/conf/nginx. conf # Add google_perftools_profiles/tmp/tcmalloc to the next line of pid;

Verify whether tcmalloc is effective in Nginx

lsof -n | grep tcmalloc

Each thread (value of work_processes) has a row of records. The numeric value after each thread file is the pid value of the started nginx.

Optimize redis with TCMalloc
Note: The redis-2.4 above comes with jemalloc, you do not need to add any parameters through zmalloc. in the c source code, we can see that Redis will first determine whether to use tcmalloc during compilation. If yes, it will replace the function implementation in the standard libc with the function implementation corresponding to tcmalloc. The second step is to determine whether jemalloc is enabled. If it is not used, the memory management function in the standard libc will be used. Therefore, use tcmalloc for optimization with caution. The split rate of the two sub-shards is not much different. We recommend that you use jemalloc

cd lnmp/src/redis-2.6.16make USE_TCMALLOC=yes FORCE_LIBC_MALLOC=yes/bin/cp src/{redis-benchmark,redis-check-aof,redis-check-dump,redis-cli,redis-sentinel,redis-server}/usr/local/redis/bin

Other references:

Http://blog.csdn.net/chosen0ne/article/details/9338591