Redis Combat (a) Introduction to Redis

about Redis

Redis is an open source high-performance key-value pair database,
The original author was Salvatore Sanfilippo of Italy,
His GitHub is Antirez, and Redis's source code is also hosted on Git:
Https://github.com/antirez/redis.
Currently, VMware is funding the development and maintenance of REDIS projects.

Features of Redis

Key value pairs such as dict["key"]= "value", "key" is the key name, "value" is the key value,
The key value for Redis can be not only a string, but also a variety of data types, including string types, hash types, list types, collection types, and ordered collection types.

All redis data is stored in memory, but it also provides support for persistence to asynchronously write data in memory to the hard disk.
Redis can set the lifetime (time to Live,ttl) for each key, and the time-to-live key is automatically deleted, so redis can be used as a caching system.

Redis can also limit the maximum amount of memory occupied by data, and can automatically retire unwanted keys after the data has reached space limits.

The Redis list type key supports blocking reads and can implement a high-performance priority queue.

Redis supports the "publish/subscribe" message pattern, which can be applied in a number of message scenarios.

the difference between Redis and memcached (turn to talk about memcached and Redis)

For memory-based database systems such as Redis and memcached, the efficiency of memory management is the key factor affecting the performance of the system. The Malloc/free function in traditional C language is the most commonly used method of allocating and releasing memory, but there is a big flaw in this approach: first, the mismatch of malloc and free for developers is prone to memory leaks, and second, frequent calls can cause large amounts of memory fragmentation to be recycled , reducing memory utilization; Finally, as a system call, its overhead is much larger than the general function call. Therefore, in order to improve the management efficiency of memory, the efficient memory management scheme will not use the Malloc/free call directly. Both Redis and memcached use their own design of memory management mechanisms, but there are significant differences in implementation methods.

Memcached's memory management mechanism

Memcached uses the slab allocation mechanism to manage memory by default, and the main idea is to divide the allocated memory into blocks of specific lengths to store the corresponding length of key-value data records in a predetermined size to completely resolve the memory fragmentation problem. The Slab allocation mechanism is designed to store external data only, which means that all key-value data is stored in the Slab allocation system, while other memcached memory requests are applied by ordinary malloc/free. Because the number and frequency of these requests determines that they do not affect the performance of the entire system

The principle of Slab allocation is quite simple. , it first requests a large chunk of memory from the operating system and divides it into blocks of various sizes chunk, and divides the same size blocks into group slab Class. Where chunk is the smallest unit used to store key-value data. The size of each slab class can be controlled by making growth factor when the memcached is started. Assume that the value of growth factor in Figure 1 is 1.25, so if the size of the first set of chunk is 88 bytes, the second group chunk is 112 bytes in size, and so on.

Memcached Memory Management Architecture

When memcached receives the data sent by the client, it will first select the most appropriate slab Class based on the size of the data received, and then memcached save the slab by querying the A list of idle chunk in class allows you to find a chunk that can be used to store data. When a database expires or is discarded, the chunk that the record occupies can be reclaimed and re-added to the free list. From the above process, we can see that memcached's memory management system is efficient and does not cause memory fragmentation, but its biggest drawback is that it can lead to wasted space. Because each chunk allocates a specific length of memory space, variable-length data cannot take full advantage of these spaces. As shown, the remaining 28 bytes are wasted by caching 100 bytes of data into a 128-byte chunk.

Memcached Waste of storage space

Redis's memory management mechanism

Redis memory management mainly through the source code in the zmalloc.h and zmalloc.c two files to achieve. To facilitate memory management, Redis allocates a chunk of memory to the head of the memory block. As shown in 5, REAL_PTR is the pointer returned by the REDIS call to malloc. Redis takes the size of a block of memory into the head, size occupies a known amount of memory, is the length of the size_t type, and then returns RET_PTR. When the memory needs to be freed, the ret_ptr is passed to the memory management program. With Ret_ptr, the program can easily calculate the value of real_ptr and then pass real_ptr to free to release memory.

Redis block Assignment

Redis records all memory allocations by defining an array whose length is zmalloc_max_alloc_stat. Each element of the array represents the number of memory blocks allocated by the current program, and the size of the memory block is the subscript for that element. In the source code, this array is zmalloc_allocations. ZMALLOC_ALLOCATIONS[16] represents the number of memory blocks that have been allocated for a length of 16bytes. ZMALLOC.C has a static variable used_memory used to record the total amount of memory currently allocated. So, on the whole, Redis uses packaging mallc/free, which is much simpler than Memcached's memory management approach.

Redis and memcached Overall comparison

The Salvatore Sanfilippo, the author of Redis, has previously compared these two memory-based data storage systems and summarized the following:

1) Performance comparison: Since Redis uses only single cores, and memcached can use multicore, on average, Redis has a higher performance than memcached for storing small data on each core. In more than 100k of data, Memcached performance is higher than Redis, although Redis has recently been optimized for the performance of storing big data, but it is slightly inferior to Memcached.

2) Memory usage efficiency comparison: With simple key-value storage, memcached memory utilization is higher, and if Redis uses hash structure to do key-value storage, its memory utilization will be higher than memcached because of its combined compression.

3) Redis supports server-side data manipulation: Redis has more data structures and supports richer data operations than memcached, usually in memcached, you need to get the data to the client for similar modifications and set it back. This greatly increases the number of network IO and the volume of data. In Redis, these complex operations are often as efficient as the general get/set. Therefore, Redis is a good choice if you need caching to support more complex structures and operations.

Redis Combat (a) Introduction to Redis