Redis Design Idea

Unlike the Nginx, the style of Redis code tends to be concise and practical. Concise notice, the following is no longer enumerate any source code, do not piece together any foreign information. Remove the last branch, the following straight into the Redis theme, as succinctly as possible to describe redis design ideas. Overall Model: Single process single-threaded event-driven mode. Redis in the main processing process, a single process is adopted to accept various client requests and return the results, and the whole process is implemented in event-driven manner. The Aeeventloop event is monitored by its IO multiplexing, which, during event handling, requires the processing function to perform an event-handling process as quickly as possible. Therefore, there are several ways to deal with this: 1. For not long time processing events, direct processing. such as the set command; 2. For events that require lengthy processing, step through them until they are complete. such as the rehash process, only one bucket is moved at a time during a single event process. 3. For blocking functions, such as reading VM data from disk, the asynchronous pattern is used. From disk load data, is a time-consuming operation, such as the processing function directly read data, will block the whole single-threaded process of other client requests. The complexities of multithreaded development are clear to us. Adopt single process single-threaded mode, can avoid a lot of complexity, another transactional operation, no additional lock, greatly reduce the difficulty of development. IO multiplexing Redis io multiplexing layer, can be seen as a tiny libevent, its implementation is only four files, ae.c,ae_select.c, ae_kqueue.c,ae_epoll.c. At a glance, AE is an event listener total Interface,select, Kqueque, and Epoll are three optional IO multiplexing methods. Small and sufficient, concise and efficient, fully embodies the Redis design concept. Redis default:select,c10k Max. Event sniffing is in the Single-threaded event-driven model, often encounter two kinds of tasks, timed processing tasks and event trigger tasks, so there will be the following problems: Event Trigger task In the absence of tasks, will be in a blocking state; Timed tasks require regular processing, so there are often the following two kinds of processing methods: 1.  If the timed task interval is T, the general set of IO multiplexing layer timeout is T/10, so that the timing task can be processed in a timely manner, when all cron tasks are invoked, the interval time to judge. 2.  calculates the interval between the current event and the start time of the most recent scheduled task, setting the IO reuse timeout to this event. Such timing tasks can also be dealt with in a timely manner. Note: The processing time of the above scheduled tasks is estimated time, not accurate time. The previous method of calculation for each round less, when easy to cause idling, the latter a large amount of calculation, but reduced the number of idling. Redis adopted theis the latter way. Some people here may ask: Why not use Sigaction, Setitimer and so on to set the exact clock, no longer do not have to do for cron event additional processing. The reasons are as follows: timing clock, often let the process into the kernel state, the kernel soft interrupt will often break the integrity of a function of the user state, it will destroy the relevant state transfer, and add soft interrupt program, will likely to bring inevitable complexity of the program. Take the experience of a project that has been a setback for example: a call to multiple calls to the project, the use of a single-threaded asynchronous event-driven model, each time the task has a free channel to "Hang Up" task. Prior to the use of prefabricated driving clock, clock signal for real-time signal (signal macro value of 32-64 between a number), the signal in the Sigqueue queue, so if the task blocking return, will be thrown in a large number of clock events, the idle channel repeatedly failed to hang the machine. After using Setitimer to set the system clock, soft interrupts interrupt the user state of a mutex_lock and Mutex_unlock, and when the lock is used, causing the whole process of the deadlock phenomenon sometimes. Data structure Intset, this set is different from the hash set data structure, the data structure is actually an ordered array, for: insert operation: Binary find the ordered array--> find the location--> return, no then insert the position, memmove subsequent elements. Find operation: Two-point lookup. The model is simpler and more space-saving relative to the set based on the hash table. The complexity of the time, in the author's presentation mentioned in the size of the 20-50k data, basic and hashset the same. Here it can be seen that the hash set traversing the linked list time, and Intset binary lookup time is basically the same. But on the other hand, the author does not mention that the insertion speed of intset is actually reduced because the amount of data needed to be Memmove is O (n), but more advantageous than the hash set is that the array set is ordered. Ziplist, this data structure is for small size data design array list, like a common index array, unlike the linked list, which is stored in an array, and with Value-header Save the link item and its own offset, length, encoding, etc. , which is used to traverse the list. On demand, encoding guarantees a variety of data that can be stored in the same list, such as Int16, Int32, and Str. Zset:hash dict + Skip list. The common ordered set is based on RB tree. And throughout the Redis, it's hard to see any complex data structure tree traces.。 Redis to hash dict to ensure the efficiency of hash lookup, the jump-table structure to ensure the order and scope to find the rapid satisfaction of demand, range lookup abandoned RB tree,b+-* tree, but also embodies its not to complicate the concise aesthetics. ...... There is no dedicated memory pool structure throughout the redis. All memory allocations, unlike Stl,niginx, memory allocations have their memory pool mechanism allocated, redis not. Redis is all size+malloc zmalloc distribution, its common small object, in the intset,ziplist and other data structures to apply in an array, and constantly realloc the way resize, indirectly embodies the idea of the memory pool, But did not implement a memory pool management module, the author of simple and violent aesthetics, reflected incisively and vividly.