Read the redis source code.

Source: Internet
Author: User
I have heard about redis a long time ago, but I don't know much about it until I recently saw this article about several things V2EX learned in the past year and a half. I was surprised to mention that redis can be used independently instead of Mysql. I was prepared to make up my mind to study redis. In addition, I have recently seen some twemcache? (Twemcache is twitter's me.

I have heard about redis a long time ago, but I don't know much about it until I recently saw this article about several things V2EX learned in the past year and a half. I was surprised to mention that redis can be used independently instead of Mysql. I was prepared to make up my mind to study redis. In addition, I have recently seen some twemcache? (Twemcache is twitter's me.

I have heard about redis a long time ago, but I don't know much about it until I recently saw this article about several things V2EX learned in the past year and a half. I was surprised to mention that redis can be used independently instead of Mysql. I was prepared to make up my mind to study redis. In addition, I have recently seen some twemcache? (Twemcache is the source code of twitter's memcache rewrite version, hereinafter referred to as memcache), so I will share my experiences with them here.

Redis is a pure memory, key-value database, a kind of NO-SQL. Its value can be string, list, set, zset (sorted set-Ordered set), or hashs (hash type ). Redis? It also supports persistence and master-slave replication.

In contrast, memcache is much more pure. Memcache is a memory-only key-value cache system. The value type is only a buffer segment and does not support persistence or master-slave replication.

Redis and memcache are both implemented using ansi c, and the code quality and readability are high, which is of great learning value.

Redis Data Structure

When I read the redis source code, I mainly read some basic data structures used inside redis. These data structures are very common.

Sds:Simple Dynamic String, a Simple Dynamic String. Is the string representation used at the underlying layer of Redis. It is used in almost all Redis modules. It is a heap-memory-based string with a length that can be automatically expanded and compatible with strings in C.

Adlist: Two-way linked list, linked list, which has been implemented by countless programmers countless times.

Dict: Dictionary. What is interesting is the process of resizing. Each dictionary contains two hash tables. Generally, the first hash table is used. When the capacity of the hash table is too large to be expanded, the second hash table is created. At this time, you cannot import all the data in the first hash table to the second hash table at once, because this may cause blocking. Instead, a node is migrated from the first hash table to the second hash table every time you perform the search or add operation. This way, you can migrate the table from the first hash table until the expansion is completed.

Skiplist:Skip table, this data structure is also very interesting, it is an ordered linked list, coupled with the Randomization data structure, based on the parallel linked list, the efficiency is comparable to that of the Binary Search Tree (O (log n) Average time is required for most operations ). How does it work? Can I refer to this article? Http://dsqiu.iteye.com/blog/1705530

Ziplist :?A compressed linked list that uses a compressed data structure minus pointer consumption can significantly reduce memory usage. When the data volume of list, map, and zset is smaller than the specified value, ziplist is used to reduce memory consumption.

In addition, data structures such as intset and zipmap are not described in detail.

? Master-slave replication of redis

The redis master and slaver are very elegant. I have encountered problems with master-slave replication and synchronization before, but I have not found a better solution. Synchronization of the master-slave structure requires a full synchronization and subsequent incremental synchronization. The key to synchronization is the sequence. For example, if you first del a key and then add the key, if you synchronize it to add first and then del, the data is inconsistent. Redis uses its persistence capability to generate an RDB memory dump file after receiving the synchronization request, maintains a linked list for all write operations starting at this time. After the RDB file is generated, it is sent to slaver and then all the write operation commands generated when the RDB file is generated to slaver for execution. This completes a full update. Next, the master will synchronize the write operation to slaver each time.

Redis and memcache

Memory Allocation:The main selling point of memcache is its memory allocation. Memcache implements its own memory allocation because glibc's built-in malloc/free has poor performance and is prone to memory fragmentation. Memcache manages the memory through the memory allocation mechanism called slab. Redis uses a third-party memory distributor instead of allocating its own memory. Redis source code comes with jemalloc, or google's tcmalloc can be used, both of which have high performance and almost no memory fragmentation issues.

About event handling:Memcache uses the open-source libevent ?? Process events, including network I/O and timer. Redis implements its own event processing library and encapsulates functions such as epoll based on its independent principle.

Configuration:Use getopt for memcache? Command Line parsing only supports configuration in the form of command line parameters, and does not support configuration files. Redis also supports configuration files and command lines. When redis parses command lines, it does not use getopt or other standardized command line parsing tools. Instead, it uses strcmp for parsing. At first, it thought this method was quite ugly, later I found myself misunderstanding the hard work of the author of redis. Redis removes the-Prefix parameters except basic parameters such as-help-version and adds them to a string. If a configuration file exists, the file content is also read into the string, and then configured and parsed by the unified function. In this way, the configuration file and command line parameters are unified. as long as some of the configuration files can be configured through the command line, I personally think it is very clever.

About threads: Memcache is multi-threaded. A main thread processes connections, Schedules other threads to process requests, and fully utilizes the capabilities of multi-core CPUs. Redis is a single process. Strictly speaking, redis is a mix of threads and processes. It uses threads for fsync, and also uses multi-processes to implement RDB persistence using the copy-on-write feature of fork. However, redis processes requests in a single thread. Why not use multithreading? One reason is that a single thread is fast enough to process memory data. In addition, a single thread can provide certain transaction features. To make full use of resources, you can create multiple redis instances on a single machine.

Redis Performance

Is redis equipped with a benchmark testing tool? Redis-benchmark is used to test the performance of Common commands. According to the test, the average get set command can be used up to times per second for blocking calls by a single CLIENT. It can be said that it is enough in most cases.

Refer:

Official redis documentation

Redis Design and Implementation

Original article address: Read the redis source code. Thank you for sharing it with the original author.

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.