To build a redis cluster, you must first consider the following issues;
What is the purpose of building a redis cluster? Or why should we build a redis cluster?
The purpose of setting up a redis cluster is to build a cluster. All clusters are mainly used to solve a problem and scale horizontally. Before the emergence of the concept of a cluster, the hardware resources we use were all vertically scalable, but the vertical expansion will soon reach a limit, the CPU processing speed and memory size of a single machine, hard Disk size cannot always meet requirements, and the cost of machine vertical expansion is quite high. The emergence of a cluster is to allow multiple machines to work as one machine, achieving horizontal resource scaling.
Redis is a memory-type database. When the data to be stored reaches a certain level, the memory of a single machine cannot meet our needs. Building a cluster is a good solution.
So how can we make redis on multiple machines work like redis on one machine?
You need to solve the following three problems: 1. expose an Access Node externally. 2. request sharding 3. reasonable parts (even parts, the same request should be allocated to the same redis node)
First, we need to expose an Access Node to the outside, and there may be multiple redis instances working later. We can easily think of proxy. for clients, we only need to know proxy, use a proxy to interact with multiple redis instances in the background.
The second problem can be solved by using the hash algorithm. A key in the request is taken as the hash value and the remainder of the number of redis instances is used to allocate different redis instances; however, this does not satisfy the fact that the same request in the third point must be allocated to the same redis. In this case, consistent hash can perfectly solve this problem. Consistent hash can distribute redis nodes by another two through the hash algorithm.
In the ring of 32 nodes, the requested key is mapped to the ring using the same hash algorithm, and then the nearest redis node is searched clockwise for the key value node, this ensures consistency; a detailed introduction can refer to: http://blog.csdn.net/sparkliang/article/details/5279393.
If I want to implement a redis cluster, I will use zookeeper and a redis daemon plus a consistent hash algorithm. However, twicer's great gods have implemented and open-source a redis cluster solution, so I will not duplicate the wheel; let's take a look at the features of twemproxy (Nutcracker) implemented by one of the world's largest redis cluster users.
In addition to acting as a proxy for redis, twemproxy also supports the ASCII protocol of memerycached. Here I will mainly learn about the twemproxy solution on the redis cluster. In addition to solving the preceding three problems perfectly (Consistent hash algorithm is also adopted), twemproxy also has an important feature. It supports node ejection. If redis is used as a cache, if you do not pay much attention to data consistency, you can enable node ejection to remove a redis instance from the cluster list when it fails to reach high availability. If the redis cluster is used for data storage or data consistency is very important, you can disable node jection. However, you need to use other methods to achieve high availability, such as redis sentiel. Twemproxy detailed introduction can look at this article: http://antirez.com/news/44; twemproxy project GitHub address: https://github.com/twitter/twemproxy. So much nonsense, let's see how to install twemproxy and set up a redis cluster.
First install twemproxy as instructed by GitHub:To build nutcracker from source with debug logs enabled and assertions disabled:
$ Git clone [email protected]: Twitter/twemproxy. git $ CD twemproxy $ autoreconf-FVI $. /configure -- enable-DEBUG = Log $ make input the above commands in sequence, and then enter: $ src/nutcracker-HIf the help information is displayed, the installation is successful. Next, edit the configuration information, find a directory you like, and create a configuration file with the suffix yml, such as twemproxy. test. yml:
alpha: listen: 127.0.0.1:55555 hash: fnv1a_64 distribution: ketama auto_eject_hosts: true redis: true server_retry_timeout: 30000 server_failure_limit: 1 servers: - 192.168.1.10:6379:1 - 192.168.1.7:6379:1
Note indentation. Otherwise, twenproxy cannot be started. Among them, listen indicates the proxy IP address and port number, which are exposed to the client; hash: indicates the hash method used. twemproxy provides multiple methods. For details, refer to GitHub; distribution indicates the allocation mode. There are three options: ketama, modula, random; auto_reject_hosts: As mentioned above, nodes that fail to be automatically removed; redis: Indicates redis clusters are used, the remaining configuration is very simple ...... after such simple configuration, execute. /src/nutcracker-C. /CONF/nutcracker. test. yml starts twemproxy by using its own installation path and configuration file path. The client can connect to the proxy through redis-cli, the usage is exactly the same as that of redis (but not all commands are supported ). In this way, the redis cluster is ready, isn't it simple enough? Don't be too happy, though. twemproxy also has its shortcomings. 1. Transactions and batch operations are not supported; 2. Performance loss compared to direct access to redis;
Although twemproxy has the above shortcomings, the single-phase advantage is not worth mentioning! It does not apply to all situations. There are still many ways to build a redis cluster. Its official cluster has been in Beta state. I believe that after learning about twemproxy, you will find it the best choice now!
