Redis Cluster Study Notes

Redis Cluster Study Notes

Redis officially introduced the cluster feature in version 3.0. Redis cluster is a distributed (distributed), fault tolerant (fault-tolerant) Redis memory K/V service, A cluster can use a subset (subset) of the functions that can be used by a common single-host Redis. For example, a Redis cluster does not support commands that process multiple keys, because this requires moving data between different nodes and thus failing to achieve performance like Redis, unexpected errors may occur under high loads.

Several important features of the Redis cluster:

(1) The sharding feature of the Redis cluster is that the key space is split into 16384 slots, and each node is responsible for some of the slots.

(2) Redis provides a certain degree of availability and can continue to process commands when a node is down or not reachable.

(3) there is no central or proxy node in the Redis cluster. One of the main design goals of the cluster is to achieve linear scalability ).

1. Redis data Sharding)

The key space of the Redis cluster is divided into 16384 (2 ^ 14) slots, and the maximum number of nodes in the cluster is 16384 (the recommended maximum number of nodes is 1000 ), similarly, each master node can process 1 to 16384 slots.

When all the 16384 slots are processed by the master node, the cluster enters the "stable" online state and can start to process data commands. When the cluster is not in a stable state, you can perform the reconfiguration operation so that each hash slot is processed by only one node.

Reconfiguration refers to moving one or more slots from one node to another. A master node can have any number of slave nodes. These slave nodes are used to replace the master node in the event of a network disconnection or node failure on the master node.

Cluster formula CRC16 (Key) & 16383 calculate which slot the key belongs:

HASH_SLOT = CRC16(key) mod 16384

The result length of CRC16 is 16 bits.

You may also like the following articles about Redis. For details, refer:

Install and test Redis in Ubuntu 14.04

Basic configuration of Redis master-slave Replication

Redis cluster details

Install Redis in Ubuntu 12.10 (graphic explanation) + Jedis to connect to Redis

Redis series-installation, deployment, and maintenance

Install Redis in CentOS 6.3

Learning notes on Redis installation and deployment

Redis. conf

2. Redis cluster node

Part of the content is taken from Appendix 2. The nodes in the Redis cluster must not only record key-value ing, but also record the cluster status, including key-to-correct node ing. It also has the function of automatically discovering other nodes, Identifying Abnormal nodes, and selecting a new master node from the node when necessary.

To execute the tasks listed above, each node in the cluster establishes a "cluster bus" with other nodes. The connection is a TCP connection and uses binary protocol for communication.

Nodes use the Gossip protocol to perform the following tasks:

A) propagate information about the cluster to discover new nodes.

B) send PING packets to other nodes to check whether the target node is operating normally.

C) Send cluster information when a specific event occurs.

In addition, the cluster connection is also used to publish or subscribe to information in the cluster.

The cluster node cannot forward PROXY command requests, so the client should return-MOVEDOr-ASKWhen a redirection error occurs, the command request is forwarded to another node.

The client can freely send command requests to any node in the cluster, and forward commands to the correct node based on the information provided by the redirection error when necessary, therefore, theoretically, the client does not need to save the cluster status information. However, if the client can save the ing information between keys and nodes, it can effectively reduce the number of possible redirects, which improves the efficiency of command execution.

Each node is identified by a unique ID in the cluster. This ID is a 160-bit random number in hexadecimal notation. It is generated by/dev/urandom when the node is started for the first time. The node will save its ID to the configuration file. As long as the configuration file is not deleted, the node will continue to use this ID. A node can change its IP address and port number without changing the node ID. The cluster can automatically identify IP/port number changes and broadcast this information to other nodes through the Gossip protocol.

The associated information of each node is shown below, and the node sends the information to other nodes:

A) the IP address and TCP port number used by the node.

B). node flag (flags ).

C) hash slots processed by nodes.

B) The last time the node sent a PING packet (packet) using a cluster connection.

E) the time when the node received the PONG packet in the last response.

F). The cluster marks the node as the deprecation time.

 

G). Number of slave nodes of the node.

If the node is a slave node, it records the node ID of the master node. If this is a master node, the value of the master node ID column is 0000000.

After learning about the basic features of the Redis Cluster, we first build the Redis Cluster.

3. Install Redis 3.0.x

The latest version is 3.0.1.

Wget http://download.redis.io/releases/redis-3.0.1.tar.gztar xvzf redis-3.0.1.tar.gzcd redis-3.0.1/make-j # apt-get install tclmake test # install redis on/usr/local/redis3 # cd src & make PREFIX =/usr /local/redis3 install ### create a symbolic link # ls/usr/local/redis3/bin/redis-*/usr/local/redis3/bin/redis-benchmark/usr/ local/redis3/bin/redis-check-dump/usr/local/redis3/bin/redis-sentinel/usr/local/redis3/bin/redis-check-aof/usr/ local/redis3/bin/redis-cli/usr/local/redis3/bin/redis-server # for I in 'CD/usr/local/redis3/bin; ls redis-* 'doln-s/usr/local/redis3/bin/$ I/usr/local/bin/$ idone; # mkdir-p/usr/local/redis3/conf # ln-sf/usr/local/redis3/conf/etc/redis3 ### check the version information redis-cli-vredis- cli 3.0.1

4. Redis Cluster configuration

The Redis instance running in cluster mode is different from the common Redis instance. In cluster mode, you need to enable the cluster feature through configuration, after the cluster mode is enabled, the Redis instance can use the unique commands and features of the cluster. the following is the configuration file of the cluster with the minimum options: Note that if you are using a standalone test, it is best to set cluster-config-file nodes. conf is set to the corresponding port nodes-xxx.conf, and pid is set to the current directory pidfile. /redis. pid

port 7001cluster-enabled yescluster-config-file nodes.confcluster-node-timeout 5000appendonly yes

The cluster-enabled option in the file is used to enable the cluster mode of the instance, while the cluster-conf-file option sets the path for saving the node configuration file. The default value is nodes. conf. The node configuration file does not need to be manually modified. It is automatically created by the Redis cluster at startup and updated as needed.

To make the cluster operate normally, at least three master nodes are required. In our environment, each master node has one slave node, so there are six nodes in total. The port is 7001-7006.

In/app/redis3, create six subdirectories with the port number as the name. Then we will run a Redis instance in each directory:

cd /app/redis3mkdir 7001 7002 7003 7004 7005 7006 cp /etc/redis3/conf/redis.conf  /app/redis3/7001/......cp /etc/redis3/conf/redis.conf  /app/redis3/7006/

Modify the port number in redis. conf to the corresponding port. Next, open the corresponding directory and start the redis instance. When starting, enter the corresponding directory and start.

cd /app/redis3/7001;nohup redis-server redis.conf &cd /app/redis3/7002;nohup redis-server redis.conf &......

The logs printed by the instance are displayed. Because the nodes. conf file does not exist, each node specifies a new ID for itself,

/app/redis3/7006# tail -f nohup.out27040:M 09 May 22:53:50.197 * No cluster configuration found, I'm 1984c27297c6ef50bbfcbd35c11b93cc40ba17e4/app/redis3/7006# cat nodes.conf d2b437ca8b9007dcdb63ac16210f6540860361e3 :0 myself,master - 0 0 0 connectedvars currentEpoch 0 lastVoteEpoch 0

Now we have six running Redis instances. We need to use these instances to create clusters. By using Redis-trib, The redis cluster command line tool, you can easily write node configuration files: redis-trib is located in the src folder of Redis source code, and is a Ruby program, this program sends a special command to the instance to create a new cluster, check the cluster, or reshare the cluster. The create command is used to create a cluster and replicas = 1 is specified, that is, each master instance has a slave instance. Redis-trib will print out an expected configuration for you. If you think there is no problem, you can enter yes, redis-trib will apply this configuration to the cluster, so that each node can start to communicate with each other, and finally obtain the following information:

~/redis-3.0.1/src# apt-get install ruby gem~/redis-3.0.1/src# gem sources -a http://ruby.taobao.org/~/redis-3.0.1/src# gem install redis~/redis-3.0.1/src# cp redis-trib.rb /usr/local/redis3/bin/~/redis-3.0.1/src# ln -sf /usr/local/redis3/bin/redis-trib.rb /usr/bin/redis-trib.rb~/redis-3.0.1/src# redis-trib.rb create --replicas 1  127.0.0.1:7001 127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005 127.0.0.1:7006M: 1984c27297c6ef50bbfcbd35c11b93cc40ba17e4 127.0.0.1:7001   slots:0-5460 (5461 slots) masterM: 481e256be4c724f5a2c64a761e52b4be61ca45b4 127.0.0.1:7002   slots:5461-10922 (5462 slots) masterM: b5b652fa02d9999861e66c843b01fd2700c02adf 127.0.0.1:7003   slots:10923-16383 (5461 slots) masterS: 821ec823dc0c2d4f65319e84fe74157fb1014155 127.0.0.1:7004   replicates 1984c27297c6ef50bbfcbd35c11b93cc40ba17e4S: b3b8541b9520d707180d56a2fb3cf3ee6895ed10 127.0.0.1:7005   replicates 481e256be4c724f5a2c64a761e52b4be61ca45b4S: d2b437ca8b9007dcdb63ac16210f6540860361e3 127.0.0.1:7006   replicates b5b652fa02d9999861e66c843b01fd2700c02adfCan I set the above configuration? (type 'yes' to accept):[OK] All nodes agree about slots configuration.>>> Check for open slots...>>> Check slots coverage...[OK] All 16384 slots covered.

Some of the above information can be obtained by sending the cluster nodes command to any node in the CLUSTER (either the master node or slave node. This command can also obtain the node ID, IP address, port number, flag, the last PING time, the last time PONG is received, the connection status, and the slot that the node is responsible for processing.

redis-cli -p 7001 cluster nodes481e256be4c724f5a2c64a761e52b4be61ca45b4 127.0.0.1:7002 master - 0 1431186119174 2 connected 5461-10922b3b8541b9520d707180d56a2fb3cf3ee6895ed10 127.0.0.1:7005 slave 481e256be4c724f5a2c64a761e52b4be61ca45b4 0 1431186120677 5 connectedd2b437ca8b9007dcdb63ac16210f6540860361e3 127.0.0.1:7006 slave b5b652fa02d9999861e66c843b01fd2700c02adf 0 1431186119174 6 connectedb5b652fa02d9999861e66c843b01fd2700c02adf 127.0.0.1:7003 master - 0 1431186118673 3 connected 10923-16383821ec823dc0c2d4f65319e84fe74157fb1014155 127.0.0.1:7004 slave 1984c27297c6ef50bbfcbd35c11b93cc40ba17e4 0 1431186120176 4 connected1984c27297c6ef50bbfcbd35c11b93cc40ba17e4 127.0.0.1:7001 myself,master - 0 0 1 connected 0-5460

5. Connect to the Redis Cluster

Through the above output, we can see the slot range of the three Redis master nodes. A Redis client can send command requests to any node (including slave nodes) in the cluster. First, connect to the first node:

redis-cli -p 7001127.0.0.1:7001> set a 1 (error) MOVED 15495 127.0.0.1:7003127.0.0.1:7001> get a(error) MOVED 15495 127.0.0.1:7003127.0.0.1:7001> set b 1OK

The node analyzes the command request and calculates the key slot, and finds the slot of the key to be processed by the command. If the hash slot to be searched is processed by the node that receives the command, the node directly executes the command.

On the other hand, if the slot to be searched is not processed by this node, the node will view the ing record of the hash slot saved inside itself to the node ID, and return a MOVED error to the client. The error message above contains the hash slot 15495 to which key x belongs, and the IP address and port number of the node responsible for processing the slot 127.0.0.1: 7003.

Although we use Node ID to identify the nodes in the cluster, in order to make the client's redirection operation as simple as possible, the Node directly returns the IP address and port number of the target Node in the MOVED error, instead of the ID of the target node. The client should record the slot 15495 and the node 127.0.0.1: 7003 is responsible for processing this information. In this way, when there is a command to execute slot 15495 again, the client can speed up searching for the correct node. In this way, when the cluster is in a stable state, all clients will eventually save a ing record from the hash slot to the node, making the cluster very efficient: the client can directly send command requests to the correct node, there is no need for turning, proxy, or any entity that may experience single point of failure ).

For more details, please continue to read the highlights on the next page:

1 2 Next Page