Redis cluster set-up and testing course

Please note that this tutorial is used for Redis3.0 (including 3.0) versions above

Redis Cluster Introduction

The Redis cluster is an assembly that provides

data sharing between nodes across multiple Redis .

The Redis cluster does not support commands that handle multiple keys, because it requires moving data between different nodes to not achieve performance like Redis, which can cause unpredictable errors in high load situations.

The Redis cluster provides a degree of availability through partitioning, and continues to process commands when a node is down or unreachable in the actual environment. Advantages of Redis cluster: Automatic segmentation of data to different nodes. The entire cluster can continue processing commands if some of its nodes fail or are unreachable. 

data fragmentation of Redis cluster

Instead of using a consistent hash, the Redis cluster introduces the concept of a hash slot .

The Redis cluster has 16,384 hash slots, and each key determines which slot to place by CRC16 16384 modulo. Each node of the cluster is responsible for a portion of the hash slot, for example, for example, the current cluster has 3 nodes, then: node A contains 0 to No. 5500 hash slots. Node B contains 5501 to 11,000th hash slots. Node C contains 11001 to 16,384th hash slots.

This structure is easy to add or remove nodes. For example, if I want to add a new node D, I need to get some slots from node A, B, C to D. If I were to remove node A, I would need to move the slots in a to the B and C nodes. You can then remove a node without any slots from the cluster. Since a hash slot is moved from one node to another without stopping the service, adding a deletion or changing the number of hash slots for a node does not cause the cluster to be unusable .

  the master-slave replication model of Redis cluster

In order for a cluster to be available if a partial node fails or most nodes cannot communicate, the cluster uses the master-slave replication model, and each node has a N-1 replica.

In our example, a cluster with a,b,c three nodes, without replicating the model, if node B fails, the entire cluster will assume that it is missing 5501-11000 slots for this range and is not available.

However, if, at the time of the cluster creation (or over time) we add a node a1,b1,c1 to each node, the entire cluster has three master nodes and three slave nodes, so that after Node B fails, the cluster will elect B1 to continue serving the new master node. The entire cluster will not be available because the slot is not found.

But when both B and B1 fail, the cluster is not available. 

Redis Consistency Guarantee

Redis does not guarantee strong consistency of data. This means that in practice the cluster may lose write operations under certain conditions.

The first reason is that the cluster uses asynchronous replication. Write procedure: The client writes a command to the primary Node B. The main node B responds to the command state to the client. The master node copies the write to him from node B1, B2 and B3.

The primary node's replication of the command occurs after returning the command reply, because if the command request needs to wait for the replication operation to complete, the master node's handling of the command request will be greatly reduced-we must weigh the performance against the consistency. Note: The Redis cluster may provide a way to write synchronously in the future. Redis cluster Another scenario where a command might be lost is when a cluster appears with a network partition, and a client is orphaned from a few instances, including at least one primary node.

For example, suppose a cluster contains a, B, C, A1, B1, C1 six nodes, where a, B, C as the main node, A1, B1, C1 as A,b,c from the node, there is a client Z1 assume that the cluster occurs in the network partition, then the cluster may be divided into two sides, most of the one The side contains nodes A, C, A1, B1, and C1, and a small portion contains node B and client Z1.

Z1 is still able to write to Master B, and if the network partition occurs for a short time, the cluster will continue to function properly, and if the partition time is sufficient for the majority of the party to elect B1 as the new master, then the Z1 write B data will be lost.

Note that the maximum time that a client Z1 can send write commands to primary B is limited during network fragmentation, a time limit known as node timeout timeout, an important configuration option for the Redis cluster: building and using Redis clusters

The first thing to build a cluster we need some Redis instances running in cluster mode. This means that the cluster is not made up of ordinary redis instances, and cluster mode needs to be configured to enable the Redis instance of cluster mode to use the cluster-specific commands and attributes.

The following is a configuration file for a cluster with the fewest options:

Port 7000
cluster-enabled Yes
cluster-config-file nodes.conf
cluster-node-timeout 5000
appendonly Yes

The cluster-enabled option in the file is used for the clustered mode of the instance, while the Cluster-conf-file option sets the path to save the node profile, and the default value is nodes.conf. The node profile does not need to be artificially modified by the Redis The cluster is created at startup and automatically updated when necessary.

It takes at least three primary nodes for the cluster to function properly, but it is strongly recommended to use six nodes when the cluster functionality is first tested: Three are primary, while the remaining three are the nodes from each master node.

First, let's go into a new directory and create six subdirectories with the name of the port number, and later we'll run a Redis instance in each directory: the commands are as follows:

mkdir cluster-test
cd cluster-test
mkdir 7000 7001 7002 7003 7004 7005

In folders 7000 through 7005, each creates a redis.conf file whose contents can use the example configuration file above, but remember to change the port number in the configuration from 7000 to the same number as the folder name.

Remove the latest Redis source from the unstable branch of the Redis Github page, compile the executable redis-server, copy the files to the Cluster-test folder, and then use a command like this to open a real Cases:

CD 7000
... /redis-server./redis.conf

The log that the instance prints is displayed because the nodes.conf file does not exist, so each node specifies a new ID for itself:

[82462] Nov 11:56:55.329 * No cluster configuration found, I ' m 97a3a64667477371c4479320d683e4c8db5858b1

The instance always uses the same ID to maintain a unique (unique) name in the cluster.

The above is the reference http://www.redis.cn/topics/cluster-tutorial.html here's the note,

Here is my own cluster environment:

My computer is a CENTOS7 system directly on the top of this set up, there is no to get the virtual machine;

I used my computer here to build 6 nodes, creating 3 master Nodes (master) and 3 nodes (slave)

The Redis version is the redis-3.2.8 version.

installation process:

1. Download Reis and Unzip

Download Address: Https://redis.io/download

TAR-ZXVF redis-3.2.8.tar.gz

2. Compile and install

CD redis-3.2.8

Make && make install

3. Create Redis node

First create a redis-cluster directory under the installation directory (redis-3.2.8)

mkdir Redis-cluster

And then in the Redis-cluster directory to create a new 6 directories, respectively: 7000-7005

mkdir 7000 7001 7002 7003 7004 7005

Then copy the redis-3.2.8/redis.conf profile to the 7000-7005 directory separately

CP redis.conf redis-cluster/7000

CP redis.conf redis-cluster/7001

CP redis.conf REDIS-CLUSTER/7002

CP redis.conf redis-cluster/7003

CP redis.conf redis-cluster/7004

CP redis.conf redis-cluster/7005

Modify the configuration files separately and modify the content format as follows:

Port  7000       //Port 7000,7001,7002,7003,7004,7005       
bind native IP      //default IP for 127.0.0.1 need to be changed to other node machine accessible IP Otherwise, when the cluster is created, the corresponding port cannot be accessed and the cluster cannot be created.
daemonize    Yes                          //redis background run
pidfile  /var/run/redis_7000.pid          //pidfile file corresponds  
to 7000,7001,7002,7003,7004,7005 cluster-enabled  Yes                      //Open cluster  take note #
out Cluster-config-file  nodes_7000.conf   //cluster configuration Profile  first starts automatic generation 7000,7001,7002,7003,7004,7005
Cluster-node-timeout  15000               //Request Timeout  default 15 seconds, you can set the
appendonly  Yes//aof log Open Open if necessary, and it will record a log every time it writes

4. Copy the REDIS-TRIB.RB to the/usr/local/bin directory

Redis-trib.rg under the SRC directory

CP Redis-trib.rg/usr/local/bin

5. Start each node, if it is placed on the top of 6 computers, you can directly set up to start automatically, do not need us to start.

[Root@localhost redis-3.2.8]# Redis-server redis-cluster/7002/redis.conf
[Root@localhost redis-3.2.8]# Redis-server redis-cluster/7003/redis.conf
[Root@localhost redis-3.2.8]# Redis-server redis-cluster/7004/redis.conf
[Root@localhost redis-3.2.8]# Redis-server redis-cluster/7005/redis.conf
[Root@localhost redis-3.2.8]# Redis-server redis-cluster/7006/redis.conf
[Root@localhost redis-3.2.8]# Redis-server redis-cluster/7001/redis.conf

6. Check the start-up of the Redis, you can find a cluster logo behind.

[Root@localhost redis-3.2.8]# Ps-ef | grep Redis

Root 1364 1 0 08:51? 00:00:02/usr/local/bin/redis-server 127.0.0.1:6379
Root 5916 1 0 09:22? 00:00:00 redis-server 127.0.0.1:7000 [cluster]
Root 5931 1 0 09:22? 00:00:00 redis-server 127.0.0.1:7001 [cluster]
Root 6006 1 0 09:24? 00:00:00 redis-server 127.0.0.1:7002 [cluster]
Root 6013 1 0 09:24? 00:00:00 redis-server 127.0.0.1:7003 [cluster]
Root 6020 1 0 09:24? 00:00:00 redis-server 127.0.0.1:7004 [cluster]
Root 6027 1 0 09:24? 00:00:00 redis-server 127.0.0.1:7005 [cluster]

7. Build the cluster

Now that we have six running Redis instances, we need to use these instances to create the cluster and write a configuration file for each node.

By using the Redis Cluster command-line tool Redis-trib, the task of writing node profiles can be done very easily: Redis-trib is located in the src folder of Redis source code,

It is a Ruby program that works by sending special commands to an instance to create a new cluster, check the cluster, or reshared the cluster.

Because the tool is a Ruby program, you need a ruby environment, and there is no ruby environment in the system. To Install the Ruby environment:

The installation commands are as follows:

Yum-y install ruby Ruby-devel rubygems rpm-build

Gem Install Redis

Above in the execution command: Gem install Redis will be stuck there, or error: Error:while executing gem ... (Openssl::ssl::sslerror)
Hostname "upyun.gems.ruby-china.org" does not match the server certificate

So I found the information on the Internet, the card is not moving there because of the inability to access the site caused,

So we used a mirror of Taobao to install the Redis, the command is as follows

[Root@localhost bin]# Gem sources-a https://ruby.taobao.org/
https://ruby.taobao.org/added to sources
[root@localhost bin]# gem install Redis
Error:while Executing gem ... (Openssl::ssl::sslerror)
Hostname "upyun.gems.ruby-china.org" does not match the server certificate
[root@localhost bin]# sudo gem install Redis

Fetching:redis-3.3.3.gem (100%)

Successfully installed redis-3.3.3
Parsing documentation for redis-3.3.3
Installing RI documentation for redis-3.3.3
1 Gem installed

Note: If you want to add sudo, there will be an error.

Error:while Executing gem ... (Openssl::ssl::sslerror)
Hostname "upyun.gems.ruby-china.org" does not match the server certificate

reason :
is because the OpenSSL version is too low or missing OpenSSL, you need to install the OpenSSL package

You can use the command openssl-version to see if there are OpenSSL.

My system version is as follows, should be the version is too low

[Root@localhost ~]# OpenSSL version
OpenSSL 1.0.1e-fips Feb 2013

can refer to the article here: http://ruby-china.org/topics/3324

If you are installing OpenSSL you can refer to: http://blog.csdn.net/huang930528/article/details/51027915

using tool Redis-trib to create a cluster:

Enter the following command: REDIS-TRIB.RB create--replicas 1 127.0.0.1:7000 127.0.0.1:7001 127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005

There will be an input in the middle: enter Yes.

[Root@localhost redis-3.2.8]# redis-trib.rb Create--replicas 1 127.0.0.1:7000 127.0.0.1:7001 127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005
>>> Creating cluster
>>> performing hash slots allocation on 6 nodes ...
Using 3 Masters:
127.0.0.1:7000
127.0.0.1:7001
127.0.0.1:7002
Adding replica 127.0.0.1:7003 to 127.0.0.1:7000
Adding replica 127.0.0.1:7004 to 127.0.0.1:7001
Adding replica 127.0.0.1:7005 to 127.0.0.1:7002
M:3965DF7013F41EB1215E4FD9610F6900BBF911A1 127.0.0.1:7000
slots:0-5460 (5461 slots) Master
M:fe4601cbb9237b5d6c56c0d1582c15c1cc1144e2 127.0.0.1:7001
slots:5461-10922 (5462 slots) Master
M:c30c50d1bdc3112120f638411f8f03bcfad657e8 127.0.0.1:7002
slots:10923-16383 (5461 slots) Master
S:987A19E035237473936819F80D5E2080CE4F5D3C 127.0.0.1:7003
Replicates 3965DF7013F41EB1215E4FD9610F6900BBF911A1
s:10b11408aa6d735b7d6dd3cddad6f9bf3725ff69 127.0.0.1:7004
Replicates fe4601cbb9237b5d6c56c0d1582c15c1cc1144e2
S:565025FC12F7D731EDDF15B49F56D083CC3690CE 127.0.0.1:7005
Replicates C30c50d1bdc3112120f638411f8f03bcfad657e8
Can I Set the above configuration? (Type ' yes ' to accept): Yes
>>> Nodes Configuration Updated
>>> Assign a different config epoch to each node
>>> sending CLUSTER MEET messages to join the CLUSTER
Waiting for the "cluster" to join ...
>>> performing Cluster Check (using node 127.0.0.1:7000)
M:3965DF7013F41EB1215E4FD9610F6900BBF911A1 127.0.0.1:7000
slots:0-5460 (5461 slots) Master
1 additional replica (s)
s:10b11408aa6d735b7d6dd3cddad6f9bf3725ff69 127.0.0.1:7004
Slots: (0 slots) Slave
Replicates fe4601cbb9237b5d6c56c0d1582c15c1cc1144e2
S:565025FC12F7D731EDDF15B49F56D083CC3690CE 127.0.0.1:7005
Slots: (0 slots) Slave
Replicates C30c50d1bdc3112120f638411f8f03bcfad657e8
M:fe4601cbb9237b5d6c56c0d1582c15c1cc1144e2 127.0.0.1:7001
slots:5461-10922 (5462 slots) Master
1 additional replica (s)
S:987A19E035237473936819F80D5E2080CE4F5D3C 127.0.0.1:7003
Slots: (0 slots) Slave
Replicates 3965DF7013F41EB1215E4FD9610F6900BBF911A1
M:c30c50d1bdc3112120f638411f8f03bcfad657e8 127.0.0.1:7002
slots:10923-16383 (5461 slots) Master
1 additional replica (s)
[OK] All nodes agree about slots configuration.
>>> Check for open Slots ...
>>> Check Slots Coverage ...
[OK] All 16384 slots covered.

You can see the automatic allocation of 3 master and 3 slaver

8. Using Cluster

Open a terminal terminal, the connection port is 7000 nodes, because it is on a system, so you can not write parameters-H 127.0.0.1

Connection mode: Redis-cli-h ip-c-P port with connection node binding

Parameter description:-h is the IP address after bind configured in the configuration file

-C means to connect to a cluster

-P Port Port

[Root@localhost redis-3.2.8]# redis-cli-h 127.0.0.1-c-P 7000
127.0.0.1:7000> set Zero "7000"
-> redirected to slot [10551] located at 127.0.0.1:7001
Ok
127.0.0.1:7001> keys *
1) "Zero"
127.0.0.1:7001>

Then a new terminal Terminal is opened and the connection port is 7005 nodes. View the value of the key set above the 7000 node.

[Root@localhost ~]# cd/home/software/redis/redis-3.2.8/
[Root@localhost redis-3.2.8]# redis-cli-h 127.0.0.1-c-P 7005
127.0.0.1:7005> Get Zero
-> redirected to slot [10551] located at 127.0.0.1:7001
"7000"
127.0.0.1:7001>

REDIS-CLI support for clustering is very basic, so it always relies on redis cluster nodes to turn it (redirect) to the correct node.

9. Simply say the principle

Redis cluster in the design, take into account the central, to the middleware, that is, each node in the cluster is an equal relationship, are equivalent, Each node holds its own data and the state of the entire cluster. Each node is connected to all other nodes, and these connections remain active, ensuring that we only need to connect to any node in the cluster to get data from other nodes. The

Redis cluster does not allocate data using the traditional consistency hash, but instead uses a different method called a hash slot. Redis cluster The default allocation of 16,384 slot, when we set a key, will use the CRC16 algorithm to take the model to get the slot, and then the key to the hash slot interval node, the specific algorithm is: CRC16 (key)% 16384. So when we see the set and get in the test, we jump directly to the 7001-port node. We can wipe out the range of slots allocated to the main node while creating the cluster, and I am:

M:3965DF7013F41EB1215E4FD9610F6900BBF911A1 127.0.0.1:7000
slots:0-5460 (5461 slots) Master
M:fe4601cbb9237b5d6c56c0d1582c15c1cc1144e2 127.0.0.1:7001
slots:5461-10922 (5462 slots) Master
M:c30c50d1bdc3112120f638411f8f03bcfad657e8 127.0.0.1:7002
slots:10923-16383 (5461 slots) Master
S:987A19E035237473936819F80D5E2080CE4F5D3C 127.0.0.1:7003
Replicates 3965DF7013F41EB1215E4FD9610F6900BBF911A1
s:10b11408aa6d735b7d6dd3cddad6f9bf3725ff69 127.0.0.1:7004
Replicates fe4601cbb9237b5d6c56c0d1582c15c1cc1144e2
S:565025FC12F7D731EDDF15B49F56D083CC3690CE 127.0.0.1:7005
Replicates C30c50d1bdc3112120f638411f8f03bcfad657e8

The Redis cluster will have the data in a master node and then synchronize data between this master and its corresponding salve. When the data is read, the data is also obtained from the consistent hashing algorithm to the corresponding master node. Only when a master is dead will a corresponding salve node be started, acting as master.

Note that you must have 3 or more primary nodes, or you will fail when you create the cluster, and the entire cluster will not be serviced when the number of surviving primary nodes is less than half the total number of nodes.

Well, so far the Redis cluster is successful, the test is OK.

Reference Documentation:

Http://www.redis.cn/topics/cluster-tutorial.html

Http://www.cnblogs.com/wuxl360/p/5920330.html

http://blog.csdn.net/liu2012huan/article/details/49909939

http://blog.csdn.net/huang930528/article/details/51027915

http://blog.csdn.net/shuai825644975/article/details/53446621