redis3.0 Cluster Construction

prepare to use redis3.0 cluster in production environment, took a day to study, the following record the cluster building process. Server Planning:192.168.116.129 7000,7003192.168.116.130 7001,7004192.168.116.131 7002,7005 I. Installation of Redis3 Servers with 2 Redis instances installed on top of each one. The installation steps are as follows:

wget Http://download.redis.io/releases/redis-3.0.4.tar.gztar REDIS-3.0.4.TAR.GZCD Redis-3.0.4makemake INSTALLMV Redis-3.0.4/app/sinova/rediscd/app/sinova/redismkdir {bin,conf,rdb,log}mv redis.conf sentinel.conf conffind- MaxDepth 1-type f-deletecd SRCMV mkreleasehdr.sh redis-benchmark redis-sentinel redis-server redis-cli redis-check-aof R Edis-check-dump redis-trib.rb. /BIN/MV redis-3.0.4/app/sinova/rediscp-r Redis Redis2

Second, configure Redis

Redis.conf modified: daemonize yespidfile/app/sinova/redis/log/redis.pidport 7000  #不同实例用不同端口logfile "/app/sinova/ Redis/log/redis.log "#不同实例定义不同路径dir/app/sinova/redis/rdb  #定义dump. Rdb and appendonly.aof file storage path AppendOnly Yes  #开启AOF持久化cluster-enabled yescluster-config-file nodes-7000.conf  #不同实例需修改端口cluster-node-timeout 5000

third, start Redis and create a cluster1. Start the redis,3 server and execute the following command separately

/app/sinova/redis/bin/redis-server  /app/sinova/redis/conf/redis.conf/app/sinova/redis2/bin/redis-server  /app/sinova/redis2/conf/redis.conf

2. Create a clusterThe command is as follows:

/APP/SINOVA/REDIS/BIN/REDIS-TRIB.RB Create--replicas 1 192.168.116.130:7000 192.168.116.129:7001 192.168.116.131:7002 192.168.116.130:7003 192.168.116.129:7004 192.168.116.131:7005

After executing the above command, Redis-trib will print out a desired configuration to show you, if you feel no problem, you can enter Yes, Redis-trib will apply this configuration to the cluster.

After you enter Yes and press ENTER to confirm, the cluster applies the configuration to each node and joins each node--that is, the nodes begin to communicate with each other, and if everything is OK, Redis-trib will output the following information:Note: Because REDIS-TRIB.RB is written in Ruby, if you encounter ruby-related errors during the cluster creation process, you can execute the following command to resolve the environmental issue first:yum-y install ruby RubyGemsgem install Redis #安装redis API Interface>>> Check for open slots ...>>> Check Slots Coverage ...[OK] all 16384 slots covered.>>> Creating ClusterConnecting to Node 192.168.116.130:7000:okConnecting to Node 192.168.116.129:7001:okConnecting to Node 192.168.116.131:7002:okConnecting to Node 192.168.116.130:7003:okConnecting to Node 192.168.116.129:7004:okConnecting to Node 192.168.116.131:7005:ok>>> performing hash slots allocation on 6 nodes ...Using 3 Masters:192.168.116.129:7001192.168.116.131:7002192.168.116.130:7000Adding replica 192.168.116.131:7005 to 192.168.116.129:7001Adding replica 192.168.116.129:7004 to 192.168.116.131:7002Adding replica 192.168.116.130:7003 to 192.168.116.130:7000M:7BEF1E428364F2323A300FC47553889117C6E2FA 192.168.116.130:7000slots:10923-16383 (5461 slots) MasterM:794DA90978E2BD501EB2A45E1F66458D9A4C70CF 192.168.116.129:7001slots:0-5460 (5461 slots) Masterm:7c075df721b8e7524e4732174307ea724e4d49d0 192.168.116.131:7002slots:5461-10922 (5462 slots) Masters:8c36b569793c1580ae1fb6e7c198ea5d291abb56 192.168.116.130:7003replicates 7bef1e428364f2323a300fc47553889117c6e2fas:c632d284b9ab38100984d15fa65c005ca12bb438 192.168.116.129:7004replicates 7c075df721b8e7524e4732174307ea724e4d49d0s:93752d84dc8dd430a439644f0dd3288ff3b50a49 192.168.116.131:7005replicates 794DA90978E2BD501EB2A45E1F66458D9A4C70CFCan I Set the above configuration? (Type ' yes ' to accept): Yes>>> Nodes Configuration Updated>>> Assign a different config epoch to each node>>> sending CLUSTER MEET messages to join the CLUSTERwaiting for the cluster to join ....>>> performing Cluster Check (using node 192.168.116.130:7000)M:7BEF1E428364F2323A300FC47553889117C6E2FA 192.168.116.130:7000slots:10923-16383 (5461 slots) MasterM:794DA90978E2BD501EB2A45E1F66458D9A4C70CF 192.168.116.129:7001slots:0-5460 (5461 slots) Masterm:7c075df721b8e7524e4732174307ea724e4d49d0 192.168.116.131:7002slots:5461-10922 (5462 slots) Masterm:8c36b569793c1580ae1fb6e7c198ea5d291abb56 192.168.116.130:7003Slots: (0 slots) Masterreplicates 7bef1e428364f2323a300fc47553889117c6e2fam:c632d284b9ab38100984d15fa65c005ca12bb438 192.168.116.129:7004Slots: (0 slots) Masterreplicates 7c075df721b8e7524e4732174307ea724e4d49d0m:93752d84dc8dd430a439644f0dd3288ff3b50a49 192.168.116.131:7005Slots: (0 slots) Masterreplicates 794DA90978E2BD501EB2A45E1F66458D9A4C70CF[OK] All nodes agree about slots configuration.>>> Check for open slots ...>>> Check Slots Coverage ...[OK] all 16384 slots covered.3, the Client Test command: Redis-cli-h 192.168.116.130-c-P 7000 by testing to find any instance, can write, query, Redis cluster will automatically through the internal algorithm election. Redis has 16 database, if not specified, using DB0 by default, different database can use the same key. Select the database, using the Select command. Cluster mode does not support select. Detects if the cluster is healthy: $/app/sinova/redis/bin/redis-trib.rb Check 192.168.116.131:7005
Connecting to Node 192.168.116.131:7005:ok
Connecting to Node 192.168.116.131:7002:ok
Connecting to Node 192.168.116.130:7000:ok
Connecting to Node 192.168.116.129:7001:ok
Connecting to Node 192.168.116.130:7003:ok
Connecting to Node 192.168.116.129:7004:ok
>>> performing Cluster Check (using node 192.168.116.131:7005)
S:93752d84dc8dd430a439644f0dd3288ff3b50a49 192.168.116.131:7005
Slots: (0 slots) Slave
Replicates 794DA90978E2BD501EB2A45E1F66458D9A4C70CF
M:7c075df721b8e7524e4732174307ea724e4d49d0 192.168.116.131:7002
slots:5461-10922 (5462 slots) Master
1 additional replica (s)
M:7BEF1E428364F2323A300FC47553889117C6E2FA 192.168.116.130:7000
slots:10923-16383 (5461 slots) Master
1 additional replica (s)
M:794DA90978E2BD501EB2A45E1F66458D9A4C70CF 192.168.116.129:7001
slots:0-5460 (5461 slots) Master
1 additional replica (s)
S:8c36b569793c1580ae1fb6e7c198ea5d291abb56 192.168.116.130:7003
Slots: (0 slots) Slave
Replicates 7BEF1E428364F2323A300FC47553889117C6E2FA
s:c632d284b9ab38100984d15fa65c005ca12bb438 192.168.116.129:7004
Slots: (0 slots) Slave
Replicates 7c075df721b8e7524e4732174307ea724e4d49d0
[OK] All nodes agree about slots configuration.
>>> Check for open Slots ...
>>> Check Slots Coverage ...
[OK] all 16384 slots covered. View Master in a cluster /app/sinova/redis/bin/redis-cli-h 192.168.116.131-p 7005 cluster Nodes|grep master View slave in a cluster /app/sinova/redis/bin/redis-cli-h 192.168.116.131-p 7005 cluster nodes|grep slave Iv. fail-over test1, first look at the distribution of Master/slave 2, select a master node, set a data, and then kill the master process. 3, check the new Master/slave (under normal circumstances will have slave upgrade to master) 4, in the newly upgraded master get before the set data, check whether it is normal. 5, before starting to kill the master Redis, verify whether it will join the ranks of slave. 6, re-check the distribution of Master/slave. 7. Kill the Redis node in the hash slot where the data is stored, and verify that the client can retrieve the data from the other nodes. The specific process is as follows:

v. Add a new node to the cluster

Depending on the type of newly added node, we need to add the new node to the cluster in two ways:
1> if the new node to be added is a master node, then we need to create an empty node, and then move some of the hash buckets into the empty node.
2> if the new node to be added is a slave node, then we need to set the new node as a replica of a node in the cluster (replica).
The next two scenarios are tested, first testing the master node's Add method, and then testing the Add method from the node.
Regardless of which node is added, the first step is to always add an empty node.
We can continue to use the previous methods of starting 192.168.116.130:7000, 192.168.116.129:7001 and other nodes, create a new port number of 7006, using the same configuration file as before, just remember to change the port number in the configuration to 7000.
The following are the detailed steps for starting a new node with a port number of 7006:
Install a new Redis instance on the 192.168.116.130 server with a port of 7006.
Use the command: /.. /redis-server redis.conf boot node.
If everything is OK, then the node should start correctly.

The steps to add the master node are as follows:1. Start a newly installed Redis instance (port 7006)/app/sinova/redis3/bin/redis-server/app/sinova/redis3/conf/redis.conf2. Add this new node to the cluster/app/sinova/redis3/bin/redis-trib.rb add-node 192.168.116.130:7006 192.168.116.131:7005the command add-node indicates that we want to redis-trib add a node to the cluster, followed addnode by the new node's IP address and port number, and then followed by any of the existing nodes in the cluster IP address and port number, here we use 192.168.116.131:7005 . Message after the command executes successfully: $/app/sinova/redis3/bin/redis-trib.rb add-node 192.168.116.130:7006 192.168.116.131:7005>>> Adding node 192.168.116.130:7006 to cluster 192.168.116.131:7005connecting to node 192.168.116.131:7005:okconnecting to Node 192.168.116.129:7001:okconnecting to node 192.168.116.130:7003:okconnecting to node 192.168.116.130:7000: Okconnecting to node 192.168.116.129:7004:okconnecting to node 192.168.116.131:7002:ok>>> Performing Cluster C Heck (using node 192.168.116.131:7005) s:1a4c26acd5dd77c30c89fb6c19b43a1895492196 192.168.116.131:7005    Slots: (0 slots) slave   replicates 1d19da785ddb633a102224a6bfeebed58707bd4cs: 40028659FBDF7E059CA2C6BDAEF8753F22E8A4B9 192.168.116.129:7001   slots: (0 slots) slave    Replicates ab99fc3fd207e14d5db047ff7273521d0d7d4eb6s:6d7defec0ec9705b7931bc527bcde8c8441e7fd3 192.168.116.130:7003   slots: (0 slots) slave   replicates f3e8cccb5517c8225b77c6238c1a8cad8cab9e2cm:1d19DA785DDB633A102224A6BFEEBED58707BD4C 192.168.116.130:7000   slots:5461-10922 (5462 slots) master   1 additional replica (s) m:ab99fc3fd207e14d5db047ff7273521d0d7d4eb6 192.168.116.129:7004   slots : 10923-16383 (5461 Slots) master   1 additional replica (s) m:f3e8cccb5517c8225b77c6238c1a8cad8cab9e2c 192.168.116.131:7002   slots:0-5460 (5461 slots) master   1 additional replica (s)[OK] All nodes agree about slots configuration.>>> Check for open slots...>>> Check slots coverage ...[OK] all 16384 slots covered.Connecting to Node 192.168.116.130:7006:ok>>> Send CLUSTER MEET to node 192.168.116.130:7006 to make it join the Cluster.[OK] New node added correctly.The cluster nodes command allows you to view the newly added nodes.

From the above you can see that the new node is now connected to the cluster, become a part of the cluster, and can be turned to the client's command request, but compared with other primary nodes, the new node has two points difference:
1> The new node does not contain any data because it does not contain any hash buckets.
2> Although the new node does not contain any hash buckets, it is still a master node, so the new node will not be selected when the cluster needs to upgrade a slave node to a new master node.

3, manually re-shard the clusterusing a redis-trib program to move some of the hash buckets in the cluster to the new node, the new node becomes the true master node. The command below is to specify the address of one of the nodes in the cluster and the redis-trib other nodes in the cluster will be found automatically. $/app/sinova/redis/bin/redis-trib.rb Reshard 192.168.116.129:7001connecting to node 192.168.116.129:7001: Okconnecting to node 192.168.116.129:7004:okconnecting to node 192.168.116.131:7002:okconnecting to node 192.168.116.13 0:7006:okconnecting to node 192.168.116.130:7003:okconnecting to node 192.168.116.130:7000:okconnecting to node 192.168 .116.131:7005:ok>>> performing Cluster Check (using node 192.168.116.129:7001) S: 40028659FBDF7E059CA2C6BDAEF8753F22E8A4B9 192.168.116.129:7001   slots: (0 slots) slave    Replicates Ab99fc3fd207e14d5db047ff7273521d0d7d4eb6m:ab99fc3fd207e14d5db047ff7273521d0d7d4eb6 192.168.116.129:7004   slots:10923-16383 (5461 slots) master   1 additional replica (s) M: F3E8CCCB5517C8225B77C6238C1A8CAD8CAB9E2C 192.168.116.131:7002   slots:0-5460 (5461 slots) master   1 additional replica (s) m:8d9502f97b416a566e0de1e33f665b7358fe807f 192.168.116.130:7006   slots: (0 Slots) Master&nbsP  0 additional replica (s) s:6d7defec0ec9705b7931bc527bcde8c8441e7fd3 192.168.116.130:7003   slots: (0 Slots) slave   replicates f3e8cccb5517c8225b77c6238c1a8cad8cab9e2cm: 1D19DA785DDB633A102224A6BFEEBED58707BD4C 192.168.116.130:7000   slots:5461-10922 (5462 slots) Master    1 additional replica (s) s:1a4c26acd5dd77c30c89fb6c19b43a1895492196 192.168.116.131:7005    Slots: (0 slots) slave   replicates 1d19da785ddb633a102224a6bfeebed58707bd4c[OK] All nodes agree about slots configuration.>>> Check for open slots...>>> Check slots coverage ...[OK] all 16384 slots covered.How many slots does want to move (from 1 to 16384)?4000#设定移动hash The number of slots, here we set the 4000What is the receiving node ID?8d9502f97b416a566e0de1e33f665b7358fe807f# 7006 The ID of the node can be seen through the/app/sinova/redis/bin/redis-cli-c-H 192.168.116.131-p7002 cluster nodes|grep Master command, and the 1th column is. ...... Please enter all the source node Ids.type "All" to the nodes as source nodes for the hash slots. Type ' Done ' once your entered all the source nodes ids.source node #1: All # Here's what it means redis-trib will ask you for the source node of the re-shard, which is the node from which to remove the 4000 Hash slots and move them to the target node. If we do not intend to remove the specified number of hash slots from a particular node, then we can redis-trib enter all , so that all the primary nodes in the cluster become the source nodes, and each of the redis-trib source nodes will be taken out of a portion of the hash slot, to gather 4,000, and then Move to the top of the target node. ..... Moving slot 12252 from ab99fc3fd207e14d5db047ff7273521d0d7d4eb6moving slots 12253 from Ab99fc3fd207e14d5db047ff7273521d0d7d4eb6moving slot 12254 from ab99fc3fd207e14d5db047ff7273521d0d7d4eb6moving slot 12255 from Ab99fc3fd207e14d5db047ff7273521d0d7d4eb6do do want to proceed with the proposed Reshard plan (yes/no)?Yes#输入 yes and using press ENTER, the redis-trib re-shard operation is formally started, and the specified hash slot is moved from the source node to the target node individually.....moving slot 12253 from 192.168.116.129:7004 to 192.168.116.130:7006:moving slot 12254 from 192.168.116.129:7004 to 192.168.116.130:7006:moving slot 12255 from 192.168.116.129:7004 to 192.168.116.130:7006:4, re-check the cluster is normal view hast slots the distribution of the new node to see if the history data can be re-obtained in re-sharding the steps to add from the node are as follows:1. Install a new Redis instance on the 192.168.116.129 server and launch the newly installed Redis instance (port 7007)/app/sinova/redis3/bin/redis-server/app/sinova/redis3/conf/redis.conf2. Add this new node to the cluster/app/sinova/redis3/bin/redis-trib.rb add-node 192.168.116.129:7007 192.168.116.131:7005the command add-node indicates that we want to redis-trib add a node to the cluster, followed addnode by the new node's IP address and port number, and then followed by any of the existing nodes in the cluster IP address and port number, here we use 192.168.116.131:7005 . The newly added node (7007) is currently in the master state, but there is no data3. Now we're going to make the new node a192.168.116.130:7006From the node, then we just use the client to connect to the new node, and then execute the following command/app/sinova/redis/bin/redis-cli-c-H 192.168.116.129-p 7007

Where 8d9502f97b416a566e0de1e33f665b7358fe807f is the ID of the master node.
Verify that Redis for 7006 and 7007 ports is a master slave relationship:

On the new slave node above the Get data:

vi. redis cluster command-line commandCluster (cluster) cluster info print cluster information cluster NODES lists all nodes currently known to the cluster (node), and information about those nodes.   Node CLUSTER MEET <ip> <port> Add the node specified by IP and port to the cluster, making it part of the cluster. CLUSTER Forget <node_id> removes node_id specified node from the cluster. CLUSTER REPLICATE <node_id> Sets the current node to node_id node from the specified node. CLUSTER Saveconfig Save the configuration file of the node to the hard disk. Slot (slot) CLUSTER addslots <slot> [slots ...] assigns one or more slots (slots) (assign) to the current node. CLUSTER delslots <slot> [slots ...] removes the assignment of one or more slots to the current node. CLUSTER Flushslots removes all slots assigned to the current node, making the current node a node without any slots assigned. CLUSTER setslot <slot> Node <node_id> assigns the slot slot to the node specified by node_id, and if the slot is already assigned to another node, let the other node delete the slot before assigning it. CLUSTER setslot <slot> Migrating <node_id> migrates the slot slots of this node to the nodes specified in the node_id. CLUSTER setslot <slot> importing <node_id> import slot slots from node_id specified node to this node. CLUSTER Setslot <slot> STABLE cancels the import or migration (migrate) of the slot slots. Key (key) CLUSTER keyslot <key> calculate key should be placed on which slot. CLUSTER Countkeysinslot <slot> Returns the number of key-value pairs currently contained in the slot slot. CLUSTER Getkeysinslot <slot> <count> returns the key in count slot slots.  

redis3.0 Cluster Build