Redis Learning Notes (13) Redis Sentinel Introduction and Deployment _redis

Redis Sentinel Introduction and Deployment 1. Sentinel about 1.1 Master-slave replication issues

Redis Master-slave replication can synchronize the main node data from the node, from the node at this time has two functions: once the main node down, from the node as the main node of the backup can be on the top. Extend the reading ability of the main node and share the reading pressure of the main node.

But the problem is: once the main node is down, the promotion from the node to the main node, as well as the need to modify the principal node address of the application, you also need to command all from the node to replicate the new master node, the entire process requires human intervention. The primary node's ability to write is limited by a single machine. The primary node's storage capacity is limited by a single machine.

The first question, the next sentinel we'll be able to solve. The latter two problems, Redis also gave a solution Redis Cluster. High availability of 1.2 Redis Sentinel

Redis Sentinel is a distributed architecture that contains a number of sentinel nodes and Redis data nodes, each sentinel node monitors data nodes and the rest of the Sentinel nodes, and identifies nodes when they are unreachable.

If the primary node is identified, he also chooses to "negotiate" with other sentinel nodes, and when most sentinel nodes assume that the master node is unreachable, they elect a sentinel node to complete the automatic failover process and notify the Redis application of the change.

The whole process is completely automated and does not require human intervention, so it can be a good solution to redis high availability issues.

Next, we'll look at the overall framework by deploying a Redis Sentinel instance. 2. Redis Sentinel Deployment

The topology we deploy is as shown in the figure:

There are 3 Sentinel nodes, 1 primary nodes, and 2 from nodes to form a Redis Sentinel. Role IP Port master 127.0.0.1 6379 slave1 127.0.0.1 6380 slave2 127.0.0.1 6381 Sentinel1 127.0.0.1 26379 Sentinel2 127.0.0 .1 26380 Sentinel3 127.0.0.1 26381 2.1 Start the primary node configuration:

Port 6379
daemonize Yes
logfile "6379.log"
dbfilename "Dump-6379.rdb"
dir "/var/redis/data/"

1 2 3 4 5 Start the main node

➜   sudo redis-server redis-6379.conf

1 using the ping command to detect whether to start

➜   redis-cli-h 127.0.0.1-p 6379 Ping
PONG

1 2 2.2 Boot two from node configuration (two from node configuration same, except file name is differentiated)

Port 6380
daemonize Yes
logfile "6380.log"
dbfilename "Dump-6380.rdb"
dir "/var/redis/data/" 
slaveof 127.0.0.1 6379      /Slave master node

1 2 3 4 5 6 start two from nodes

➜   sudo redis-server redis-6380.conf 
➜   

1 2 Use the ping command to detect whether to start

➜   redis-cli-h 127.0.0.1-p 6380 ping PONG ➜ redis-cli-h 127.0.0.1-p 6381   Ping
PONG

1 2 3 4 2.3 Confirm master-Slave principal node perspective

➜   redis-cli-h 127.0.0.1-p 6379 INFO replication
# replication
role:master connected_slaves:2 Slave0:ip=127.0.0.1,port=6380,state=online,offset=85,lag=0
Slave1:ip=127.0.0.1,port=6381,state=online, Offset=85,lag=0 ...

1 2 3 4 5 6 7 from node perspective (6380 port)

➜   redis-cli-h 127.0.0.1-p 6380 INFO replication
# replication
role:slave
master_host:127.0.0.1< c16/>master_port:6379
master_link_status:up
...

1 2 3 4 5 6 7

Established from the relationship, as shown in the following illustration:

2.4 Department Sentinel Node

The deployment method for 3 Sentinel nodes is the same (different ports). Take 26379 for example. Configuration

Sentinel node Port
26379  
dir/var/redis/data/
logfile "26379.log"

//Current sentinel node monitoring 127.0.0.1:6379 this master
//2 is to determine that the primary node fails at least 2 Sentinel node node consent
//MyMaster is the alias of the master Node
Sentinel Monitor MyMaster 127.0.0.1 6379 2

//Each sentinel node will ping the command periodically to determine whether the Redis data node and the remaining sentinel nodes are up to, and if there is more than 30000 milliseconds and no reply, then the decision cannot be reached
Sentinel Down-after-milliseconds MyMaster 30000

//When the Sentinel node set agrees to the fault judgment of the main node, the Sentinel leader node will do the failover operation to select the new master node. The original from node initiates a replication operation to the new master node, limiting the number of nodes from 1
Sentinel Parallel-syncs mymaster 1

//failover timeout of 180000 milliseconds to the new primary node each time the replication operation is initiated.
Sentinel Failover-timeout MyMaster 180000

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15-16 17 18 Start (two methods)
Redis-sentinel sentinel-26379.conf redis-server sentinel-26379.conf--sentinel

sudo redis-sentinel sentinel-26379.conf--sentinel

1 Confirmation

➜   redis-cli-h 127.0.0.1-p 26379 INFO Sentinel
# Sentinel
sentinel_masters:1 sentinel_tilt:0 sentinel_running_scripts:0
sentinel_scripts_queue_length:0
sentinel_simulate_failure_flags:0
Master0:name=mymaster,status=ok,address=127.0.0.1:6379,slaves=2,sentinels=1//sentinels=1 says 1 Sentinel have been launched

1 2 3 4 5 6 7 8

After deploying three sentinel nodes, the entire topology is as shown in the figure:

When the deployment number Redis Sentinel, the following changes occur
The Sentinel node automatically discovers from the node, the remaining sentinel nodes. The default configuration is removed, for example: Parallel-syncs, Failover-timeout. The New era (epoch) parameter was added.

We take the example of Port 26379 and start all the Sentinel and data nodes after the configuration file is as follows:

Port 26379
dir "/var/redis/data"
Sentinel myID 70a3e215c1a34b4d9925d170d9606e615a8874f2
Sentinel Monitor MyMaster 127.0.0.1 6379 2
Sentinel Config-epoch mymaster 0
Sentinel Leader-epoch mymaster 0
Daemonize Yes
logfile "26379.log"
//Found two
Sentinel Known-slave mymaster 127.0.0.1 6381 Sentinel
Known-slave mymaster 127.0.0.1 6380
//sent a sentinel node Sentinel Known-sentinel mymaster 127.0.0.1
26381 E1148ad6caf60302dd6d0dbd693cb3448e209ac2
Sentinel Known-sentinel mymaster 127.0.0.1 26380 39DB5B040B21A52DA5334DD2D798244C034B4FC3
Sentinel Current-epoch 0

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 2.5 failover Experiment

First look at the node's process PID

➜   ps-aux | grep redis
root     18225  0.1 0.0 40208  ?        SSL  22:10   0:05 redis-server 127.0.0.1:6379
root     18234  0.0  0.0 38160?        SSL  22:10   0:04 redis-server 127.0.0.1:6380
root     18244  0.0  0.0 38160 8308 ?        SSL  22:10   0:04 redis-server 127.0.0.1:6381
root     20568  0.1  0.0 38160 8460 ?        SSL  23:05   0:02 redis-sentinel *:26379 [Sentinel]
root     20655  0.1  0.0  38160  8296?        SSL  23:07   0:02 redis-sentinel *:26380 [Sentinel]
root     20664  0.1  0.0  38160  8312?        SSL  23:07   0:02 redis-sentinel *:26381 [Sentinel]

1 2 3 4 5 6 7

We kill the primary node of Port 6379.

➜   sudo kill-9 18225
➜   ps-aux | grep redis
root     18234        0.0 0.0 38160? SSL  22:10   0:05 redis-server 127.0.0.1:6380
root     18244  0.0  0.0 38160 8308 ?        SSL  22:10   0:05 redis-server 127.0.0.1:6381
root     20568  0.1  0.0 38160 8460 ?        SSL  23:05   0:03 redis-sentinel *:26379 [Sentinel]
root     20655  0.1  0.0  38160  8296?        SSL  23:07   0:03 redis-sentinel *:26380 [Sentinel]
root     20664  0.1  0.0  38160  8312?        SSL  23:07   0:03 redis-sentinel *:26381 [Sentinel]

1 2 3 4 5 6 7

At this point, Redis Sentinel to the main node of the objective offline (objectively down, referred to as Odown) the judgment, to confirm the main node is not up to, then notify the node to abort the replication of the main node operation.

Redis Sentinel performs a failover operation when the main node is offline longer than the configured offline length of 30,000 seconds.

At this point, we look at the primary node information that the Sentinel node monitors:

127.0.0.1:26379> Sentinel Masters 
1)  1) "Name"
    2) "MyMaster"
    3) "IP"
    4) "127.0.0.1"
    5) " Port "
    6" "6380"           //You can see that the master node has become the 6380 port node
    7 "Runid"
    8) "084850ab4ff6c2f2502b185c8eab5bdd25a26ce2"
    9) "The Flags" "
   Master" ........
    

1 2 3 4 5 6 7 8 9 10 11-12

Take a look at the information from the nodes monitored by the sentinel node:

127.0.0.1:26379> Sentinel Slaves MyMaster
1)  1) "Name"
    2) "127.0.0.1:6379"             //ip:port
    3) "IP"
    4 "127.0.0.1" 5 ""
    Port "
    6" "6379"
    7) "Runid"
    8) ""
    9 "" Flags ""
   S_down, The slave,disconnected "  //Port 6379 's primary node has been disconnected
... ..... 2)  1) "Name"
    2 "127.0.0.1:6381"             
    3) "IP"
    4 "" 127.0.0.1 "
    5)" Port "
    6)" 6381 "
    7)" Runid "
    8" 24495fe180e4fd64ac47467e0b2652894406e9e4 "
    9" "Flags" "
   slave"                      //Original from node, or from the role of the node ... .....
    

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22-23

As the above information is available, the Redis data node with Port 6380 becomes the new master node, and the old primary node with Port 6379 is disconnected. As shown in the figure:

We're trying to focus on starting port 6379 data nodes.

➜   sudo redis-server redis-6379.conf 
➜   ps-aux | grep redis              
root     18234  0.1  0.0  40208 11392?        SSL  May   0:06 redis-server 127.0.0.1:6380
root     18244  0.1        0.0 40208? SSL  May   0:07 redis-server 127.0.0.1:6381
root     20568  0.1        0.0 38160? SSL  May   0:05 redis-sentinel *:26379 [Sentinel]
root     20655  0.1 0.0 38160 8296?        SSL  May   0:05 redis-sentinel *:26380 [Sentinel]
root     20664  0.1 0.0 38160 8312?        SSL  May   redis-sentinel *:26381 [Sentinel] Menwen 22475 0:05 0.0  14216  5920 PTS/2    S+   May   0:00 redis-cli-p 26379
//6379 Data node has been restarted
root     22617  0.0 0.0 38160  8304?        SSL  00:00   0:00 redis-server 127.0.0.1:6379   

1 2 3 4 5 6 7 8 9 10

See what happens:

127.0.0.1:26379> Sentinel Slaves MyMaster
1) "  name"
    2) "127.0.0.1:6379" The     node of the//6379 port reboots, becomes "live" From Node
    3) "IP"
    4 "127.0.0.1"
    5 "" Port "
    6)" 6379 "
    7)" Runid "
    8)" de1b5c28483cf150d9550f8e338886706e952346 "
    9" "Flags" "
   slave"
........ 2)  1) "Name" The               node of the//6381 port does not change, still from node
    2 "127.0.0.1:6381"
    ...

1 2 3 4, 5 6 7 8 9 10 11 12 13 14 15

He was demoted to port 6380 from the node.

From the above logical architecture and failover test, we can see the following functions of Redis Sentinel. Monitoring: The sentinel node periodically detects whether the Redis data node and the rest of the sentinel nodes are up to date. Notification: The Sentinel node notifies the application of the failover. Master node failover: Implement the promotion from the node to the main node and maintain the correct master-slave relationship. Configuration provider: In the Redis Sentinel structure, the client connects to the Sentinel node collection at initialization time to obtain the master node information. 3. Sentinel Configuration Instructions

Sentinel Monitor MyMaster 127.0.0.1 6379 2 Current sentinel node monitoring 127.0.0.1:6379 This master node 2 indicates that the primary node failure requires at least 2 sentinel node nodes to agree MyMaster is the alias of the primary node

Sentinel Down-after-milliseconds MyMaster 30000 each sentinel node periodically pings to determine whether the Redis data node and the remaining sentinel nodes are up to, If there is more than 30000 milliseconds and no reply, the decision is not reached

Sentinel Parallel-syncs MyMaster 1 when the Sentinel node set agrees to the fault of the main node, the Sentinel leader node will do the failover operation, select the new master node, and the original node will initiate the copy operation to the new master node. Restricts the number of nodes that initiate a replication operation to the new master node at 1 each time.

Sentinel failover-timeout MyMaster 180000 failover timeout is 180000 sentinel \ \
If the Sentinel monitored master has a password configured, you can prevent sentinel nodes from monitoring the primary node by adding a password to the master node through the Sentinel Auth-pass configuration. For example: Sentinel Auth-pass mymaster mysuper--secret-0123passw0rd Sentinel notification-script \ \
During failover, when some warning levels of Sentinel events occur (referring to important events, such as the subjective line, objective offline, etc.), the script that triggers the corresponding path is triggered, and the corresponding event parameters are sent to the script. For example: Sentinel notification-script mymaster/var/redis/notify.sh Sentinel client-reconfig-script \ \
After the failover is over, the script that triggers the response path is fired and the parameters for the failover result are sent to the script. For example: Sentinel Client-reconfig-script mymaster/var/redis/reconfig.sh.