Redis High-availability architecture best Practices

Redis High-availability architecture best Practices

First, preface

May 13, 2017, the application performance Management auditorium Guangzhou station successfully ended, among them from 37 Mutual entertainment DBA Wen in the venue and you have a wonderful Redis technology sharing.

Redis is an open source API that is written in ANSI C, supports the web, can be persisted in memory, key-value databases, and provides multiple languages.

Today, Internet business data is growing at a faster rate and the data types are becoming richer, which puts higher demands on the speed and capability of data processing. Redis is an open-source, memory-non-relational database that is disruptive to the developer experience. High performance has been considered throughout the design process, making Redis the fastest NoSQL database today.

High availability is also an important consideration when considering high performance. The Internet 7x24 uninterrupted service, in the fault period with the fastest speed Failover, can bring the smallest loss to the enterprise.

So, what are the high-availability architectures in real-world applications? What is the pros and cons of architecture? What we should choose. What are the best practices?

Second, Sentinel principle

Before we go into the Redis high-availability scenario, let's take a look at how the Redis Sentinel principle (Https://redis.io/topics/sentinel) works.

The Sentinel cluster discovers master through a given configuration file, which is monitored at startup. Get all the slave servers under the server by sending info info to master.

The Sentinel cluster sends Hello information (once per second) to the monitored master and slave servers via a command connection, which includes Sentinel's own IP, port, ID, and so forth to announce its presence to other Sentinel.

The Sentinel cluster receives hello information sent by other Sentinel through a subscription connection to discover other Sentinel monitors on the same primary server; The cluster creates a command connection to communicate with each other, as there are already master and slave servers as intermediaries for sending and receiving Hello Information , a subscription connection is not created between Sentinel.

The Sentinel cluster uses the ping command to detect the state of the instance, and if there is no reply or an incorrect reply within the specified time (down-after-milliseconds), then the instance is sentenced to a downline.

When the failover primary and standby switch is triggered, failover is not immediately available and requires most Sentinel authorization in Sentinel before failover can be performed, that is, Sentinel failover will go to the designated quorum The Sentinel's authorization, after successful entry into the Odown state. If 2 quorum are configured in 5 Sentinel, wait until 2 Sentinel believes that Master is dead and executes failover.

Sentinel sends the slaveof NO one command to the slave that is selected as Master, and the criteria for selecting Slave is that Sentinel first sorts according to the priority of the slaves, and the smaller the priority, the higher the ranking. If the priority is the same, look at the subscript of the copy, which one receives more copy data from Master, and which one is on top. If the priority and subscript are the same, select a small process ID.

When Sentinel is authorized, it will get a new configuration version number (Config-epoch) for the outage master, which will be used for the latest configuration after failover execution is completed, and the other sentinel can be notified by broadcast Sentinel updates the configuration that corresponds to master.

1 to 3 is the Autodiscover mechanism:

The info command is sent to the monitored master at a frequency of 10 seconds, and the master current information is obtained based on the reply.

Send a PING command to all Redis servers, including Sentinel, at a frequency of 1 seconds, to determine if the server is online by replying.

A message that sends the current Sentinel master information to all monitored master,slave servers at a frequency of 2 seconds.

4 is the detection mechanism, 5 and 6 are the failover mechanism, and 7 is the update configuration mechanism. [1]

Third. Redis High-availability architecture

After the Redis Sentinel principle is explained, the following is a common Redis high-availability architecture.

Redis Sentinel Cluster + Intranet DNS + custom scripts

Redis Sentinel cluster + VIP + custom Script

Encapsulating the client direct-attached Redis Sentinel port

Jedissentinelpool, suitable for Java

PHP self-encapsulation based on Phpredis

Redis Sentinel cluster + keepalived/haproxy

Redis m/s + keepalived

Redis Cluster

Twemproxy

Codis

Next with graphics and text to explain.

1. Redis Sentinel Cluster + Intranet DNS + custom script

Redis Sentinel Cluster + Intranet DNS + custom scripts

The above diagram is a scenario that has been applied on-line environment. The bottom layer is the Redis Sentinel cluster, which is a proxy for Redis master-slave and Web-connected intranet DNS services. Intranet DNS According to certain rules assigned, such as Xxxx.redis.cache/queue.port.xxx.xxx, the first segment represents a business shorthand, the second segment indicates that this is the Redis intranet domain name, the third segment represents the Redis type, the cache represents The queue represents the queues, and the fourth segment represents the Redis port, and the second and sixth segments represent the intranet primary domain name.

When the primary node fails, such as a machine failure, a Redis node failure, or a network unreachable, Sentinel cluster invokes the Client-reconfig-script configured script to modify the corresponding port's intranet domain name. The intranet domain name of the corresponding port points to the new Redis master node.

Advantages:

Seconds switch to complete the switching operation within 10s

Script customization, architecture controllable

Transparent to the application, front-end don't worry about what's going on back end

Disadvantages:

Slightly higher maintenance costs, the Redis Sentinel cluster is recommended to invest more than 3 machines

DNS dependent, parsing delay present

Sentinel mode is unavailable for a short time service

Service is not available through extranet access this scenario

2. Redis Sentinel cluster + VIP + custom Script

Redis Sentinel cluster + VIP + custom Script

This scenario is slightly different than the previous scenario. The first scenario uses the intranet DNS, and the second scheme replaces the intranet DNS with the virtual IP. The bottom layer is the Redis Sentinel cluster, which is a proxy for Redis master-slave and Web-based services through VIPs. When you deploy Redis master-slave, you need to bind the virtual IP to the current Redis master node. When the primary node fails, such as a machine failure, a Redis node failure, or a network unreachable, the Sentinel cluster invokes the Client-reconfig-script configured script and shifts the VIP to the new master node.

Advantages:

Seconds switch to complete the switching operation within 5s

Script customization, architecture controllable

Transparent to the application, front-end don't worry about what's going on back end

Disadvantages:

Slightly higher maintenance costs, the Redis Sentinel cluster is recommended to invest more than 3 machines

Using VIPs to increase maintenance costs, there is a risk of IP confusion

Sentinel mode is unavailable for a short time service

3.3 Package Client Direct-attached Redis Sentinel port

3. Encapsulating the client direct connection to the Redis Sentinel port