Redis 3.0 cluster learning path [1]

Zhou's family, sort out technical documents, and leave some tutorials for the next generation ......

Currently, the project uses the redis Master/Slave, which is extremely uncomfortable and has many limitations. Especially after the redis master is down, the maintenance is very troublesome and you are thinking about creating a redis cluster, redis released the cluster version in 10.1 this year. Start exploring...

Many people, once they see the latest cluster version on the official website, are full of blood. The first thing is to build the cluster environment. In fact, the monks do not need to start from the grassroots level, first, let's take a look at the official information. Most of the information below is based on the network and the same exists. It is purely for reference. Sorry !!!

Redis Master/Slave Problems

1. The Master/Slave Mode is too single, especially when the master server is down, data cannot be written.

2. Business Development and huge data volumes. The same data must be synchronized between the master and slave databases, which requires a large amount of storage resources.

3. Although sentinel exists, the efficiency is still very low.

A redis cluster is a facility for data sharing between multiple redis nodes. There is no doubt about this!

Redis clusters use partitions to provide a certain degree of availability: even if some nodes in the cluster fail or cannot communicate, the cluster can continue to process command requests.

Redis clusters provide the following two benefits:

(1) ability to automatically split data into multiple nodes.

In the big data era, a cluster must be able to implement a downtime without affecting the operation of the entire cluster. In other words, each machine must own the data of the other party, however, due to the slow expansion of the business, the data will become larger, and it is obviously difficult to store. Redis is doing well at this point, that is, "slice ". The internal "Private algorithm technology" is used to store data distributed to the Cluster machine. to read data, the "Private internal algorithm" is used to map the data to the corresponding machine for reading.

(2) When some nodes in the cluster fail or cannot communicate with each other, they can still process command requests.

In redis 3.0, redis adopts the master n-1 salver mode. That is to say, a master can only have several slave nodes, or even no slave node. However, I strongly recommend the mode, if conditions permit, the 1: N mode is recommended.

2. Cluster data sharding:

The purpose of a cluster is to distribute different keys to different redis nodes. Here we need a rule or algorithm. The common practice is to obtain the hash value of the key, the model is then calculated based on the number of nodes, but this method has obvious drawbacks. When we need to increase or decrease a node, it will cause a large number of keys to fail to hit. This proportion is quite high, so some people have proposed the concept of consistent hash.

Consistent Hash has four important features:

Balance: Some people define it as balance, which means that the hash result can be distributed to all nodes as much as possible, so that resources on each node can be effectively used.

Monotonicity: I am puzzled by the many translations of monotonicity, what I want is that when the number of nodes changes, the hash result should be as much as possible to protect the allocated content from being reassigned to the new node.

Dispersibility and load: These two actually mean the same thing, that is, the consistency hash algorithm should avoid duplication of key hashing as much as possible.

However:

Redis cluster does not use consistent hash, but introduces the concept of hash slots.

The redis cluster has 16384 hash slots. After each key passes crc16 verification, it modulo 16384 to determine which slot to place. each node in the cluster is responsible for part of the hash slots. This structure is easy to add or delete nodes and does not cause cluster unavailability.

The benefit of using a hash slot is that you can easily add or remove nodes.

When adding nodes, you only need to move some hash slots of other nodes to the new node;

To remove a node, you only need to move the hash slot on the removed node to another node;

In this regard, we do not need to stop all redis services when adding or removing nodes in the future. Very good

3. Master-slave architecture of redis cluster:

To make the cluster still available if some nodes fail or most nodes cannot communicate, the cluster uses a master-slave replication model, with each node having a N-1 replica.

For example, if a cluster with three nodes A, B, and C does not have a replication model, if Node B fails, the entire cluster is unavailable because the hash slot of Node B is missing.

However, if we add a slave node A1, B1, C1, the entire cluster is composed of three master nodes and three slave nodes. After Node B fails, the cluster will elect B1 as the new master node to continue service, the entire cluster will not be unavailable because the slot cannot be found. Of course, if both B and B1 are down, the cluster will still be unavailable. However, this situation is almost insignificant and you don't have to worry about it. If your switch crashes, or the data center loses power.

4. redis Architecture

650) This. width = 650; "src =" http://s3.51cto.com/wyfs02/M00/4B/72/wKiom1Qsx06wtVlXAAGEwXnBz08131.jpg "Title =" 134caad7-0591-3edd-9162-6ae43d068333.jpg "alt =" wkiom1qsx06wtvlxaagewxnbz08131.jpg "/>

Architecture details:

(1) All redis nodes are interconnected (Ping-Pong mechanism), and the binary protocol is used internally to optimize the transmission speed and bandwidth.

(2) fail takes effect only when more than half of nodes in the cluster fail to be detected.

(3) The client is directly connected to the redis node, without the intermediate proxy layer. The client does not need to connect to all nodes in the cluster, just connect to any available node in the cluster.

(4) redis-cluster maps all physical nodes to the [0-16383] slot, and the cluster maintains the node <-> slot <-> Value

5. redis-cluster election: Fault Tolerance

650) This. width = 650; "src =" http://s3.51cto.com/wyfs02/M01/4B/74/wKioL1Qsx8uwkQU2AAEcQpIDk1E403.jpg "Title =" empty "alt =" wkiol1qsx8uwkqu2aaecqpidk1e403.jpg "/>

(1) The election process involves the participation of all the master nodes in the cluster. If more than half of the master nodes communicate with the master node (cluster-node-timeout), the current master node is considered to have crashed.

(2) When the entire cluster is unavailable (cluster_state: fail). When the cluster is unavailable, all operations on the cluster are unavailable and receive (error) clusterdown the cluster is down) Error

A: If any master node of the cluster fails and the current master node does not have a Server Load balancer instance. The cluster enters the Fail status, it can also be understood as the Fail status when the slot ing to the cluster [0-16383] is not completed.

B: if more than half of the master nodes in the cluster are down, whether or not the slave cluster is in the Fail status

This article is from the "Zhou's family" blog and will not be reposted!

Redis 3.0 cluster learning path [1]