Redis cluster (redis 3.x) design highlights

Redis 3.0.0 RC1 version released on November 10.9, release note
This version supports Redis Cluster I believe many of you have been waiting for a long time. However, this version is only an RC version, so it must be applied to the production environment.

Redis cluster design points:

Architecture: no center Redis cluster adopts a non-central structure. Each node stores data and the status of the entire cluster.
Each node is connected to all other nodes. These connections are active.
Use the gossip protocol to disseminate information and discover new nodes
Node does not act as the proxy of the client request. The client redirects the request based on the error message returned by the node.

Data Distribution: Pre-divides 16384 buckets into pre-divides. Based on the value of crc16 (key) mod 16384, you can decide which bucket to store a key.
Each redis physical node is responsible for the management of a part of the bucket. When the redis node is increased or decreased, adjust the distribution of the bucket.
For example, assume that redis cluster has three nodes A/B/C
Node A can contain bucket numbers from 0 to 5500.
Node B can contain bucket numbers: 5500 to 11000.
Node C can contain bucket numbers: 11001 to 16384.
When a redis node is increased or decreased, adjust the bucket distribution.
The pre-sharding Bucket solution is between "Hard hash" and "consistent hash", which sacrifices some flexibility. However, compared with "consistent hash", data management costs are greatly reduced.

Availability: to ensure service availability, the solution adopted by redis cluster is master-slave.
Each redis node can have one or more slave instances. When the master fails, an slave is elected to form a new master.
A redis node contains a certain number of buckets. When the master and slave of these buckets are down, the data corresponding to these buckets is unavailable.

Write redis cluster using asynchronous replication
A complete write procedure:
1. The client writes data to the master
2. The master tells the client "OK"
3. Master propagation updates to slave
Data Loss Risks:
1. After the above write steps 1) and 2) are successful, the master crash, and the data has not been transmitted to the slave
2. As a result of the partition, two masters exist simultaneously, and the client writes data to the old master.
Of course, because redis cluster has a timeout and fault recovery mechanism, the 2nd risks are basically impossible.

Data migration to redis cluster supports online addition/reduction of nodes.
The bucket-based data distribution method greatly reduces the migration cost. You only need to migrate the data bucket from one redis node to another redis node to complete the migration.
When a bucket is migrated from one node A to another node B, node A and Node B both have this bucket, and the status of the bucket on node A is set to migrating, the status of the bucket on Node B is set to importing.
When the client requests:
All requests on node A are processed by a, and all keys not on a are processed by Node B. At the same time, no new key will be created on node.

Multi-key operations when the system scales from a single node to multiple nodes, multi-key operations are always a very difficult problem to solve. The redis cluster solution is as follows:
1. Multi-key operations are not supported
2. If you must use multiple keys, make sure that all keys are on the same node. The specific method is to use the "hash tag" scheme.
The hash tag scheme is an exception to data distribution.


Reference:
Redis cluster tutorial
Redis cluster Specification

Redis cluster (redis 3.x) design highlights