Redis cluster details

Currently, Redis does not provide the cluster function. To provide services for multiple Redis instances at the same time, you can only implement it through the client itself (Memchached is also the client for distributed implementation ). According to the document, we can see that Redis is developing the cluster function. Some of them have been developed, but it is still unknown when it can be used. Source: http://redis.io/topics/cluster-spec

I. Introduction

This document describes the implementation details of the redis cluster in development. This document is divided into two parts: the first part has been implemented on the Code branch of the redis unstable version, and the other part also needs to be implemented. In the future, if the cluster implements design changes, these changes may be modified. However, the unimplemented part is more likely to be modified than the implemented part. This document includes various details required to implement the client, but the client author must note that these details may be modified.

Ii. What is a Redis cluster?

A cluster is a subset of independent servers for distributed and fault-tolerant implementations. No central node or proxy node exists in the cluster. One of the main purposes of the design is linear scalable expansion (I .e. adding or deleting nodes at will ). The cluster sacrifices fault tolerance to ensure data consistency. Therefore, when network and node faults occur, the system will try its best to ensure data consistency and effectiveness. (Here we think that node failure is a special case of network failure)

To solve the single point of failure (spof) problem, we need both masters and slaves. Even if the master and slave nodes are functionally consistent, they are even deployed on the same server, slave nodes are only used to replace the faulty master node (that is, the slave node is not used unless the master node fails and is used to replace the master node ). In fact, if the slave node does not have the need to read-after-write (write and read data immediately to avoid data unavailability during data synchronization), the slave node only accepts read-only operations.

Iii. Implemented subset

The cluster implements all single commands in non-distributed versions. Complex Multi-command operations, such as the intersection of set sets, have not yet been implemented. In general, theoretically, operations not on the same node will not be implemented.

In the future, a new Node type (computing Node) called "Computation Node" may be added, which is mainly used to process multi-key read-only operations in the cluster. However, read-only operations on the multi-key are not implemented by transmitting the cluster to the Computation Node for computing.

(Look at English: in the future it is possible that using the migrate copy command users will be able to use Computation Nodes to perform multi-key read only operations in the cluster, but it is not likely that the Redis Cluster itself will be able to perform complex multi key operations implementing some kind of transparent way to move keys around .)

The nodes in the cluster do not support multiple databases as the independent servers do. Only database 0 is supported and the select command is not supported.

4. client and server roles in the cluster Protocol

Nodes in the cluster are responsible for storing data, understanding the status of the cluster, and including ing the data key to the corresponding node. Any node in the cluster can automatically discover other nodes and faulty nodes. If necessary, the slave node can be upgraded to the master node.

Cluster nodes use TCP bus and binary protocol to interconnect and dispatch tasks. That is, each node uses TCP bus to connect to other nodes in the cluster. Each node uses the gossip protocol to send messages:

1) propagate information to discover new nodes,

2) send ping packets to other nodes in the cluster to check whether other nodes are working properly,

3) Sending cluster messages requires specific signal conditions

The cluster connection bus can also spread PUB/SUB messages between nodes.

When the node in the cluster cannot meet the client's request, the errors-MOVED and-ASK command may be used to notify the redirection to the available node. Theoretically, the client can send requests to any node in the cluster and receive a redirection response when necessary, that is, the client does not have to care about the cluster status. However, the client can cache the ing between the data keys and nodes to avoid redirection on the server side, which can improve the performance to a certain extent.

5. Distributed keyword Model

The key space is divided into 16384 slots (nodes). In fact, the maximum number of nodes in the cluster is 16384. The recommended maximum value is several hundred nodes. All master nodes process 4096 percentage slots. When the cluster is stable (that is, a slot is not being transferred to another node), a slot must be processed by only one node, and then a node can process multiple slots at the same time.

When tively setting an upper limit for the cluster size of 16384 nodes (however the suggested max size of nodes is in the order ~ 1000 nodes ).

The algorithm for ing key values to specified slot values is as follows:

HASH_SLOT = CRC16 (key) mod 4096.

The CRC16 algorithm is described in Note 1 of this document.

Using the 16-bit output of the CRC16 with a full 12 score, CRC16 can map various types of keys to a 4096 space in our test.

6. cluster node attributes

Each node in the cluster has a unique ID in the cluster. Its ID is a hexadecimal representation of a 160-bit random number. When a node is started for the first time, it obtains its ID and stores it in its configuration file. It will continue to use this ID until the configuration file is deleted by the system administrator.

The node ID is used as the node identification in the cluster. A node may modify the IP address or address but does not have to change the node name. The cluster can also detect node IP and PORT changes and then send change Protocol notifications through the connection bus.

Each node has some related information, which is known to other nodes:

1) IP address and port of the node

2) some flag Spaces

3) key-slot of the node service

4) the last ping time sent through the cluster connection Bus

5) The last time pong was received

6) Number of backup nodes of the node

7) if it is a backup node, the ID of the master node (if the node is the master node, the value is 0000000

The cluster nodes command can be used to obtain information about all NODES in the CLUSTER, including the master node and backup node.

The following is an example:

$ Redis-cli cluster nodes

D1861060fe6a534d42d8a19aeb36600e18785e04: 0 myself-0 1318428930 connected 0-1364

3886e65cc906bfd9b1f7e7bde4681_a052d1dae 127.0.0.1: 6380 master-1318428930 1318428931 connected 1365-2729

D289c575dcbc4bdd2931585fd4339089e461a27d 127.0.0.1: 6381 master-1318428931 1318428931 connected 2730-4095

The preceding information is displayed in sequence: ID, IP: PORT, FLAGS, and the time when the PING was last sent, the time when PONG was received, the connection status, and slots.

Redis details: click here
Redis: click here

Recommended reading:

Install Redis in Ubuntu 12.10 (graphic explanation) + Jedis to connect to Redis

Redis series-installation, deployment, and maintenance

Install Redis in CentOS 6.3

 

For more details, please continue to read the highlights on the next page:

1 2 3 Next Page