Practical application of consistent hashing

Objective

Today re-read the consistency of the hash of the paper, there are several areas of unclear, to seek guidance

Scene

Four server servers (192.168.1.1-4), redis databases, storing Key-value key-value pairs

Question 1

First, Redis's Key-value data typically requires 3 copies, corresponding to a consistent hash scenario, which can be said to have a master server, and 2 slave servers. Question: Select three different servers from the server's consistent hash code, choose a server, and then add two slave servers to the server (so that the servers from the original 4, increased to 4 + 2 * 4 = 12 units), I'm thinking of using memcache to record key, master, and slave distribution tables.

Question 2

If redundant to the other two servers, assuming that the a\b\c\d four servers, Key's main library is a, backup library on the BC, that when a single point of failure, how to choose between BC, BC on a node for the deletion and modification of data how to restore to a node? I saw the NRW model, but I didn't fully understand it.

Answer 1

In the group, it is suggested that the server can use haproxy+keepalived to implement each machine master mode, in fact, the equivalent of a consistent hash of the ring really only 4 available target, but requires 8 servers to complete

Reply content:

Objective

Today re-read the consistency of the hash of the paper, there are several areas of unclear, to seek guidance

Scene

Four server servers (192.168.1.1-4), redis databases, storing Key-value key-value pairs

Question 1

First, Redis's Key-value data typically requires 3 copies, corresponding to a consistent hash scenario, which can be said to have a master server, and 2 slave servers. Question: Select three different servers from the server's consistent hash code, choose a server, and then add two slave servers to the server (so that the servers from the original 4, increased to 4 + 2 * 4 = 12 units), I'm thinking of using memcache to record key, master, and slave distribution tables.

Question 2

If redundant to the other two servers, assuming that the a\b\c\d four servers, Key's main library is a, backup library on the BC, that when a single point of failure, how to choose between BC, BC on a node for the deletion and modification of data how to restore to a node? I saw the NRW model, but I didn't fully understand it.

Answer 1

In the group, it is suggested that the server can use haproxy+keepalived to implement each machine master mode, in fact, the equivalent of a consistent hash of the ring really only 4 available target, but requires 8 servers to complete

According to the landlord want to use consistent hashing algorithm, basically is to use Redis as a cache, rather than a database;

The main purpose of using a consistent hash is usually to reduce the cache failure range and prevent "avalanche" when the number of machines in the cluster changes;

In this case, I think of another question : Is it always a hash consistency algorithm?

In my opinion, depending on some of the features that Redis currently offers, perhaps using the following common architecture will solve just the problem;

1 master,n Station Slave, then take the read and write separation, master processing write requests, Slave is responsible for processing read requests (load Balancing)

What are the advantages of this deployment approach?

• When business visits soar, we can quickly add a redis machine, connected to master, when the master-slave replication is completed, put into the slave cluster, and start to provide services;

• When a machine in the slave cluster is hung, it can be identified by a mechanism (for reference Sentinel) and kicked out of the slave cluster;

This way no matter how the slave cluster changes, there will be no cache failure or partial failure problem;

What are the disadvantages of this deployment approach?

• Redis Current Master-slave replication process has a very bad place is: when the slave and master in the synchronization process, the network problems, this time slave requires master retransmission instead of continuous (regenerate an RDB file and then transfer); So when the data in master is very large, The impact will be very large, so a master under the hang of multiple slave stability will decrease;

Above only for personal thoughts, take out and discuss with the landlord

These two days to see something of the architecture, this problem basically can be closed

Solution Solutions

1. or the use of 8 server, the primary and standby way, the use of a consistent hash algorithm to increase the node server can be virtual nodes and weights to prevent data migration, where the node server can use VIP to identify

2. The primary and standby server can adopt keepalived scheme, the external provides virtual IP, automatic failover, as for the primary and standby server database synchronization, you can use Redis provide the master-slave mechanism, you can also crontab+aof logs, You can then encapsulate a layer of insert or update on the REDIS database, provide API interface externally, need to access the interface when operating

