Intimate contact redis-The third day (Redis's load Balance)

Objective

The last two days described the basic building of Redis and the deployment of HA-based clusters and related strategies and points of attention. Today began to tell the cluster features of Redis, and this piece of information on the web is not too full, even if there are 1, 2 is only a single-spoke service side of the building also did not mention the relevant client how and Redis cluster between the call problem.

The Redis cluster we're going to talk about today is the real load Balance, unlike Sentinel, Sentinel, which is also called a cluster, but it's an HA strategy that is high available or popular as a "disaster transfer" strategy.

Load Balance introduced in the redis3.x

The load balance functionality has been supported since redis3.x.

Redis Cluster is a redis cluster implementation, built-in data auto-sharding mechanism, the cluster internally maps all the keys into 16,384 slots, each Redis instance in the cluster is responsible for reading and writing the slots in one of these. The cluster client connects to any Redis instance in the cluster to send commands, and when Redis instance receives a request for its own non-responsible slot, it returns the Redis instance address in the slot where the key is requested to the client. After receiving the client, the original request is automatically re-sent to this address, externally transparent. Exactly which slot a key belongs to is determined by CRC16 (key)% 16384. In Redis cluster, the support for load balancing and HA correlation has been quite complete.

• Load Balancer: Data can be migrated between Redis instance in the cluster, in slots, but not automatically, requiring external command triggering. Balance
• Cluster member Management: Cluster nodes (Redis Instance) and nodes 22 periodically exchange intra-cluster node information and update, from the point of view of the sending node, this information includes: What nodes in the cluster, IP and port is what, the node name is what, the state of the node (such as OK, Pfail,fail, which is detailed later), including the node role (master or slave), and so on.

With regard to availability, the cluster consists of n sets of master-slave Redis instance.

The master may not be available from, but not from, the slot read and write service that means that the primary outage is responsible.

One master can have multiple from, main down, one from being promoted to the other, which is promoted primarily, and the protocol is similar to raft, see here. How do I detect a primary outage? The Redis cluster uses the quorum+ heartbeat mechanism. From the node point of view, the node will periodically send to all other nodes ping,cluster-node-timeout (configurable, second-level) time did not receive the other party's reply, then unilaterally considered the end node down, the node is marked as Pfail state. Through the exchange of information between the nodes collected to quorum nodes that this node is Pfail, the node is marked as fail, and it is sent to all other nodes, all other nodes received immediately after the node was considered down. From here, it can be seen that, after the main outage, at least cluster-node-timeout time, the master's responsible slot read and write service is not available.

The features of the Redis cluster are as follows:

• Node Auto discovery
• Slave->master elections, cluster fault tolerance
• Hot resharding: Online sharding
• Cluster Management: CLUSTERXXX
• Configuration-based (nodes-port.conf) cluster management
• ASK steering/moved Steering mechanism
• The deployment does not require the designation of master
• can support clusters of more than 1,000 nodes

Implementation of Redis Cluster

Redis Cluster relies on its official REDIS-TRIB.RB file in the Redis compilation package (which we use in the redis-stable version)/src directory, which is a Ruby script, so you have to prepare your server environment in advance.

Preparing for the environment before building cluster

• Installation of CentOS or rhe
• When installing Linux, it is necessary to remember to install the GCC library, LibC, libstdc++, Rubby Library (1.9.2 or above), zlib library (1.2.6 or above), if you do not install these "optional package" can be through Yum Install GCC commands for installation of additional packages after Linux installation in Linux networking scenarios
  
• Install Ruby Gems Library (1.8.16 or above)-rubygems-1.8.16.tgz (Run intrinsic setup.rb), download rubygems-1.8.16.tgz run after unpacking the/path/gem

• After the Ruby Gems installation, you must install the gem's Redis module, which can be downloaded via the official website or the associated trusted connection (download. gem file and then install with gem) such as:

Gem Install-l Redis-3.2.1.gem

Planning for Redis cluster nodes

The minimum requirement for a continuous build of at least 6 Nodes,redis cluster with a command such as make Prefix=/usr/local/redis1 install is (3 master,3 slave), which is in our " Intimate contact redis-The first day is described in detail how to compile and install a redis, all in this version of Redis-stable.

Configuration of cluster nodes

Each node is removed when it is first configured:

• . pid Files and Paths
• /LogFile Files and Paths
• Data storage Path
• Port

The rest of the configuration, exactly the same, and no master and slave.

Port Ports Configuration essentials:

All master (for example, 3 master port numbers are incremented in +1 ways: 7001,7002,7003)

The port number of all slaver must conform to the principle that the port number of the slave is greater than the associated master 1000th number, such as 7001 of the slave port numbers 8001.


For example:

3 Master is 7001,7002,7003, our 3 slave is 8001,8002,8003

In redis.conf file:

• Maxmemory-policy ALLKEYS-LRU
• cluster-enabled Yes (enable, comment out the previous #)
• Cluster-config-file nodes-7001.conf (Enable, put the previous # comment out, here is recommended to use the port number of this node as the file name, so as to facilitate the distinction between
• Cluster-node-timeout 15000 (Enable, put the previous # comment off)
• Cluster-migration-barrier 1 (Enable, put the previous # comment off)

Creating a cluster using the Rubb Gem's Redis module +redis-trib.rb

When everything is ready, start up all 6 Redis nodes.

REDIS-TRIB.RB is located in the/SRC directory of the downloaded redis-stable directory.

Create a Redis cluster using the following command:

./redis-trib.rb Create--replicas 1 192.168.0.1:7001 192.168.0.1:7002 192.168.0.1:7003 192.168.0.1:8001 192.168.0.1:8002 192.168.0.1:8003

Note:

After all the master is listed in the command line, list all slave (slave in the Order of Master), and all 6 nodes are listed on the command line.

After the command has been executed for a period of time the system will prompt you with the following information

When the cluster is created successfully, the following information is displayed:

This means that the Redis cluster has already been created, so you can start each node with a command such as Redis-server redis.conf.

Using Jedis client to connect to a Redis cluster (Load Balance)

The current combination of Redis and spring is based on the redistemplate of Jedis and spring data, and now spring data seems to support the functionality of the Redis cluster is not good (this refers to the load Balance functionality), we have encapsulated a Redis client to support Redis's cluster approach, and also with spring (if Redistemplate has supported Redis's load The balance feature recommends that you use the Redis template for spring data, because that will be better in terms of encapsulation or performance.

First, give the relevant jar package that the code needs.

Pom.xml

<!--Redis start--><dependency><groupid>redis.clients</groupid><artifactid>jedis </artifactId><version>2.8.0</version></dependency><dependency><groupId> Org.redisson</groupid><artifactid>redisson</artifactid><version>1.0.2</version> </dependency><!--Redis End-to-

Spring configuration file Redis-cluster.xml

<?xml version= "1.0" encoding= "UTF-8"? ><beans xmlns= "Http://www.springframework.org/schema/beans" xmlns: Xsi= "Http://www.w3.org/2001/XMLSchema-instance" xmlns:p= "http://www.springframework.org/schema/p" xmlns:context= "Http://www.springframework.org/schema/context" xmlns:jee= "Http://www.springframework.org/schema/jee" xmlns:tx= " Http://www.springframework.org/schema/tx "xmlns:aop=" HTTP://WWW.SPRINGFRAMEWORK.ORG/SCHEMA/AOP "xsi: schemalocation= "Http://www.springframework.org/schema/beans Http://www.springframework.org/schema/beans/spri Ng-beans.xsd Http://www.springframework.org/schema/context Http://www.springframework.org/schema/context/spri Ng-context.xsd "><context:property-placeholder location=" Classpath:/spring/redis.properties "/>< Context:component-scan base-package= "Org.sky.redis" ></context:component-scan><bean name= " Genericobjectpoolconfig "class=" Org.apache.commons.pool2.impl.GenericObjectPoolConfig "><property name=" MaxwaitmIllis "value="-1 "/><property name=" Maxtotal "value=" + "/><property name=" Minidle "value=" 8 "/>< Property Name= "Maxidle" value= "/></bean><bean id=" Jediscluster "class=" Com.qf.platform.redis.cluster.JedisClusterFactory "><property name=" Addressconfig "><value> Classpath:/spring/redis.properties</value></property><property name= "AddressKeyPrefix" value= " Clusternode "/> <!--the key prefix in the properties file--><property name=" timeout "value=" 300000 "/><property name=" maxred Irections "value=" 6 "/><property name=" Genericobjectpoolconfig "ref=" Genericobjectpoolconfig "/></bean ><bean id= "CustomExceptionHandler" class= "sample.  Myhandlerexceptionresolver "/></beans>

Here we see a class called Com.qf.platform.redis.cluster.JedisClusterFactory, and this class is what we encapsulate with "genericobjectpoolconfig" to support Redis The client interface for the cluster feature.

Com.qf.platform.redis.cluster.JedisClusterFactory class Content

/** * @Title: [Jedisclusterfactory.java] * @Package: [com.qf.platform.redis.cluster] * @author: [Mingkaiyuan] * @Creat EDate: [March 9, 2016 PM 3:42:26] * @UpdateUser: [Mingkaiyuan] * @UpdateDate: [March 9, 2016 PM 3:42:26] * @UpdateRemark: [ Description of this modification] * @Description: [TODO (describe the file in a word)] * @version: [V1.0] */package Com.qf.platform.redis.cluster;import Java.util.hashset;import Java.util.properties;import Java.util.set;import Java.util.regex.pattern;import Org.apache.commons.pool2.impl.genericobjectpoolconfig;import Org.springframework.beans.factory.FactoryBean; Import Org.springframework.beans.factory.initializingbean;import Org.springframework.core.io.resource;import Redis.clients.jedis.hostandport;import redis.clients.jedis.jediscluster;/** * @ClassName: Jedisclusterfactory * @ Author: [Mingkaiyuan] * @CreateDate: [March 9, 2016 PM 3:42:26] * @UpdateUser: [Mingkaiyuan] * @UpdateDate: [March 9, 2016 PM 3:42:26] * @UpdateRemark: [Instructions for this modification] * @Description: [TODO (describe the file in a word)] * @verSion: [V1.0] */public class Jedisclusterfactory implements Factorybean<jediscluster>, Initializingbean {private Resource addressconfig;private string addresskeyprefix;public void Setaddresskeyprefix (string addresskeyprefix) { This.addresskeyprefix = Addresskeyprefix;} Private jediscluster jediscluster;private integer timeout;private integer maxredirections;private Genericobjectpoolconfig genericobjectpoolconfig;private Pattern p = pattern.compile ("^.+[:]\\d{1,5}\\s*$"); Overridepublic Jediscluster GetObject () throws Exception {return jediscluster;} @Overridepublic class<? Extends Jediscluster> Getobjecttype () {return (This.jediscluster! = null? This.jedisCluster.getClass (): Jediscluster.class);} @Overridepublic Boolean Issingleton () {return true;} Private Set
Through the configuration in this class and spring


We learned that the class would read the information for each Redis cluster node through the redis.properties in the/spring directory in the/src/main/resources directory

Redis.properties File Contents
# Redis settingscach.host.ip=192.168.0.101cach.host.port=6380  redis.host.ip=192.168.0.101redis.host.port= 6379  redis.maxtotal=1000  Redis.maxidle=100redis.maxwait=2000redis.testonborrow=falseredis.testonreturn =trueredis.sentinel.addr=172.30.32.127:26379clusternode1=192.168.0.101:7001clusternode2= 192.168.0.101:7002clusternode3=192.168.0.101:7003clusternode4=192.168.0.101:8001clusternode5= 192.168.0.101:8002clusternode6=192.168.0.101:8003

This class is easy to use, see the sample code below

Client calls to Redis cluster sample code
@AutowiredJedisCluster Jediscluster;......jediscluster.set (key, value);

It's simple.

Considerations for Redis clusters (with pits)
REDIS-CLI Client Usage Issues

 
 

  
  
 
   
Use REDIS-CLI to connect cluster when using: redis-cli–c–h–p such a format
 
   
After the client connects to any Redis node set a value, the value does not necessarily exist in the node, but instead automatically turns to a node that is elected by the Redis cluster to coexist in the value, at which point the client's physical connection is also pointed to the node. Example:
 
  
 
 



At this point your REDIS-CLI connection has been redirected to 7002 instead of 7001, although the display > front is 7001

Use REDIS-CLI to connect any of the nodes in the cluster, such as:


You can see elements without key=1 within a 7003 node, but there are key=1 elements within the cluster, so:



REDS-CLI dynamically detects the node containing the Key=1 from the cluster, redirects the client connection to that node and invokes the value bound by the key

Add new Master Node

 
 

  
  
 
   
Add a Master node: Create an empty node, and then move some slots to this empty node, a process that requires manual intervention at this time
 
   
establish_config.sh generate configuration files based on port establish_config.sh 6386?> conf/redis-6386.conf
 
   
Start the node
 
   
Add an empty node to the cluster redis-trib.rb Add-node newip:newport already have any IP in the cluster: Any port in the existing cluster
 
  
 
 
Note:

The new node does not contain any data because it does not contain any slots. The newly added add-on is a master node that will  not be selected when the cluster needs to upgrade one of the nodes from a node to a new master node, while the new master node does not contain any slots and does not participate in the election and failover.



 
 

  
  
 
   
Assigning slot:redis-trib.rb Reshard ip:port to new nodes
 
  
 
 

Add slave node

 
 

  
  
 
   
Follow the first 3 steps to add the master node
 
   
REDIS-CLI Connect the new node shell, enter the command: Cluster replicate corresponds to the Node-id of the master
 
  
 
 


Note:

When adding slave online, it is necessary to bgsave the entire master data, pass it to slave, and then load the Rdb file into memory by slave, and consume master large amount of memory and network IO in the process of RDB generation and transfer, so it is not recommended that single instance memory is too large and the line is carefully operated.

Delete Master node

 
 

  
  
 
   
Before deleting the master node, first use Reshard to remove all slots from the current master to the new master
 
   
Delete the empty master node.
 
  
 
 




Some pits for Jedis (Redis Java Client)

 
 

  
  
 
   
Cluster slave default does not accept any read and write operations, after slave execute readonly command, read operations can be performed
 
   
The client side does not support multi-key operations (Mget,mset, etc.), but when the keys collection corresponds to a slot that supports mget operation
 
   
Multiple databases are not supported, only one db,select 0.
 
   
Jediscluster no API for byte[], need to expand (Jedis-3.0.0 and above to support byte[] operation)
 
  
 
 
As of today, all the construction and use of Redis is complete, there is more in-depth function by its performance optimization, configuration also needs everyone in peacetime work constantly practice, I can only relate to the text of about 50% of the content, more or rely on the reader himself to explore.

To be fair, Redis is a pretty good nosql, and most websites use it as the main choice for NoSQL or cache.

Use, everyone will!

But with the fine, it is necessary to pay a certain degree of research spirit.

Thank you!

Intimate contact redis-The third day (Redis's load Balance)