Detailed steps for building a Redis cluster, tutorials and troubleshooting, and detailed steps for redis

Detailed steps for building a Redis cluster, tutorials and troubleshooting, and detailed steps for redis

Redis cluster construction Guide

First chicken or egg?

Recently, a friend asked me a question about whether to go to a big city or a small city after graduation? Going to a big company or a small company? My answers are all big cities! Big company!

Why do you think about this? No matter how many girls and boys look for friends, they like to find a girl with a big chest. In the same way, "big" is always great.

Of course, if you want to be able to find a big man, it will be more perfect.

Redis cluster Introduction

Redis is an open-source key-value storage system. Thanks to its outstanding performance, most Internet companies use it for server-side caching. Redis versions earlier than 3.0 only support single-instance mode. Although it supports master-slave mode and guard mode deployment to solve single point of failure, Internet enterprises are now prone to hundreds of GB of data, it is completely unable to meet the business needs, so Redis launched the cluster mode after version 3.0.

Redis clusters adopt the P2P mode and are completely decentralized. Redis divides all keys into 16384 slots, and each Redis instance is responsible for part of the slots. All information in the cluster (nodes, ports, slots, etc.) is updated through regular data exchanges between nodes.

The Redis client can send a request in any Redis instance. If the required data is not in the instance, use the redirection command to direct the client to access the desired instance.

Build a cluster at will

Installing and deploying any application is actually easy. You only need to install the application step by step. The following describes how to build a Redis cluster. Because the cluster requires at least six nodes (three master nodes and three slave nodes), there are not so many machines to play, I cannot afford so many local virtual machines (computers are too bad). Now I plan to simulate a cluster on one machine. Of course, this is essentially different from the cluster construction in the production environment.

Now, I will start building a cluster under CentOS that has already installed Redis. If you are not very clear about how to install Redis in Linux, you can refer to this Article "learn about Redis and install and configure it under CentOS". Note that all the following cluster environments are based on the installed Redis.

1. Create a folder

We plan to set the port number of the Redis node in the cluster to 9001-9006. The port number is the instance folder under the cluster. Data is stored in the port number/data folder.

 

 

Mkdir/usr/local/redis-cluster

Cd redis-cluster/

Mkdir-p 9001/data 9002/data 9003/data 9004/data 9005/data 9006/data

2. Copy the execution script

Create a bin folder under/usr/local/redis-cluster to store the cluster running script and copy the running script under the src path of the installed Redis. View the command:

Mkdir redis-cluster/bin

Cd/usr/local/redis/src

Cp mkreleasehdr. sh redis-benchmark redis-check-aof redis-check-dump redis-cli redis-server redis-trib.rb/usr/local/redis-cluster/bin

3. Copy a new Redis instance

Now we copy a new instance from the installed Redis to the 9001 folder and modify the redis. conf configuration.

 

 

Cp/usr/local/redis/*/usr/local/redis-cluster/9001

Note: The only difference between modifying the redis. conf configuration and single-point configuration is that the rest are the general items:

Port 9001 (port Number of each node)

Daemonize yes

Bind 192.168.119.131 (bind the IP address of the current machine)

Dir/usr/local/redis-cluster/9001/data/(data file storage location)

Pidfile/var/run/redis_9001.pid (pid 9001 corresponds to port)

Cluster-enabled yes (cluster Startup Mode)

Cluster-config-file nodes9001.conf (9001 corresponds to the port)

Cluster-node-timeout 15000

Appendonly yes

The focus of cluster construction configuration is to cancel the comments of the three configurations:

 

 

4. Copy five new Redis instances.

We have already completed a node. In fact, the next step is to mechanically complete the other five nodes. In fact, we can do this: copy the 9001 instance to the other five folders, the only thing to modify is redis. all the information related to the port in conf is enough. In fact, there are four locations. Start the operation. See the figure below:

 

 

\ Cp-rf/usr/local/redis-cluster/9001/*/usr/local/redis-cluster/9002

\ Cp-rf/usr/local/redis-cluster/9001/*/usr/local/redis-cluster/9003

\ Cp-rf/usr/local/redis-cluster/9001/*/usr/local/redis-cluster/9004

\ Cp-rf/usr/local/redis-cluster/9001/*/usr/local/redis-cluster/9005

\ Cp-rf/usr/local/redis-cluster/9001/*/usr/local/redis-cluster/9006

The \ cp-rf command does not use aliases for replication. Because cp is actually the alias cp-I, interactive confirmation will be performed during the operation, which is annoying.

5. Modify the redis. conf file 9002-9006

In fact, it is very simple. You will find that there are only four points to be modified through the search. Let's replace them globally, enter the corresponding node folder, and replace them. The command is very simple, as shown in the figure:

 

 

Vim redis. conf

: % S/9001/9002g

After you press enter, there will be a prompt that the replacement is successful in a few places. Do not worry, you can manually check it:

 

 

In fact, we replaced the following four rows:

Port 9002

Dir/usr/local/redis-cluster/9002/data/

Cluster-config-file nodes-9002.conf

Pidfile/var/run/redis_9002.pid

Now, we have finished the most basic environment, and the next step is to start it.

In fact, we have almost built

1. Start 9001-9006 six nodes

Take a look at the figure:

 

 

/Usr/local/bin/redis-server/usr/local/redis-cluster/9001/redis. conf

/Usr/local/bin/redis-server/usr/local/redis-cluster/9002/redis. conf

/Usr/local/bin/redis-server/usr/local/redis-cluster/9003/redis. conf

/Usr/local/bin/redis-server/usr/local/redis-cluster/9004/redis. conf

/Usr/local/bin/redis-server/usr/local/redis-cluster/9005/redis. conf

/Usr/local/bin/redis-server/usr/local/redis-cluster/9006/redis. conf

Check whether the instance is successfully started: ps-el | grep redis

As you can see, all the six nodes have been successfully started. (If you find that the bind cannot be started, you can try to remove the bind configuration)

2. Test A node.

/Usr/local/redis-cluster/bin/redis-cli-h 192.168.119.131-p 9001

Set name mafly

 

 

The connection is successful, but it seems that an error is reported ???

(Error) CLUSTERDOWN Hash slot not served (the Hash slot of the cluster is not provided). What is this?

This is because although we have configured and started the Redis cluster service, they are not in a single cluster yet, and they cannot find each other directly, and there is no storage location, it is the so-called slot ).

3. install the software required by the cluster

Because Redis clusters require ruby commands, We need to install ruby and related interfaces.

Yum install ruby

Yum install rubygems

Gem install redis

 

 

This is the real cluster creation

Run the following command:

/Usr/local/redis-cluster/bin/redis-trib.rb create -- replicas 1 192.168.119.131: 9001 192.168.119.131: 9002 192.168.119.131: 9003 192.168.119.131: 9004 192.168.119.131: 9005 192.168.119.131: 9006

 

 

A brief explanation of this command: Call the ruby command to create a cluster. -- replicas 1 indicates that the master-slave replication ratio is, that is, one master node corresponds to one slave node. Then, by default, each master node and corresponding slave node service and solt size are allocated to us. Because there are only 16383 solt in the Redis cluster, we will be allocated evenly by default, of course, you can specify that the nodes can be re-allocated in the future.

M: 10222dee93f6a1700ede9f5424fccd6be0b2fb73 master node Id

S: 9ce697e49f47fec47b3dc290042f3cc141ce5aeb 192.168.119.131: 9004 replicates 10222dee93f6a1700ede9f5424fccd6be0b2fb73 corresponds to the master node Id under the slave

Currently, 9001-9003 is the master node, and 9004-9006 is the slave node, and confirm with you whether you agree to this configuration. Enter yes to create the cluster.

 

 

It indicates that the cluster is successfully created !!!

Verify:

It is still connected through client commands. You can view the status and node information through cluster commands.

/Usr/local/redis-cluster/bin/redis-cli-c-h 192.168.119.131-p 9001

Cluster info

Cluster nodes

 

 

Through commands, you can see in detail the cluster information and the status of each node, master and slave information, as well as the number of connections and slot information. We have successfully built and deployed the Redis cluster!

Set a mafly:

You will find that when we set name mafly, the Redirected to slot information appears and is automatically connected to the 9002 node. This is also a data allocation feature of the cluster, which is not detailed here.

 

 

To sum up

This article on Redis cluster deployment and setup is really step by step. As long as you install my steps, you will be able to successfully build a Redis cluster, it can also be said that, apart from the tedious steps, there is almost no technical content. It is estimated that people who can read it will feel tired (to be honest, it is really tiring to write this article ).

The next step may be dynamic resizing, adding nodes, reducing nodes, and rescheduling the slot size. Of course, the most important thing is how to integrate with our programs, and how to make better use of the Redis cache cluster, these are the most important.

The typhoon was not very bad tonight. It seems that I haven't slept and woke up in a day for a long time...

Other problems:

Redis requires Ruby version-= 2.2.2

When executing gem install Redis, the following error occurs:

[Root @ localhost Redis] # gem install redis-4.0.0.gem

ERROR: Error installing redis-4.0.0.gem:

Redis requires Ruby version> = 2.2.2.

2.2.2 error. We found that Centos supports ruby to 2.0.0 by default, and the minimum required for installing redis on gem is 2.2.2.

The solution is to first install rvm and then upgrade the ruby version.

So

1. Install curl

Sudo yum install curl

2. Install rvm

Curl-L get. rvm. io | bash-s stable

3. source/usr/local/rvm/scripts/rvm

4. view known ruby versions in the rvm Library

Rvm list known

5. Install a ruby version.

Rvm install 2.4.0

6. Use a ruby version

Rvm use 2.4.0

7. Uninstall a known version

Rvm remove 2.0.0

8. view the version

Ruby -- version

9. Install redis again.

Gem install redis-4.0.0

Error curl-L get. rvm. io | bash-s stable

Failed to install RVM: public key not found

$ Gpg -- keyserverhkp: // keys.gnupg.net -- recv-keys409B6B1796C275462A1703113804BB82D39DC0E3

The key indicates the key in which the error message is returned.