Redis Cluster in Redis Cluster

Redis Cluster in Redis Cluster
1. Linux System Configuration 1.1. vm. overcommit_memory settings

The overcommit_memory file specifies the kernel's memory allocation policy. The value can be 0, 1, or 2.

0 indicates that the kernel will check whether there is enough available memory for use by the process. If there is enough available memory, the memory application will be allowed; otherwise, the memory application will fail, and return the error to the application process.
1 indicates that the kernel allows all physical memory allocation regardless of the current memory status.
2. indicates that the kernel is allowed to allocate more memory than the total physical memory and swap space.

Jenkins_service @ jenkinsservice :~ /Redis-3.0.1 $ sudo sysctl vm. overcommit_memory = 1

Vm. overcommit_memory = 1

1.2. enabled Transparent Huge Pages

THP (Transparent Huge Pages) is an abstraction layer that automates the management of Huge Pages.

At present, THP will affect the performance of memory locks due to implementation problems. Especially when the program is not specially developed for large memory pages, it is briefly introduced as follows:

There is a process named khugepaged in the background of the operating system. It will always scan the memory occupied by all processes and switch 4kpage to Huge Pages whenever possible. In this process, various memory locks are required for various memory allocation activities, which directly affects the program's memory access performance. In addition, this process is transparent to the application and cannot be controlled at the application level, for a program specially optimized for 4 k pages, it may cause random performance degradation.

Redis Cluster command line

// Cluster)

1. Cluster info prints CLUSTER information
2. Cluster nodes lists all known NODES of the CLUSTER and their information.
4. // Node)
5. Cluster meet <ip> <port> adds the node specified by the ip address and port to the CLUSTER to make it part of the CLUSTER.
6. Cluster forget <node_id> Removes the node specified by node_id from the CLUSTER.
7. Cluster replicate <node_id> sets the current node as the slave node of the node specified by node_id.
8. Cluster saveconfig saves the node configuration file to the hard disk.
10. // Slot)
11. Cluster addslots <slot> [slot...] assigns one or more slots (assign) to the current node.
12. Cluster delslots <slot> [slot...] removes the assignment of one or more slots to the current node.
13. The cluster flushslots removes all slots assigned to the current node, turning the current node into a node without any slots.
14. Cluster setslot <slot> NODE <node_id> assigns the slot to the NODE specified by node_id. If the slot has been assigned to another NODE, let the other NODE Delete the slot first.>, and then proceed with the assignment.
15. Cluster setslot <slot> MIGRATING <node_id> migrates the slot of the current node to the node specified by node_id.
16. Cluster setslot <slot> IMPORTING <node_id> imports slot from the node specified by node_id to the current node.
17. Cluster setslot <slot> STABLE cancels the import or migrate operation on the slot ).
19. // Key)
20. Cluster keyslot <key> the slot in which the computing key should be placed.
21. Cluster countkeysinslot <slot> returns the number of key-value pairs currently contained in the slot.
22. Cluster getkeysinslot <slot> <count> returns the keys in the count slot.

 

2. Configuration File

Redis enabled for cluster must be an empty Server

Modify redis. conf to enable cluster

 

Set three clusters

10.24.6.7: 6379

10.24.6.4: 6379

10.24.6.6: 6379

3. view the status of the initial redis Cluster

Start the Redis server on the three nodes. At this time, all three Redis server nodes will start running in Redis Cluster mode, but the Cluster is not automatically built because the three are still in the "I don't know you, you do not belong to me. Each of them is a single Redis node, or a cluster that only contains one node. We can connect to the server through the Redis client to view their status. Figure 1 shows the status query method and query results. The cluster nodes command is used to view all the nodes in the Redis cluster to which the current Redis node belongs, cluster info is used to view the overall status of the Redis cluster to which the current Redis node belongs. As shown in the figure, the Redis cluster contains only one Redis node, that is, the current node. The status of the entire cluster is fail.

 

4. Allocate hash slot

Through the above operations, we have planned three independent Redis nodes to the same cluster. Have we completed all the work of cluster construction now? None! By checking the cluster status in Figure 2, we can know that the current cluster status is still fail. At this time, the Redis cluster does not work and cannot process any Redis commands. Why is the cluster status still fail? The author finds the cause by viewing the instructions in the official documentation. The source text is as follows:

The FAIL state for the cluster happens in two cases.

1) If at least one hash slot is not served as the node serving it currently is in FAIL state.

2) If we are not able to reach the majority of masters (that is, if the majorify of masters are simply in PFAIL state, it is enough for the node to enter FAIL mode ).

Obviously, the cause of the cluster being in the fail state is not the second one, that is, at least one hash slot is not served! Think about it a bit, no! Why is there a hash slot that has not been served? At the root, there is no Redis node serving any hash slot! As we all know, Redis Cluster uses hash slot to partition data based on the primary key, so a key-value data will be automatically mapped to a hash slot according to the algorithm, however, the Redis node on which a hash slot is stored is not automatically mapped and needs to be allocated by the Cluster Administrator. So how many hash slots do we need to allocate Redis nodes? According to the source code, we can see that there are 16384, that is, we need to allocate 16384 hash slots to three nodes in the cluster. There are many commands for allocating hash slots in Redis, including cluster addslots, cluster delslots, and cluster setslot. Since we are in the cluster initialization phase, we can select cluster addslots to allocate hash slots. the syntax of this command is cluster addslots slot1 [slot2]... [slotN].

4.1. nodes-6379.conf allocation

Each redis client allocates its own responsibility

The modification content is as follows: cda76a0a094d2ce624e33bed7f3c75689a4128fd: 0 myself, master-0 0 connected 0-5000 (note that it is the description on its own node, that is, the range of the hash slot appended to the row of myself ). Similarly, append 5001-10000 to the nodes-6379.conf file on Redis Cluster Node2, and append 10001-16383 to the nodes-6379.conf file on Redis Cluster Node3. After such configuration, Redis Cluster Node1 is responsible for storing all hash slots between 0 and 5000, and Redis Cluster Node2 is responsible for storing all hash slots between 5001 and 10000, redis Cluster Node3 stores all hash slots from 10001 to 16383.

 

4.2. Redis cluster command allocation

 

5. Build a Redis Cluster

To put it bluntly, how can we achieve this by connecting the three Redis nodes started earlier to each other and realizing their existence? The answer is the cluster meet command. This command is used to introduce the current node to another node. Figure 2 shows the execution method and effect of the cluster meet command, as shown in the figure, we use the cluster meet command to introduce Redis Cluster Node1 to Redis Cluster Node2 (the node IP address is 192.168.32.3 and the running port is 6379) and Redis Cluster Node3 (the node IP address is 192.168.32.4, the running port is 6379). Then we can check the cluster node and cluster status again. The three nodes have been successfully merged into the same cluster.

 

Three nodes are found here.

View the 10.24.6.6 Client

 

So far, three Nodes

 

Build complete ID:

 

After the cluster status is displayed as OK, we can execute the Redis command just like on the Redis single-host version.

 

6. Test 6.1. Non-cluster mode 6.1.1. Redis-cli

 

When a client in non-cluster mode jumps to a cluster, the system prompts a MOVED error.

6.1.2. Redis client library

FromRedis. sentinelImportSentinel
Sentinel = Sentinel ([('10. 24.6.7',26379)], Socket_timeout =0.1)
Master = sentinel. master_for ('10. 24.6.5master', Socket_timeout =0.1)
PrintMaster
Master. set ('Aaaaaaaaaaaaaaa','Bar')
Master. set ('Bbbbbbbbbbbbbbbbbbbbbb','Bar')
PrintMaster. get ('Aaaaaaaaaaaaaaa')
PrintMaster. get ('Bbbbbbbbbbbbbbbbbbbbbb')

Traceback (most recent call last ):

File "E:/HomeInternet/server/utest_workspace/utest_utils/utest_unit/_ init _. py", line 23, in <module>

Master. set ('bbbbbbbbbbbbbbbbbbbb', 'bar ')

File "build \ bdist. win32 \ egg \ redis \ client. py", line 1055, in set

File "build \ bdist. win32 \ egg \ redis \ client. py", line 565, in execute_command

File "build \ bdist. win32 \ egg \ redis \ client. py", line 577, in parse_response

File "build \ bdist. win32 \ egg \ redis \ sentinel. py", line 55, in read_response

File "build \ bdist. win32 \ egg \ redis \ connection. py", line 574, in read_response

Redis. exceptions. ResponseError: MOVED 9577 10.24.6.6: 6379

6.2. cluster mode (-C) 6.2.1. Redis-cli

 

In cluster mode, the client automatically redirects to the node

6.2.2. Redis Cluster

Https://github.com/Grokzen/redis-py-cluster

 

FromRedisclusterImportStrictRedisCluster
Startup_nodes = [{"Host":"10.24.6.7","Port":"6379"}]
Rc = StrictRedisCluster (startup_nodes = startup_nodes, decode_responses = True)
PrintRc. set ("Bbbbbbbbbbbbbbbbbb","Bar")
True
PrintRc. get ("Bbbbbbbbbbbbbbbbbb")

 

6.2.3. Redis Sentinel + Cluster

FromRedis. sentinelImportSentinel
FromRedisclusterImportStrictRedisCluster
Sentinel = Sentinel ([('10. 24.6.7',26379)], Socket_timeout =0.1)
Ip, port = sentinel. discover_master ('10. 24.6.5master')
Rc = StrictRedisCluster (host = ip, port = port, decode_responses = True)
PrintRc. set ("Bbbbbbbbbbbbbbbbbb","Bar")
PrintRc. get ("Bbbbbbbbbbbbbbbbbb")

You may also like the following articles about Redis. For details, refer:

Install and test Redis in Ubuntu 14.04

Basic configuration of Redis master-slave Replication

Redis cluster details

Install Redis in Ubuntu 12.10 (graphic explanation) + Jedis to connect to Redis

Redis series-installation, deployment, and maintenance

Install Redis in CentOS 6.3

Learning notes on Redis installation and deployment

Redis. conf

Redis details: click here
Redis: click here

This article permanently updates the link address: