Redis builds distributed locks and redis builds

Redis builds distributed locks and redis builds

1. Preface

Why should we build a lock? Because building a proper lock can maintain data consistency in high concurrency, that is, the data locked when the client executes a coherent command is not changed by other clients and an error occurs. It also ensures the success rate of command execution.

Here you can't help wondering if there are transaction operations in redis? Cannot transaction operations implement the above functions?

Indeed, redis transactions can watch to monitor data, so as to ensure data consistency during consistent execution, but we must be aware that, when multiple clients process the same data at the same time, it is easy to cause transaction execution to fail, or even data errors.

In a relational database, the user first sends BEGIN to the database server, and then performs consistent write and read operations. Finally, the user can choose to send COMMIT to confirm the previous modification, or send a ROLLBACK to perform ROLLBACK.

In redis, the special command MULTI is the start, and then the user passes in a coherent command, and the last EXEC is the end (watch can be used to monitor some keys in this process ). For further analysis, commands in redis transactions will be pushed into the queue first, and commands will not be executed until the EXEC command appears. If the key of the watch monitoring changes, the transaction will fail. This means that there is no lock in the Redis transaction, and other clients can modify the data in the transaction being executed, which is why there are often errors when multiple clients process the same data at the same time.

2. simple understanding of Single-thread IO multiplexing in redis

Redis uses a single-thread IO multiplexing model to implement high-memory data services. What is single-thread I/O multiplexing? From the literal meaning, we can know that redis uses a single thread and multiple I/O operations. The whole process is simply to say, the data flow of which command is executed first when it arrives.

See the image below to understand the figure: a narrow bridge is shown in the figure. Only one vehicle can pass through. On the left is the vehicle access channel. That is, the IO stream is processed first when it first arrives.

In Linux, network I/O uses socket Sockets for communication. The general I/O model can only listen to one socket, while IO multiplexing can monitor multiple sockets at the same time. IO multiplexing avoids blocking on IO. A single thread saves the status of multiple sockets and then processes the rear wheel.

3. Concurrent Testing

We will simulate a simple and typical concurrency test, get problems from this test, and then study it further.

Concurrent test ideas:

1. Set a string count in redis, use the program to extract it and add it to + 1, and store it back. It has been circulating for 100,000 times.

2. Execute this code on both browsers simultaneously

3. Obtain count and view the result.

Test procedure:

1. Create the test. php file.

<?php$redis=new Redis();$redis->connect('192.168.95.11','6379');for ($i=0; $i < 100000; $i++) {  $count=$redis->get('count'); $count=$count+1; $redis->set('count',$count); }echo "this OK";?>

2. Access the test. php file in two browsers respectively.

We can see from the results that the count should have been 0.2 million only twice in total, but in fact the count is equal to more than 0.13 million, far less than 0.2 million. Why?

According to the previous content, redis adopts a single-thread IO multiplexing model. Therefore, we use two browsers, that is, two sessions (A and B). Removing, adding, and saving these three commands is not an atomic operation, in addition, when the two redis commands are executed, the client will first execute them first.

For example:

1. count = 120

2. A retrieves count = 120, followed by B's FETCH Command, and also retrieves count = 120.

3. Add 1 immediately after A is taken out, and save count = 121 back.

4. At this time, B is also followed, and count = 121 is saved.

Note:

1. Set the number of cycles to be as large as possible. If the number of cycles is too small, when the first browser is executed, the second browser is not started yet.

2. Both browsers must be executed simultaneously. If you execute the test. php file twice in a browser at the same time, whether or not the file is executed at the same time, the final result is count = 200000. Because the execution in the same browser belongs to the same session (all commands are passed through the same channel), redis will complete the first 100,000 executions, then execute the other 100,000 times.

4. Transaction solution and atomic operation Solution

4.1. Transaction Solution

The changed test. php file

<? Phpheader ("content-type: text/html; charset = utf8;"); $ start = time (); $ redis = new Redis (); $ redis-> connect ('2017. 168.95.11 ', '123'); for ($ I = 0; $ I <6379; $ I ++) {$ redis-> multi (); $ count = $ redis-> get ('Count'); $ count = $ count + 1; $ redis-> set ('Count', $ count ); $ redis-> exec () ;}$ end = time (); echo "this OK <br/>"; echo "execution time :". ($ end-$ start);?>

The execution result fails, and the table name cannot be resolved using transactions.

Cause analysis:

We all know that when redis is enabled, the commands in the transaction are not executed, but the commands are first pushed into the queue, and then when the exec command appears, in a blocking manner, all commands are executed one by one.

Therefore, when using the Redis class in PHP for redis transactions, all commands related to redis will not be actually executed, but will only be sent to redis for storage.

Therefore, the $ count in the circle is actually not the data we want, but an object. Therefore, 11 rows in test. php are incorrect.

View object count:

4.2 atomic operation incr Solution

# Update the test. php file

<? Phpheader ("content-type: text/html; charset = utf8;"); $ start = time (); $ redis = new Redis (); $ redis-> connect ('2017. 168.95.11 ', '123'); for ($ I = 0; $ I <6379; $ I ++) {$ count = $ redis-> incr ('Count');} $ end = time (); echo "this OK <br/>"; echo "execution time: ". ($ end-$ start);?>

The two browsers run simultaneously, which takes 14 or 15 seconds and count = 200000 to solve this problem.

Disadvantages:

It only solves the problem of removing and adding 1 here. In essence, it still fails to solve the problem. In the actual environment, what we need to do is a series of operations, not just pulling and adding 1, therefore, it is necessary to build a 10 thousand lock.

5. Build distributed locks

The purpose of constructing a lock is to eliminate the competition of choice and maintain data consistency under high concurrency.

When constructing a lock, we need to pay attention to the following issues:

1. prevent the process from crashing when the lock is held, resulting in a deadlock. Other processes will not be able to get the lock.

2. The lock held by the lock process is automatically released due to a long operation time, but the process itself does not know, And the locks of other processes are released incorrectly.

3. After a process lock expires, multiple other processes attempt to obtain the lock at the same time, and the lock is successfully obtained.

We will not modify the test. php file, but directly create a relatively standard object-oriented Lock. class. php file.

# Create a Lock. class and PHP File

<? Php # Distributed Lock class Lock {private $ redis = ''; # store redis object/*** @ desc constructor ** @ param $ host string | redis host * @ param $ port int | port */public function _ construct ($ host, $ port = 6379) {$ this-> redis = new Redis (); $ this-> redis-> connect ($ host, $ port );} /*** @ desc lock method ** @ param $ lockName string | lock name * @ param $ timeout int | lock expiration time ** @ return success return identifier/failure return false */public function getLock ($ lockName, $ ti Meout = 2) {$ identifier = uniqid (); # obtain the unique identifier $ timeout = ceil ($ timeout); # Make sure it is an integer $ end = time () + $ timeout; while (time () <$ end) # loop lock acquisition {if ($ this-> redis-> setnx ($ lockName, $ identifier )) # Check whether $ lockName is locked {$ this-> redis-> expire ($ lockName, $ timeout); # Set the expiration time for $ lockName to prevent deadlock return $ identifier; # returns a one-dimensional identifier} elseif ($ this-> redis-> ttl ($ lockName) ===- 1) {$ this-> redis-> expire ($ lockName, $ timeout); # Check whether an expiration time has been set. If no expiration time is set, add (assume that User A failed to set the time in the previous step, and the process crashed. Client B can detect and set the time)} usleep (0.001); # Stop 0.001 ms} return false ;} /*** @ desc release lock ** @ param $ lockName string | lock name * @ param $ identifier string | unique lock value ** @ param bool */public function releaseLock ($ lockName, $ identifier) {if ($ this-> redis-> get ($ lockName) ==$ identifier) # Check whether the lock has been modified by other clients {$ this-> redis-> multi (); $ this-> redis-> del ($ lockName ); # Release the lock $ this-> redis-> exec (); return true ;} Else {return false; # other clients have modified the lock, cannot delete others' locks}/*** @ desc test ** @ param $ lockName string | lock name */public function test ($ lockName) {$ start = time (); for ($ I = 0; $ I <10000; $ I ++) {$ identifier = $ this-> getLock ($ lockName ); if ($ identifier) {$ count = $ this-> redis-> get ('Count'); $ count = $ count + 1; $ this-> redis-> set ('Count', $ count); $ this-> releaseLock ($ lockName, $ identifier) ;}}$ end = time (); echo "this OK <br/> "; Echo "execution time :". ($ end-$ start) ;}} header ("content-type: text/html; charset = utf8;"); $ obj = new Lock ('2017. 192. 168.95.11 '); $ obj-> test ('lock _ count');?>

Test results:

In two different browsers, the final result is count = 200000, but it takes a relatively large amount of time and takes about 80 seconds. However, in high concurrency, it is acceptable to execute the lock release operation for 0.2 million locks on the same data, or even quite good.

The above simple example is just to simulate concurrent testing and test. In fact, we can use the Lock in Lock. class. php and modify it in combination with our own project to use the Lock. For example, the shopping mall's crazy flash sales, the virtual mall players in the game buy and sell things, and so on.

The above is all the content of this article. I hope this article will help you in your study or work. I also hope to provide more support to the customer's home!