Java Jedis Operations Redis Example (iii)--SETNX/GETSET implementation of distributed locks __java

Reprint: http://www.cnblogs.com/0201zcr/p/5942748.html

Reprint: http://blog.csdn.net/fengshizty/article/details/53561562

Reprint: http://www.hollischuang.com/archives/1716

Reprint: http://www.cnblogs.com/zhongkaiuu/p/redisson.html

Reprint: https://yq.aliyun.com/articles/60663

Reprint: http://blog.csdn.net/josn_hao/article/details/78412694 a distributed lock

Because in peacetime work, the online server is a distributed multiple deployment, often face the problem of solving the data consistency under the distributed scene, then we should use the distributed lock to solve these problems.

The distributed Cap theory tells us that "no single distributed system can meet both consistency (consistency), availability (availability), and partition fault tolerance (Partition tolerance)at a maximum of two." "So, many systems at the beginning of the design of the three to make trade-offs." In the vast majority of the Internet scene, the need to sacrifice strong consistency in exchange for the high availability of the system, the system often only need to ensure "final consistency", as long as the final time is acceptable to the user within the range can be.
1. Optimistic lock and pessimistic lock

Pessimistic lock (pessimistic lock), as the name implies, is very pessimistic, every time to get the data are thought that others will be modified, so every time when the data will be locked, so that others want to take this data will block until it gets the lock. The traditional relational database inside the use of a lot of this locking mechanism, such as row locks, table locks, read locks, write locks , etc., are locked before doing the operation.

Optimistic lock (optimistic lock), as the name suggests, is very optimistic, every time to get the data are thought that others will not be modified, so will not be locked, but in the update will be judged in this period when others have to update this data, you can use the version number and other mechanisms. Optimistic locks are suitable for multiple-read application types, which can increase throughput, as if the database provides optimistic locks that are provided similar to the write_condition mechanism
2. Distributed Lock's general realization way

For the implementation of distributed locks, the following schemes are commonly used at present:

Implementing distributed lock based on database based on cache (Redis,memcached,tair) to realize distributed lock based on zookeeper

1 Implement distributed lock based on database

Distributed locks based on database implementations are implemented in the form of row locks (for update implementation, record Lock) and exclusive lock/table locks, which add unique identifier segments.

Implementation Principle

The easiest way to implement a distributed lock is to create a lock table directly and then manipulate the data in that table to implement it. When we want to lock a method or resource, we add a record to the table and delete the record when we want to release the lock. Create such a database table:

CREATE TABLE ' Methodlock ' (
  ' id ' int (one) not NULL auto_increment COMMENT ' primary key ',
  ' method_name ' varchar () NOT NULL DEFAULT ' COMMENT ' locked method name ',
  ' desc ' varchar (1024) NOT NULL DEFAULT ' Memo info ',
  ' update_time ' timestamp not null Defaul T current_timestamp on UPDATE current_timestamp COMMENT ' Save data time, automatically generate ',
  PRIMARY key (' id '),
  UNIQUE key ' Uidx_ Method_name ' (' method_name ') USING btree
) engine=innodb DEFAULT Charset=utf8 comment= ' method in lockdown ';

When we want to lock a method, execute the following sql:

Insert into Methodlock (METHOD_NAME,DESC) VALUES (' method_name ', ' desc ')

Because we have a unique constraint on method_name, where multiple requests are submitted to the database at the same time, the database guarantees that only one operation can succeed, then we can assume that the successful thread has obtained the lock of the method and can execute the method body content. When the method completes, you want to release the lock, you need to execute the following sql:

Delete from methodlock where method_name = ' method_name '

In addition to the operation of the data table can be added to the record, in fact, can also use the data from the lock to implement the distributed lock. Distributed locks can be implemented through exclusive locks on the database. MySQL based InnoDB engine, you can use the following methods to achieve the lock operation:,

public Boolean lock () {
    Connection.setautocommit (false) while
    (true) {
        try{result
            = select * FROM Methodlock where method_name=xxx for update;
            if (result==null) {return
                true;
            }
        } catch (Exception e) {

        } sleep
        (1000);
    }
    return false;
}

Add a for update after the query statement, and the database adds to the database table during the query Exclusive Lock(Here's one more word, InnoDB engine is locked, only row-level locks are used when retrieving through an index, otherwise you will use table-Level locks。 Here we want to use row-level locks, we need to add an index to method_name, it is noteworthy that this index must be created into Unique index, otherwise there will be problems where multiple overloaded methods cannot be accessed concurrently. It is recommended that the parameter type be added to the overloaded method. When an exclusive lock is added to a record, no other thread can increase the exclusive lock on that row record.
After you finish executing the method, unlock it by using the following methods:

public void Unlock () {
    connection.commit ();
}

Advantages

It is easy to understand by using database directly.

Disadvantage

There are all kinds of problems that will make the whole solution more and more complex in the process of solving the problem.
Operating a database requires a certain amount of overhead, and performance issues need to be considered. (Problem with SQL Timeout exception {frame layer Transaction Timeout/JDBC query timeout/socket read timeout})
The use of database row-level locks is not necessarily reliable, especially when our lock table is not large. 2 distributed lock based on cache implementation

memcached Lock:

Implementation Principle

Memcached with the Add function, the use of the characteristics of the Add function can implement distributed locks. The difference between add and set is that if you have a multithreaded concurrency set, each set succeeds, but the last stored value is the thread of the last set . Add, in contrast, adds the first arriving value and returns True, and subsequent additions return false. With this point, distributed locks can be implemented easily.
Advantages
Concurrent and efficient.
Disadvantages
(1) Memcached adopts the inclusion LRU substitution policy, so if there is not enough memory, the lock information in the cache may be lost.
(2) memcached cannot be persisted, once restarted, will cause information loss.

(3) It is not very reliable to control the expiration time of the lock by the time of timeout.

3 Implement distributed lock based on zookeeper Implementation principle:
Distributed lock based on zookeeper instantaneous ordered node, the idea is as follows: When each client adds a lock to a function, it generates a unique instantaneous ordered node in the directory of the specified node corresponding to the function on the zookeeper. The way to determine whether or not to acquire a lock is simple, just to determine the smallest ordinal number in an ordered node. When releasing the lock, simply remove the instantaneous node. At the same time, it can avoid the deadlock caused by the failure of service to release the lock, resulting in deadlock problems.
Advantages
Lock security is high, ZK can be persisted
Disadvantages
Performance overhead is relatively high. Because it needs to dynamically generate, destroy instantaneous node to achieve the lock function.
Realize
The distributed lock can be conveniently realized by using the Zookeeper third party library curator directly. The following is the ZK distributed lock core code based on the curator implementation:

3. Examples of applicable scenarios for distributed locks

Scene One: such as assigning task scenes. In this scenario, because it is the company's business background system, mainly for the audit staff audit work, the concurrent volume is not very high, and the task of the allocation of rules designed to be through the auditor each initiative to pull the request, and then the server from the task pool of random selection tasks to allocate. This scene you will feel more single, but the actual allocation process, due to the user clustering problem, so it is more complex than I described, but here to illustrate the problem, we can simplify the question. In the course of use, it is primarily to avoid problems that the same task was acquired by two reviewers at the same time. I finally used a distributed lock based on the database Resource table to solve the problem.

scene Two: such as the payment scene. In this scenario, I provide the user with three mobile phone numbers to protect the privacy of the user (these numbers are obtained from the operator, and the real phone number looks the same), let the user choose one to buy, after the user purchase payment, I need to assign the user's choice of the number of users to use, At the same time, will not choose to release. In this process, the number that is filtered to the user should be exclusive to the current user for a certain period of time (within the user's normal time range) to ensure that the payment is 100% available, and that resource flows are maintained due to the limited resources of the resource pool, That is, you cannot allow resources to be occupied by a user for a long time. For service design objectives, the first phase of the project on the line can at least support the peak QPS 300 request, while the design process to take into account the user experience problems. I finally used the memecahed Add () method and the distributed lock based on the database Resource table to solve the problem.

scene Three: I have a data service, the number of calls per day in 300 million, 86,400 seconds per day in the calculation of the QPS at about 4000, as the service of the daytime adjust the dosage is significantly higher than the evening, so the peak day afternoon QPS reached 6000, a total of 4 servers, A single Taiwan QPS can reach more than 3000. I finally used the problem of Redis setnx () and expire () distributed lock solutions.

scene Four: scene one and scene two of the upgraded version. In this scenario, payment is not involved. However, as resources are allocated in a single process, there is a need to maintain the increase in consistency, and the first phase of the design goal to achieve peak qps500, so we need to further optimize the scene. I ended up using the Redis setnx (), expire () and distributed locks based on database tables to solve the problem. reprint: http://www.cnblogs.com/PurpleDream/p/5559352.html two Redis distributed lock implementation principle: Setnx/getset

1) setnx (SET if not eXists)

Syntax:setnx key value
SETNX is shorthand for "set if not EXists" (set) if it does not exist, and its action is to set the value of the key to value, if and only if the key does not exist. If a given key already exists, SETNX does not do any action.
return value:
The setting succeeds and returns 1.
Setting failed, returns 0.

So we use the following command:

If 1 is returned, the client obtains the lock and sets the Lock.foo key value to the time value indicating that the key is locked and the client can finally release the lock through Del Lock.foo.
If you return 0 to indicate that the lock has been acquired by another client, we can return or retry the other party to complete or wait for the lock timeout.

2) Getset

Syntax:getset key value
Set the value of the given key to value and return the old value of the key . Returns an error when the key exists but is not a string type.
return value:
Returns the old value of the given key.
Returns nil when key does not have an old value , that is, when key does not exist.

3) Get
Syntax: Getkey
return value:
Returns nil when key does not exist, otherwise, returns the value of the key.
If the key is not a string type, then an error is returned

three Redis distributed lock Basic mode

Redis can generally use SETNX to implement distributed locks.

1. Get lock

public static void Lock (Jedis Jedis, String Lockkey, string requestid, int expiretime) {

    Long result = Jedis.setnx (loc Kkey, RequestID);
    if (result = = 1) {
        //If the program suddenly crashes, you cannot set the expiration time, the deadlock
        Jedis.expire (Lockkey, expiretime) will occur;
    }

2. Release lock

public static void Releaselock1 (Jedis Jedis, String lockkey) {
    Jedis.del (lockkey);
}

Setnx to create a key, if the key does not exist, create a successful return of 1, if the key already exists, return 0. According to the above to determine whether to acquire the lock. Get to the execution business logic of the lock, then delete the Lock_key to implement the release lock, the other not acquire the lock, then try to retry until you get the lock.

The above logic is ok under normal circumstances, but once the client that gets the lock is dead, and the above release lock is not performed, the other client cannot acquire the lock.

implementation of four Redis distributed lock

A distributed lock that is implemented in a simple way cannot release resources when the client is dropped, so there are 2 ways to resolve this situation:

Set an expiration time for Lock_key to determine whether the value of Lock_key expires in the first example, with the expiration time on the set key value, even if it hangs, the other clients will be able to compete for the lock after the expiration time.

public class Redistool {

    private static final String lock_success = "OK";
    private static final String set_if_not_exist = "NX";
    private static final String Set_with_expire_time = "PX";

    /**
     * Attempt to acquire distributed lock
     * @param jedis redis Client
     * @param lockkey lock
     * @param requestid Request Identification
     * @param expiretime Extended Time
     * @return whether to obtain
     success
    /public static Boolean Trygetdistributedlock (Jedis Jedis, String Lockkey, String RequestID, int expiretime) {

        String result = Jedis.set (Lockkey, RequestID, Set_if_not_exist, Set_with_expire_time, Expiretime);

        if (lock_success.equals (Result)) {return
            true;
        }
        return false;
    }
}

As you can see, we lock on one line of code: Jedis.set (String key, String value, String nxxx, string expx, int time), this set () method has five parameters:
The first is key, we use key to be a lock, because key is unique.
The second for value, we preach is RequestID, many children's shoes may not understand, there is key as a lock is enough, why use the value. The reason is that when we talk about reliability, distributed lock to meet the fourth condition to solve the bell also need to fasten the person, by assigning value to the RequestID, we know this lock is which request added, in the unlock time can have the basis. RequestID can use Uuid.randomuuid (). toString ()method is generated.
The third is nxxx, this parameter we fill in NX, meaning is set if not EXIST, that is, when the key does not exist, we do set operation; If key already exists, do not do any operation;
The fourth one is EXPX, this parameter we pass is px, meaning we want to give this key to add an expiration setting, the time is decided by the fifth parameter.
The fifth is time, echoing the fourth parameter, representing the expiration of the key.
In general, executing the Set () method above will only result in two kinds of results: 1. There is currently no lock (key does not exist), then the Lock Operation, and the lock is set to a Validity period, while value represents a locked client. 2. There is a lock existing, do not do any operation.

In the second example, once the value of the Lock_key is found to be less than the current time, the key expires, and then the key is Getset set, once the Getset return value is the original expiration value, indicating that the current client is The first to operate, the representative to obtain the lock, once the Getset return value is not the original expiration time is said to have been modified , the representative did not acquire the lock, detailed see Using Redis to implement distributed locks, corrected as follows:

# get lock
lock = 0 while
lock!= 1:
    timestamp = current_unix_time + lock_timeout
    lock = setnx Lock.foo ti Mestamp
    If lock = = 1 or (now () > [Get Lock.foo] and now () > (getset lock.foo timestamp)): Break
        ;
    else: Sleep
        (10ms)

# Does your job
do_job ()

# release
if now () < get Lock.foo:
    DEL Lock.foo

The existence of lock timeout also causes the loss of the meaning of the lock, that is, the phenomenon of concurrency. Once the lease time of the lock is present, it means that the client acquiring the lock The business logic must be executed within the lease, once the business logic is executed too long and the lease expires, it raises Concurrency Issues。 So the reliability of lock timeout is not so high.

For the client requesting the lock, how do you know that the lock was freed? There are generally 2 ways to achieve this:

1 clients that do not acquire the lock are constantly trying to acquire the lock
2 server-side notification client lock was released
Of course the second situation is optimal (the client does not have the least effort), such as zookeeper through the registration watcher to get the lock release notification. And the database, Redis no way to notify the client lock released, the client can only be silly constantly try to acquire the lock.

The deletion of the lock;

public class Redistool {

    private static final Long release_success = 1L;

    /**
     * Release distributed lock
     * @param jedis redis Client
     * @param lockkey lock
     * @param RequestID Request Identity
     * @return whether or not to release success
     * /Public
    Static Boolean Releasedistributedlock (Jedis Jedis, String Lockkey, String RequestID) {

        string script = "I F Redis.call (' get ', keys[1]) = = Argv[1] then return Redis.call (' del ', Keys[1]) else return 0 ";
        Object result = jedis.eval (script, Collections.singletonlist (Lockkey), Collections.singletonlist (RequestID));

        if (release_success.equals (Result)) {return
            true;
        }
        return false;

    }

}

The first line of code, we write a simple Lua scripting code. First gets the lock corresponding to the valueValue to check whether the RequestIDEqual, if equal, delete the lock (unlock). In the second line of code, we pass the LUA code to the Jedis.eval () method and assign the parameter keys[1] to Lockkey, Argv[1] is assigned a value of RequestID。 The eval () method is to deliver the LUA code to the Redis server for execution.

Five Redlock

Redlock is an official Redis distributed lock principle. Official document Description: "There are a lot of tripartite libraries and articles describing how to implement a distributed lock manager with Redis, but the way these libraries are implemented is very different, and many simple implementations can achieve better reliability by simply adding a little more complex design." The purpose of this article is to try to propose an official authoritative algorithm for implementing a distributed lock manager using Redis, which we call the Redlock, and we believe that the algorithm is more secure and reliable than the general method. “

Why failover based scenarios are not good enough
To understand what we want to improve, let's take a look at the current status of most Redis distributed lock tri-square libraries. The simplest way to implement a distributed lock with Redis is to create a key value in the instance where the key value created is typically a timeout (this is the Redis's own timeout feature), so each lock will eventually be released. And when a client wants to release the lock, it only needs to remove the key value. On the surface, this method seems to work, but there is a problem:

there is a single point of failure in our system architecture, what if Redis's master node goes down?

One might say: Add a Slave node. Use slave when Master is down. But in fact this scheme is obviously not feasible, because this kind of scheme cannot guarantee the 1th security mutex attribute, because the Redis replication is asynchronous. In general, there is an obvious competitive condition in the scheme (race condition), for example:
Client A has a lock on the master node. The master node was down before writing the key created by A to slave. Slave became master Node B and also got the same lock as a (since there was no information on a lock in the original slave) of course, in some special scenarios, the scenario mentioned above is completely free of problems, such as multiple clients allowing locks to be held at the same time during downtime , if you can tolerate the problem, there is no problem with the copy based scheme, otherwise we suggest you take the next scenario described in this article.

Redlock algorithm
In the distributed version of the algorithm we assume that we have n Redis master nodes that are completely independent and we do not need any replication or other implicit distributed coordination algorithms. We have described how to safely acquire and release locks in a single node environment. So we should naturally use this method to acquire and release locks in each single node. In our example, we set N to 5, which is a relatively reasonable number, so we need to run 5 master nodes on different computers or virtual machines to ensure that they don't go down at the same time in most cases. A client needs to do the following to acquire a lock:

1. Gets the current time in milliseconds.
2. Use the same key and random value to request a lock on N nodes, in which the client requests a lock on each master with a smaller timeout than the total lock release time . For example, if the lock automatic release time is 10 seconds, then the timeout for each node lock request may be the range of 5-50 milliseconds, which prevents a client from blocking for a long time on a lost master node , if a master node is unavailable , we should try the next master node as soon as possible.
3. The client calculates the time taken to acquire the lock in the second step, which is considered successful only when the client successfully acquires the lock (3 in this case) on most master nodes, and the total amount of time spent does not exceed the lock release time .
4. If the lock acquisition succeeds, then the lock automatic release time is the initial lock release time minus the time spent acquiring the lock .
5. If the lock acquisition fails, either because the successful lock is not more than half (n/2+1) or because the total consumption time exceeds the lock release time, the client will release the lock on each master node, that is, the lock that he does not think has succeeded.

Redlock Implementation

Redisson is the Redis website recommended by the Java language implementation of distributed lock project. Redisson on the Netty framework based on NIO, it fully utilizes a series of advantages provided by the Redis key value database, and provides users with a series of common tool classes with distributed characteristics on the basis of common interfaces in Java Utility Toolkit. Redisson provides distributed object/distributed set and/distributed locks and distributed services.

Redisson supports 4 types of link Redis:
Cluster (cluster) Sentinel servers (Sentinel) master/slave servers (master) single Server (stand-alone)

Using Redisson to implement distributed locks can be accomplished by simply configuring and using two parts:

1, the Redissonmanager class, the management Redisson initialization and so on operation.

public class Redissonmanager {private static final String Ratomicname = "Genid_";
    private static config config = new config ();

    private static Redisson Redisson = null; public static void Init () {try {config.useclusterservers ()//This is the cluster server used. Setsca Ninterval (2000)//Set the cluster State scan time. Setmasterconnectionpoolsize (10000)//Set the number of connections. Setslave
            ConnectionPoolSize (10000). Addnodeaddress ("127.0.0.1:6379", "127.0.0.1:6380");
            Redisson = redisson.create (config);
            Empty the self added ID number ratomiclong Atomiclong = Redisson.getatomiclong (ratomicname);
        Atomiclong.set (1);
        }catch (Exception e) {e.printstacktrace ();
    } public static Redisson Getredisson () {return redisson; /** gets the atomic ID in Redis/public static Long NextID () {Ratomiclong Atomiclong = Getredisson (). Getatomiclong (
      Ratomicname);  Atomiclong.incrementandget ();
    return Atomiclong.get (); }
}

2. Distributedredislock class, provide lock and unlock method

public class Distributedredislock {
    private static Redisson Redisson = Redissonmanager.getredisson ();
    private static final String Lock_title = "Redislock_";

    public static void Acquire (String lockname) {
        string key = Lock_title + Lockname;
        Rlock Mylock = Redisson.getlock (key);
        Mylock.lock (2, timeunit.minutes); Lock provides a timeout parameter, timeout ends the forced unlock, and prevents the deadlock
        System.err.println ("======lock======" +thread.currentthread (). GetName ());
    }

    public static void Release (String lockname) {
        string key = Lock_title + Lockname;
        Rlock Mylock = Redisson.getlock (key);
        Mylock.unlock ();
        System.err.println ("======unlock======" +thread.currentthread (). GetName ());
    }

3. Test

private static void Redislock () {
        redissonmanager.init ();//Initialize for
        (int i = 0; i < i++) {
            Thread t = n EW Thread (New Runnable () {
                @Override public
                void Run () {
                    try {
                        String key = ' test123 ';
                        Distributedredislock.acquire (key);
                        Thread.Sleep (1000); After acquiring the lock, we can handle the corresponding processing
                        System.err.println ("the corresponding operation = = = After obtaining the lock");
                        Distributedredislock.release (key);
                        System.err.println ("=============================");
                    } catch (Exception e) {
                        e.printstacktrace ();}}}
            );
            T.start ();
        }
    

Test results:

======lock======thread-91
= = = = = = = = = = = = = = =
=========
======lock======thread-63
= = = = After obtaining the lock, do the corresponding operation = = =
======unlock======thread-63
=======================
======lock======thread-31
= = = = After obtaining the lock, do the corresponding operation = = =
======unlock====== Thread-31
=============================
======lock======thread-97
= = = = After obtaining the lock to do the corresponding operation = =
======unlock======thread-97
=============================
======lock======thread-8
= = Get the lock and do the appropriate action = =
======unlock======thread-8
=============================