Distributed lock 2 Java very use technical solution discussion zookeeper "reprint"

Objective:

because in the normal work, the online server is distributed multiple deployments, often faced with the problem of solving the data consistency in the distributed scenario, we should use the distributed lock to solve these problems. with their own experience in the actual work and some of the information on-line to do a lecture and summary. I have written an article about distributed locks: Distributed lock 1 Java common technical solutions. In the previous article is mainly written in the daily project, the more common methods of implementing the distributed lock. Through these methods, we can basically solve the problem of using distributed locks in most of the scenarios in our daily work.

This article is mainly on the basis of the previous article, the introduction of some although the daily work is not commonly used or relatively heavy, but can be as a technical solution to learn about the distributed lock scheme. I hope this article can be convenient for you to check later, and if you can help others that is also very good.

=============================================================== Long split-line ========================================== ==========================

Body:

The first step, using the Zookeeper node name uniqueness, is used for distributed locks:

About the zookeeper cluster, you can refer to an article I wrote earlier: ZooKeeper1 using virtual machines to build their own zookeeper clusters

The Zookeeper abstract node structure is a small tree-like directory structure similar to the file system, while the zookeeper mechanism stipulates that there can be only one unique file name in the same directory. For example: in the root directory of zookeeper, we created a single client named/mydistributelock with two clients and only one can succeed.

The above scheme and memcached's Add () method, the Redis setnx () method realizes the distributed lock has the same idea. Such a scheme realized if not consider the cost of building and maintaining zookeeper cluster, because the correctness and reliability of the zookeeper mechanism is guaranteed by itself, the implementation is relatively simple.

The second step, using the Zookeeper staging node , is used for distributed locks:

before discussing this package, it is necessary to first "nit-pick" a description of the use of zookeeper node name uniqueness to do distributed locking the disadvantage of this scheme. For example, when many threads are waiting for a lock, if the lock is released, then all the clients are awakened, but only one client gets the lock. In this process, a large number of threads do not have the possibility of locking, but also cause a lot of context switching, this system overhead is not small, for such a phenomenon has a professional noun, called "surprise group effect."

let's start by explaining the sequential nodes, the temporary nodes, and the watcher mechanism of zookeeper:

The so-called sequential node , if we create 3 nodes under the/mydislocks/directory, the zookeeper cluster will create nodes in the order in which they were created,/mydislocks/0000000001,/mydislocks/ 0000000002,/mydislocks/0000000003.

The so-called temporary node , the temporary node is created by a client, and when the client disconnects from the zookeeper cluster, the node is automatically deleted.

so-called for watcher mechanism , we can refer to the Apache ZooKeeper watcher Mechanism source code interpretation. Of course, if you do not know the watcher mechanism is a thing, do not recommend you go directly to the front I provide the article link, so you are very likely to forget our discussion main line, that is, the implementation of distributed locks, and into the watcher mechanism of the source code implementation. So you can also take a look at the following specific plan, guess what watcher is used to do, I here first summed up a sentence to do a primer: the so-called watcher mechanism, You can simply understand that any one connection zookeeper client can through the watcher mechanism to focus on their interests of the node's additions and deletions, when this node occurred additions and deletions to change the operation, will "broadcast" their own messages, all interested in this node can receive these messages, Perform the follow-up actions according to your business needs.

The following steps are used:

1. Each business thread calling the Create () method creates a node named "/mydislocks/thread", and it is important to note that the creation type of the node here needs to be set to Ephemeral_sequential, That is, the node type is a temporary sequential node. At this point, such as/mydislocks/thread0000000001,/mydislocks/thread0000000002,/mydislocks/, are present under the/mydislocks node thread0000000003 such a child node.

2. Each business thread invokes the GetChildren ("Mydislocks") method to obtain all the child nodes that have been created under the/mydislocks node.

3. After each business thread obtains the path to all child nodes, if it finds that the suffix number of the node you created in step 1 is the lowest ordinal of all nodes, then you are considered to have acquired the lock.

4. If you find in step 3 that you are not the smallest ordinal in all child nodes, you have not acquired the lock. Use the watcher mechanism to monitor nodes that are smaller than the serial number of their own nodes (the largest node that is smaller than the node you created), and enter the wait. For example, if the node created by the current business thread is/mydislocks/thread0000000003, he only needs to monitor/mydislocks/thread0000000002 if the lock is not acquired. Only when/MYDISLOCKS/THREAD0000000002 acquires the lock and is released, the current line of business friend initiates the acquisition of the lock, which prevents a business thread from releasing the lock and all other threads compete for the lock, causing unnecessary context switching, resulting in a "surprise swarm".

5. The process of releasing the lock is relatively simple, which is to delete the child node that you created.

Note: This scheme realizes the distributed lock with a bit of fair lock taste! Why is it? When we use the ordinal of each node to queue up to avoid the swarm phenomenon, the order in which all business threads acquire locks is the order in which they create the nodes themselves, which is which line of business is enters upgradeable and which is the quickest to get the lock.

Here's the code for the scenario I implemented myself:

1. There are two Java classes in the code: Mydistributedlockbyzk.java and Lockwatcher.java. The main function in Mydistributedlockbyzk.java, which uses the thread pool to start 5 threads, simulates the competition of multiple business threads, while Lockwatcher.java defines a distributed lock and implements the watcher mechanism.

2. At the same time, I use the zookeeper cluster is my previous use of the cluster built by VMware, so zookeeper link is 192.168.224.170:2181, you can replace the zookeeper link to their own.

 1 public class Mydistributedlockbyzk {2/** thread pool **/3 private static executorservice Executorservice = NULL; 4 private static final int thread_num = 5; 5 private static int threadno = 0; 6 private static Countdownlatch Threadcompletelatch = new Countdownlatch (thread_num);  7 8/** ZK-related configuration constants **/9 private static final String connection_string = "192.168.224.170:2181"; static final int session_timeout = 10000;11//This variable also has a static variable with the same name in LockWatcher, which is then extracted to a constant class for common maintenance when formally used.          The private static final String Lock_root_path = "/mydislocks"; public static void Main (string[] args) {15             Define thread pool Executorservice = Executors.newfixedthreadpool (Thread_num, New Threadfactory () {17  @Override18 public Thread Newthread (Runnable r) {String name = String.Format ("[%s] test thread",    ++THREADNO), thread ret = new Thread (Thread.CurrentThread (). Getthreadgroup (), R, name, 0); 21             Ret.setdaemon (false); return ret;23}24}); 25 26//Start-up line if (executorservice! = null) {startprocess ();}30}31/**33 * @au Thor zhangyi0334 * @date 2017-5-23 PM 5:57:2735 * @description simulate concurrent execution tasks */37 public static void start Process () {Runnable disposebusinessrunnable= new Thread (new Runnable () {$ public void run () {4 0 String threadname = Thread.CurrentThread (). GetName (); LockWatcher l                     Ock = new LockWatcher (threadcompletelatch); try {44/** Step 1: The current thread creates a ZK connection **/45  Lock.createconnection (connection_string, Session_timeout); 46 47/** Step 2: Create the root node of the lock **/48//Note that the way the root node is created here is entirely possible to create the root node individually at initialization time by the main thread, and there is no need to create it in a business thread. 49//This writing is just a way of thinking, not limited to this 50                     Synchronized (mydistributedlockbyzk.class) {Wuyi Lock.createpersistentpath (lock_ Root_path, "This node is created by" + ThreadName + ", true); 52}53 54/** Step 3 : Turn on lock competition and perform task **/55 lock.getlock (); Exception catch (e) {E.P Rintstacktrace (); (+) (+) (+) (), (int i = 0; i < Thread_ NUM;          i++) {Executorservice.execute (disposebusinessrunnable);}65 Executorservice.shutdown (); 66 try {threadcompletelatch.await (); System.out.println ("All Threads run end!"); (Interruptedexception e) {e.printstacktrace (); 72}73}74}

  1 public class LockWatcher implements watcher {2/** member variable **/3 private ZooKeeper ZK = null;  4//The node path created when the current business thread competes for a lock 5 private String selfpath = null;  6//The current business thread competes for the lock when creating the node's predecessor Path 7 private String waitpath = null; 8//Make sure the connection to ZK is successful; only after you receive the Watcher listener event, do the subsequent operations, or the request blocks in CreateConnection () the method of creating the ZK Connection 9 private Countdownlatch Connectsu Ccesslatch = new Countdownlatch (1); 10//identifies whether the thread finishes the task one by one private countdownlatch threadcompletelatch = null; The relevant configuration constants of/** ZK **/the private static final String Lock_root_path = "/mydislocks"; The private static final String Lock_sub_path = Lock_root_path + "/thread";      LockWatcher public (Countdownlatch latch) {this.threadcompletelatch = latch; 19} 20 21         @Override public void Process (Watchedevent event) {(event = = null) {25 return; } 26 27//Notification status Event.keeperstate keeperstate = Event.GetState (); 29//Event Type: Event.eventtype EventType = Event.gettype (); 31 32//According to the notification status, respectively, handle the IF (Event.KeeperState.SyncConnected = = keeperstate) { Event.EventType.None = = EventType) {System.out.println (Thread.CurrentThread (). GetName () + "successfully connected to ZK server "); 36//The main function of the code here is to assist in judging that the current thread is indeed connected to the ZK Notoginseng Connectsuccesslatch.countdown ();                 }else if (event.gettype () = = Event.EventType.NodeDeleted && event.getpath (). Equals (Waitpath)) {39 System.out.println (Thread.CurrentThread (). GetName () + "received intelligence, platoon me in front of the guy has been hanging, I am ready to confirm again I am not the smallest node!?" ");                     try {Checkminpath ()) {getlocksuccess (); 43                 } The catch (Exception e) {e.printstacktrace (); 46 } Event.KeeperState.Disconnected} else if (= = Keeperstate) {System.out.println (Thread.CurrentThread (). GetName () + "Disconnect from ZK server"), or else if (Event.kee         perstate.authfailed = = keeperstate) {Wuyi System.out.println (Thread.CurrentThread (). GetName () + "permission check Failed"); 52 } else if (Event.KeeperState.Expired = = keeperstate) {System.out.println (Thread.CurrentThread (). GetName () + "session invalidation"); * * * */** * @author zhangyi03 * @date 2017-5-23 pm 6:07:03 * @descr Iption Create ZK Connection * @param connectstring ZK server address List $ * @param sessiontimeout Session Timeout period * @throws Ioexc Eption * @throws interruptedexception * * createconnection (String connectstring, int ses Siontimeout) throws IOException, interruptedexception {in ZK = new ZooKeeper (connectstring, sessiontimeout, this) ;  When connectsuccesslatch.await (1, timeunit.seconds) is formally implemented, it is possible to consider whether the time-out is used to block the connectsuccesslatch.await ();/** * @author zhangyi03 * @date 2017-5-23 PM 6:15:48 * @description Create ZK node * @param path node path * @param data content * @param needwatch * @return * @throws K Eeperexception Bayi * @throws interruptedexception * * Createpersistentpath (String path, S Tring data, Boolean Needwatch) throws Keeperexception, interruptedexception {zk.exists (path, needwatch) = =  NULL) {$ String result = Zk.create (Path,data.getbytes (), ZooDefs.Ids.OPEN_ACL_UNSAFE, createmode.persistent); System.out.println (Thread.CurrentThread (). GetName () + "Create node succeeded, path:" + result + ", content:" + data); (a) a return of true; /** * @author zhangyi03 * @date 2017-5-23 PM 6:24:46 94 * @description Get Distributed lock * @throws keeperexception * @throws interruptedexception/98 public void GetlocK () throws Exception {Selfpath = zk.create (Lock_sub_path, NULL, ZooDefs.Ids.OPEN_ACL_UNSAFE, Createmode.epheme ral_sequential), System.out.println (Thread.CurrentThread () getName () + "Create Lock Path:" + Selfpath); 101 if (CHEC      Kminpath ()) {102 getlocksuccess (); 103}104}105 106/**107 * @author zhangyi03108 * @date 2017-5-23 pm 7:02:41109 * @description Get Lock Success * @throws KeeperException111 * @throws Interrupt edException112 */113 private void Getlocksuccess () throws Keeperexception, interruptedexception {if ( Zk.exists (Selfpath, false) = = null) {System.err.println (Thread.CurrentThread (). GetName () + "This node is gone ..."); 1 return;117}118 System.out.println (Thread.CurrentThread (). GetName () + "Get lock successfully, start processing business data!" 119 Thread.Sleep (), System.out.println (Thread.CurrentThread (). GetName () + "process business data completion, delete this node:" + Selfpath); 121 ZK.Delete (Selfpath,-1); 122 releaseconnection (); 123 Threadcompletelatch.countdown (); 124}125 126 /**127 * @author zhangyi03128 * @date 2017-5-23 pm 7:06:46129 * @description Close ZK connection */131 privat             e void Releaseconnection () {(ZK = null) {133 try {134 zk.close (); 135 } catch (Interruptedexception e) {136 e.printstacktrace (); 137}138}139 Sys      Tem.out.println (Thread.CurrentThread (). GetName () + "free ZK connection");}141 142/**143 * @author zhangyi03144 * @date 2017-5-23 pm 6:57:14145 * @description Check if you are not the smallest node 146 * @param selfPath147 * @return148 * @t Hrows KeeperException149 * @throws InterruptedException150 */151 Private Boolean Checkminpath () throws Excep           tion {list<string> subnodes = Zk.getchildren (Lock_root_path, false); 153//sorted by dictionary in ascending order of the elements 154 Collections.sorT (subnodes); 155 System.err.println (Thread.CurrentThread (). GetName () + "temporary node name created:" + selfpath.substring (Lock_roo           T_path.length () +1); 156 int index = Subnodes.indexof (selfpath.substring (Lock_root_path.length () +1)); 157               System.err.println (Thread.CurrentThread (). GetName () + "Create a temporary node for index:" + index); 158 switch (index) {159 Case-1: {System.err.println (Thread.CurrentThread (). GetName () + "created node no longer ..." + selfpath); 16 1 return false;162}163 case 0:{164 System.out.println (Thr               Ead.currentthread (). GetName () + "sub-node, I am really the eldest" + Selfpath); 165 return true;166}167 default:{168//Get the front node smaller than the current node, here only to focus on whether the predecessor node is still in existence, to avoid the surprise group phenomenon produced 169 Waitpath = Lock_root _path + "/" + Subnodes.get (index-1); System.out.println (Thread.CurrentThread (). GetName () + "Get child nodes, row in The node in front of me is: "+ waitpath); 171 try {172 Zk.getdata (Waitpath, True, New Stat ()); 173 return                            false;174} catch (Exception e) {175 if (Zk.exists (Waitpath, false) = = null) {176                            System.out.println (Thread.CurrentThread (). GetName () + "sub-node," + Waitpath + "in front of Me" is missing, that's Me "); 177 return Checkminpath (); 178} else {179 throw e;1 80}181}182}183 184}185}186}

The third step, using Memcached's CAs () method, is for distributed locks:

Next article we will elaborate!

The fourth step, using the Redis Watch, multi, and EXEC commands, is used for distributed locks:

Next article we will elaborate!

The fifth step, summary:

In summary, for the distributed lock these are very useful or relatively heavy implementation of the scheme, you can according to their own needs in the project, as appropriate, use. Recently in the course of discussions with others, and my first article on distributed locks 1 Java Common technical solutions Everyone's reply, in summary, for the implementation of a distributed lock with Redis there are a lot of details of the problem can be discussed in depth, you are welcome to leave a message, Learn from each other.

Can not help hooves, my daughter-in-law son at the moment beside me to see Abstractqueuedsynchronizer, bad?! , go out for dinner, haha ~

The sixth step, the online use of the supplementary article:

as of 2017.08.25 (Friday), when solving a distributed lock problem using the "Temporary node +watcher scheme" in the above article, it was found that in the implementation process, because the watcher mechanism resembles the characteristic of the notification waiting mechanism, if the main thread is experiencing " During the three steps of obtaining a lock operation, handling business code, and releasing a lock operation, using the watcher mechanism to block the acquisition lock results in the inability to return the resulting lock result to the main thread at all, and in the actual process, the main thread in the "Get Lock operation" , you want to be able to get a return value synchronously.

Therefore, the above "temporary node +watcher mechanism scheme" from the technical solution angle is perfect, but in the actual use of the process, the personal feel is not particularly convenient.

Reprinted from: Http://www.cnblogs.com/PurpleDream/p/5573040.html.

Distributed lock 2 Java very use technical solution discussion zookeeper "reprint"