Zookeeper Application Scenario: Distributed Lock implementation

Question guidance:
1. How does zookeeper implement distributed locks?
2. What is the herd effect?
3. How does zookeeper release the lock?






In the zookeeper application scenario, there is a description of the distributed cluster configuration file synchronization problem. Imagine if 100 machines simultaneously modify a file on the same machine, this is the simplest Distributed Lock to ensure that text is not written in disorder. This article introduces how to implement distributed locks using zk. The following are the steps to implement the write lock:

Distributed write lock
Create a persistent type znode,/Locks/write_lock

• Create a sequence | A znode of the ephemeral type on the client. Its name starts with lockid. The created znode is/Locks/write_lock/lockid1_000001.
• Call getchildren (). Do not set watcher to retrieve the znode list under/Locks/write_lock.
• Determine whether to create znode in step 2 is the smallest in the znode list. If yes, it means that the lock is obtained.
• Call exists () to determine the znode (the smallest znode in the obtained znode node list) created in step 2, and set watcher if exists () return false. Perform step 3.
• If exists () returns true, then wait for the zk notification and return the execution step 3 in the return function.

Release the lock is to delete the znode node or disconnect it.

* Note: in step 2 above, getchildren () does not set watcher because it prevents the herd effect. If getchildren () is set with Watcher, the cluster will receive a notification when it is jitters. During the entire competition of distributed locks, a large number of repeated operations, and the vast majority of the running results are determined that they are not the node with the smallest serial number, so as to continue waiting for the next notification -, this obviously does not seem scientific. The client has received too many Event Notifications unrelated to itself for no reason. If the cluster size is large, it will have a great impact on the server performance, if clients with multiple nodes are disconnected at the same time, the server will send a large number of Event Notifications like other clients-this is called the herd effect.
Below is the code implementation

Import sysclass gjzookeeper (object): zk_host = "localhost: 2181" root = "/locks" workers_path = join (root, "write_lock ") masters_num = 1 timeout = 10000 def _ init _ (self, verbose = true): Self. verbose = verbose self. masters = [] self. is_master = false self. path = none self. zk = zkclient (self. zk_host, timeout = self. timeout) self. say ("Login OK! ") # Init self. _ init_zk () # register self. register () def _ init_zk (Self): "" create the zookeeper node if not exist | -- locks | -- write_lock "nodes = (self. root, self. workers_path) for node in nodes: If not self. zk. exists (node): Try: Self. zk. create (node, "") handle T: Pass def register (Self ): "register a node for this worker | -- locks | -- write_lock | -- lockid000000000x ==> hostname" "Import SOC Ket hostname = socket. gethostname () self. path = self. zk. create (self. workers_path + "/lockid", hostname, flags = zookeeper. ephemeral | zookeeper. sequence) self. lockid = basename (self. path) self. say ("register OK! I'm % s "% self. path) # check who is the master self. get_lock () def get_lock (Self): "" Get Children znode try to get lock "@ watchmethod def watcher (event): Self. say ("child changed, try to get lock again. ") self. get_lock () Children = self. zk. get_children (self. workers_path) children. sort () min_lock_id = children [0] self. say ("% s's children: % s" % (self. workers_path, children) If CMP (self. lockid, m In_lock_id) = 0: Self. get_lock_success () return true Elif CMP (self. lockid, min_lock_id)> 0: Index = children. index (self. lockid) new_lockid_watch = join (self. workers_path, children [index-1]) self. say ("add watch on % s" % new_lockid_watch) RES = self. zk. exists (new_lockid_watch, Watcher) if not res, it's not your turn to re-Execute "" # self. get_lock_success () return false else :" "" Someone is using the lock now. Wait until the lock is released and then rob "self. say ("I can not get the lock this time, wait for the next time") return false def get_lock_success (Self): Self. say ("I get the lock !!! ") Self. write_file () self. zk. Delete (join (self. workers_path, self. lockid) self. Say (" I release the lock !!! ") Sys. exit (1) def write_file (Self): FD = open ("lock. log ", 'A') FD. write ("% s \ n" % self. lockid) FD. close () def say (self, MSG): "print messages to screen" If self. verbose: If self. path: log.info ("[% s (% s)] % s" % (self. path, "Master" If self. is_master else "slave", MSG) else: log.info (MSG) def start_get_lock (): gj_zookeeper = gjzookeeper () def main (): Th1 = threading. thread (target = start_get_lock, name = "thread_1", argS = () th1.start () th1.join () If _ name _ = "_ main __": main () time. sleep (1000)

Article from: http://www.aboutyun.com/forum.php? MoD = viewthread & tid = 9267 & ctid = 16

 

Zookeeper Application Scenario: Distributed Lock implementation