Zookeeper Application Scenario: How to run for Master and code implementation

Question guidance:
1. How to Use zookeeper to ensure cluster master availability and uniqueness?
2. What processes does zookeeper run for Master?
3. What features does Zookeeper's master election mechanism take advantage of ZK?






In the zookeeper application scenario, the problem of master node management is raised. The following describes how to ensure the availability and uniqueness of Master nodes in the cluster using zookeeper.
To ensure that zookeeper cluster nodes have been installed and configured, assuming that ZK has provided external services normally, run the following steps to run the master election:

• The client connects to the zk and determines whether znode/roles/workers exists. If no znode exists, it is created. The znode type is persistent, so that it will not disappear with the session of C1.
• The client creates a sequence | Ephemeral-type znode under/roles/workers. The prefix can be worker. ZK ensures that the znode number is incremental and temporary, once the session is disconnected, the created znode disappears.
• The client obtains the znode list of all/roles/workers instances through getchildren, and sets a watcher to wait for the notification. The number of znodes returned is the same as that of the client.
• Sort the node list returned by step 4 and find the smallest worker number. If it is the same as the one created by yourself (the result returned by step 2), it means that your number is the smallest and you are the master. If you find that your number is not the smallest, wait for the notification. Once watcher is triggered, go back to Step 3 in watcher.

The above mechanism mainly utilizes several ZK features

• For N clients that request to create a znode at the same time, ZK can ensure the order consistency and ensure that the znode nodes created by each client are incremental and unique.
• Because the created znode is temporary, once the session is disconnected, znode will disappear from ZK, so as to send a notification to each client that sets watcher, so that each client can re-run for Master, the smaller number must be the master, ensuring uniqueness.
  

Is the logic diagram above
<Ignore_js_op>



The following is the implementation code. The local ZK service is connected by default, and the port is 2181. The zkclient module is located on the zookeeper Python interface. You only need to run multiple scripts to run the master election.

• Run the following script on three terminals, simulating the client C1, C2, and C3. The created nodes are/roles/workers/worker1_00 in sequence, /roles/workers/worker1_000001, followed by/roles/workers/worker1_02
• It is found that C1 successfully runs for the master, and then C2 and C3 are both slave
• After C1 is turned off,/roles/workers/worker1_00 disappears in sequence. After a period of time, C2 and C3 will run again, C2 will become the master, and C3 will be the slave.
• Restart C1 and find that C1 is immediately added to the cluster. The change in the message indicates that the new znode is created in the order of/roles/workers/worker1_1_03, re-election, C2 or master.

PS: in step 3, after a client is closed, znode will be deleted after a period of time, because the zk session has not been cleared during this period, because the command Ctrl + C is used to close the SDK. However, when a client is added and created in znode, other clients registered with Watcher will be notified.

#!/usr/bin/env python2.7# -*- coding: UTF-8 -*-import loggingfrom os.path import basename, joinfrom zkclient import ZKClient, zookeeper, watchmethodlogging.basicConfig(    level = logging.DEBUG,    format = "[%(asctime)s] %(levelname)-8s %(message)s")log = loggingclass GJZookeeper(object):    ZK_HOST = "localhost:2181"    ROOT = "/Roles"    WORKERS_PATH = join(ROOT, "workers")    MASTERS_NUM = 1    TIMEOUT = 10000    def __init__(self, verbose = True):        self.VERBOSE = verbose        self.masters = []        self.is_master = False        self.path = None        self.zk = ZKClient(self.ZK_HOST, timeout = self.TIMEOUT)        self.say("login ok!")        # init        self.__init_zk()        # register        self.register()    def __init_zk(self):        """        create the zookeeper node if not exist        |-Roles             |-workers        """        nodes = (self.ROOT, self.WORKERS_PATH)        for node in nodes:             if not self.zk.exists(node):                try:                    self.zk.create(node, "")                except:                    pass    @property    def is_slave(self):        return not self.is_master    def register(self):        """        register a node for this worker,znode type : EPHEMERAL | SEQUENCE        |-Roles             |-workers                     |-worker000000000x         ==>>master                     |-worker000000000x+1       ==>>worker                     ....        """        self.path = self.zk.create(self.WORKERS_PATH + "/worker", "1", flags=zookeeper.EPHEMERAL | zookeeper.SEQUENCE)        self.path = basename(self.path)        self.say("register ok! I‘m %s" % self.path)        # check who is the master        self.get_master()    def get_master(self):        """        get children, and check who is the smallest child        """        @watchmethod        def watcher(event):            self.say("child changed, try to get master again.")            self.get_master()        try :            children = self.zk.get_children(self.WORKERS_PATH, watcher)        except zookeeper.ConnectionLossException:            print "losing connection with zookeeper..."            return False        except :            return False        children.sort()        self.say("%s‘s children: %s" % (self.WORKERS_PATH, children))         # check if I‘m master        self.masters = children[:self.MASTERS_NUM]        if self.path in self.masters:            self.is_master = True            self.say("I‘ve become master!")        else:            self.say("%s is masters, I‘m slave" % self.masters)    def say(self, msg):        """        print messages to screen        """        if self.VERBOSE:            if self.path:                log.info("[ %s(%s) ] %s" % (self.path, "master" if self.is_master else "slave", msg))            else:                log.info(msg)def main():    gj_zookeeper = GJZookeeper()if __name__ == "__main__":    main()    import time    time.sleep(1000)

　　

Article from: http://www.aboutyun.com/forum.php? MoD = viewthread & tid = 9277 & ctid = 16

Zookeeper Application Scenario: How to run for Master and code implementation