Zookeeper automatic re-registration and test method for registered node

Reprint:http://www.codelast.com/

In a distributed online services system, each service is usually not placed on a single server, but through zookeeper, the service information is registered to the above, The service user discovers information about each service through zookeeper so that the request can be sent to a different service for processing.

As shown, two service Provider 1 and 2 serve on 2688 ports on both 192.168.1.5 and 192.168.1.6, and the address and port of the service are registered in zookeeper. Service user can learn about these services by querying zookeeper. Typically, the communication between the service user and the service provider is implemented through the connection pool, because the service user cannot assume that after the first query to all service provider information, They are always alive, and if a service provider is dead due to a procedural problem, sending a request to it will only result in a large number of failed results, so a connection pool is usually implemented to ensure that the node's information is updated in real time, when there is a service After provider disappears from the zookeeper, the connection removed from the connection pool is always available (that is, it always sends the request to a valid service provider through it).
Article Source: http://www.codelast.com/
Here, we guarantee that when a service provider out of the zookeeper, we will not use it again, but how can we ensure that a service provider is still alive and in a state of readiness? For example, the service Provider 1 program has been very stable, but one day due to the ISP, the network between it and zookeeper was interrupted for 5 minutes, so zookeeper will not be able to send a heartbeat packet to it, and eventually zookeeper will think the session Expired, so that it will register the temporary node to remove (must be registered temporary node Ah, otherwise when the service provider hang after the node is still in, not out of trouble). removed, the problem is: 5 minutes after the interruption of the network back to normal, service Provider 1 program has not died, it can serving, but because it is registered in the Zookeeper node is not, so the service User via connection pool is never able to send request to service Provider 1, so all the request is still sent to the service Provider 2 there, causing its load has been very large, The processing power of the system is weakened.
Therefore, in order to avoid this problem, we need to provide the function of zookeeper-off automatic re-registration in service provider.
"1" How to register with Java zookeeper
The curator library is a great choice. It is a set of open source software developed by Netflix.
What the? Never heard of Netflix? One-third of all US bandwidth is occupied by this company, don't you know?
All right, so the hot mess of the American drama "The Card House" you always heard of it? This is the company that made the money. If this has not been heard, then you can only go to Google.
Digression: Technology and life and entertainment are closely related!
If you want to ask why not directly with the Zookeeper official API, but the use of packaged curator, that can only be explained in a word: Curator very good very powerful very convenient.
Article Source: http://www.codelast.com/
"2" code
"A" uses curator to register zookeeper code

12345678

CuratorFramework curator = CuratorFrameworkFactory.newClient("zookeeper.codelast.com:2181", 5000, 3000, new RetryNTimes(5, 1000));curator.start();String regContent = "192.168.1.5:2688";String zkRegPathPrefix = "/codelast/service-provider-";//TODO:curator.create().creatingParentsIfNeeded().withMode(CreateMode.EPHEMERAL_SEQUENTIAL).forPath(zkRegPathPrefix, regContent.getBytes("UTF-8"));

Explain:
zookeeper.codelast.com:2181  is the host:port of your zookeeper server.
192.168.1.5:2688  is the content of the registered Zookeeper node, which represents service Provider 1 on the 2688 port of 1912.168.1.5. When you use the GET command of the zookeeper client zkcli.sh, the first line of the obtained return value is this content. The
createmode.ephemeral_sequential  indicates that the registered node is temporary and its name is incremented in order. The
/codelast/service-provider-  is the path under which the service is registered to zookeeper. The effect of the above code is that the full path of the registered node will be similar to  /codelast/service-provider-0000000001
Article source: http://www.codelast.com/
"B" The above code is not automatically re-registered in the zookeeper in the case of session expired. To implement this functionality, you need to add a class that implements the Connectionstatelistener interface and applies it to the Curatorframework object. We need to add the following code to "TODO" in the code above:

12	MyConnectionStateListener stateListener = newMyConnectionStateListener(zkRegPathPrefix, regContent);curator.getConnectionStateListenable().addListener(stateListener);

Then implement the Myconnectionstatelistener class:

0102030405060708091011121314151617181920212223242526272829303132

/*** A class to monitor connection state & re-register to Zookeeper when connection lost.** @author Darran Zhang @ codelast.com*/public class MyConnectionStateListener implements ConnectionStateListener {private String zkRegPathPrefix;private String regContent;public MyConnectionStateListener(String zkRegPathPrefix, String regContent) {this.zkRegPathPrefix = zkRegPathPrefix;this.regContent = regContent;}@Overridepublic void stateChanged(CuratorFramework curatorFramework, ConnectionState connectionState) {if (connectionState == ConnectionState.LOST) {while (true) {try {if (curatorFramework.getZookeeperClient().blockUntilConnectedOrTimedOut()) {curatorFramework.create().creatingParentsIfNeeded().withMode(CreateMode.EPHEMERAL_SEQUENTIAL).forPath(zkRegPathPrefix, regContent.getBytes("UTF-8"));break;}} catch (InterruptedException e) {//TODO: log something break;} catch (Exception e) {//TODO: log something }}}}}

Article Source: http://www.codelast.com/
This class is responsible for monitoring the status of the connection and re-registering when the Lost State is detected (at which point the client has been dropped from the zookeeper).
Note that after the lost State is detected, the above code uses a while (true) dead loop to continually attempt to reconnect to the zookeeper server, even if it does not.

"3" Test
How to test? Of course, to create a session expired situation, let zookeeper server think it has dropped the node's registration information to remove. How to let the session expire? The official website has such a message:

Is there an easy-to-expire a session for Testing? yes, a ZooKeeper handle can take a session ID and password. This constructor was used to recover a session after total application failure. For example, an application can connect to ZooKeeper, save the session ID and password to a file, terminate, restart, read The session ID and password, and reconnect to ZooKeeper without loosing the session and the corresponding ephemeral nodes . It's up to the programmer-ensure that the session ID and password isn ' t passed around to multiple instances of an APPL Ication, otherwise problems can result. in the case of testing we want to cause a problem, so to explicitly expire a Session an application connects to ZooKeeper, saves the session ID and password, creates another ZooKeeper handle with tha T ID and password, and then closes the new handle. Since both handles reference the same session, the close on second handle would invalidate the session causing a SESSION_EX Pired on the FIRSt handle.

Article Source: http://www.codelast.com/
But does it need to be so troublesome? It can be done directly through the OS's firewall. For example, on Redhat, use the following command:

1	iptables -A INPUT -d codelast.com -p tcp --sport 2181 -j DROP

You will be able to block traffic from all inputs (input) from the codelast.com domain. This causes the heartbeat between the zookeeper server and the client to fail, thinking that the other side has dropped the line. For zookeeper server, the client node is removed from the zookeeper when it thinks that the client is dropped. This time, if the client program does not have the ability to re-register, then when the network is restored, the client program, although it is able to run normally, but also lost the ability to provide service-because the service users can not find it through zookeeper.
We start the service Provider and then use the command above (if not redhat, please Google yourself) to block the network connection for a period of time, you will eventually observe that the node registered in zookeeper is gone, that is to say we let the session It's expired.
Then, execute the following command:

1	iptables -F

Article Source: http://www.codelast.com/
This command restores the firewall settings. At this point, the network recovery connection, you will see curator print out a lot of information, prompting has been reconnected on the zookeeper server. Note, however, that if you do not have the ability to automatically re-enroll like the previous code, the node that was previously registered in zookeeper does not appear! In other words, it is useless to be connected, it cannot be found.
Once the automatic re-registration feature is added, the nodes registered in zookeeper are automatically recreated. This is what we want to achieve.

Zookeeper automatic re-registration and test method for registered node