Regionserver Exception Timeout detection

Source: Internet
Author: User
Keywords Timeout detection zookeeper cluster shutdown log regionserver exception
Tags apache client configuration controlled default host java network
Phenomenon

The main network cable unplugged, regionserver over a period of time will be shutdown out, regionserver abnormal timeout detection mechanism is controlled by which parameter?

Regionsever Exception shutdown log:

2017-08-03 21:17:42,762 WARN [Main-sendthread (ocscloud07:9501)]zookeeper. Clientcnxn:session 0x25ce22897746d12 for serverocscloud07/134.32.62.7:9501, unexpected error, closing socket Connection and attempting reconnect

Java.io.IOException:No Route to host

At Sun.nio.ch.FileDispatcherImpl.read0 (Native method)

At Sun.nio.ch.SocketDispatcher.read (socketdispatcher.java:39)

At Sun.nio.ch.IOUtil.readIntoNativeBuffer (ioutil.java:223)

At Sun.nio.ch.IOUtil.read (ioutil.java:192)

At Sun.nio.ch.SocketChannelImpl.read (socketchannelimpl.java:384)

Atorg.apache.zookeeper.ClientCnxnSocketNIO.doIO (clientcnxnsocketnio.java:68)

At Org.apache.zookeeper.ClientCnxnSocketNIO.doTransport (clientcnxnsocketnio.java:355)

At Org.apache.zookeeper.clientcnxn$sendthread.run (clientcnxn.java:1081)

2017-08-03 21:17:43,122 INFO [Leaserenewer:e3base@drmcluster]retry. Retryinvocationhandler:exception while invoking renewlease of Classclientnamenodeprotocoltr AnslatorPBover OCSCloud04 /134.32.62.4:8020. Trying to fail over immediately.

Java.net.NoRouteToHostException:No Route tohost from ocscloud11/134.32.62.11 toocscloud04:8020 failed on socket timeout Exception:java.net.NoRouteToHostExcep Tion:no route to host; For more Detailssee:http://wiki.apache.org/hadoop/noroutetohost

At Sun.reflect.NativeConstructorAccessorImpl.newInstance0 (Native method)

Problem Solving

Setting the timeout time in HBase is useless, and you must modify zookeeper's own maxsessiontimeout to 1200000 to really play the role of the extended zookeeper session timeout.

When Regionserver starts, it creates its own znode temporary node in the zookeeper cluster and obtains an exclusive lock on the node in the/hbase/rs directory in the Zookeeper cluster.

Regionserver initiates session sessions to the zookeeper cluster via a socket connection and creates its own temporary node Znode in the Zookeeper cluster after the session is established. The state of this node is maintained by the zookeeper cluster based on the state of the session.

Regionserver, as a client, initiates a session request to the server side of the zookeeper cluster. After the session is established, it will be marked with a unique sessionid. The client periodically sends a PING message to the server side to express the surviving state of the session, and the server side updates the timeout for the current session when it receives a ping message. Thus, for the client, the session is activated as long as the ping information is available, and for the server, the session is active as long as the session does not time out.

On the server side, zookeeper initiates a dedicated Sessiontrackerimpl thread to handle the associated State migration issue for the session, which is specified in Ticktime (zookeeper configuration file, default 2 s) time traverses the session list, closes this session immediately if the timeout occurs, deletes the temporary node associated with the session, and notifies the component that registered the node event. In the HBase cluster, this means that if the regionserver crashes, zookeeper needs to notify Master after the session timeout before it can initiate failback.

The timeout for the session is determined in this way: HBase default timeout is 180 s, which is passed to the server side when the session is created. The timeout time for the session determined by the final negotiation is determined by the zookeeper configuration parameters, between the Zookeeper cluster minsessiontimeout and maxsessiontimeout. The default minsessiontimeout=2xticktime (default 2 s) =4 s,maxsessiontimeout=20xticktime=40 S. Regardless of how much timeout the client delivers, the final negotiated session timeout time is between 4~40 s, the implementation code is as follows. If everything is in the default configuration, the session timeout is S.

int Sessiontimeout=connreq.gettimeout ();

int Minsessiontimeout=getminsessiontimeout ();

if (sessiontimeout

Sessiontimeout=minsessiontimeout;

}

int Maxsessiontimeout=getmaxsessiontimeout ();

if (sessiontimeout>maxsessiontimeout) {

Sessiontimeout=maxsessiontimeout;

}

Cnxn.setsessiontimeout (sessiontimeout);

Through the above analysis, we can draw the following conclusions: The survival of the session means Regionserver survival; The session timeout means that the Znode node that was created when the Regionserver was started is deleted, indicating that the regionserver exception

Related Article

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.