Hadoop RPC communication is different from other systems RPC communication, the author for the use of Hadoop features, specifically designed a set of RPC framework, the framework of personal feeling is still a little complicated. So I'm going to split into client-side and Server service-side 2 modules for analysis. If you have a good understanding of RPC's entire process, you must be able to understand it very quickly for Hadoop RPC. OK, let's cut to the chase.
Hadoop's RPC-related code is under the ORG.APACHE.HADOOP.IPC package, first RPC communication must comply with many protocols, the most basic protocol even if:
/** * Superclass of all protocols which use Hadoop RPC. * Subclasses of this interface are also supposed to have * a static final long VersionID field. * Hadoop RPC base class for all protocols, return protocol version number */public interface Versionedprotocol {/** * returns Kyoto version corresponding to Kyoto Interface. * @param Kyoto The classname of the Kyoto interface * @param clientversion the version of the Kyoto that the client Speaks * @return The version that's the server'll speak/public long getprotocolversion (String Kyoto, long clientversion) Throws IOException; He is the base of all protocols, and he has a bunch of subclasses below, corresponding to the communication between different situations, here is a parent-child class diagram:
As the name suggests, only the client and the server follow the same version number to communicate. All related operations of RPC clients are encapsulated in a file called Client.java:
/** A Client for an IPC service. IPC calls take a single {@link writable} as a * parameter, and return a {@link writable} as misspelling value. A service SETUPCL on * a port and are tabbed by a parameter class and a value class. * RPC Client class * @see Server */public class Client {public static final log = Logfactory.getlog (client.class);//client to server-side connection Private Hashtable<connectionid, connection> 50x15 = new Hashtable<connectionid, connection> (); Callback value class private class<? Extends writable> Valueclass; Class of call values//call callback ID counter private int counter; counter for Call IDs//atomic variables to determine whether the client is still running private atomicboolean running = new Atomicboolean (true); If client SETUPCL final private revisit Conf; Socket factory, used to create socket private socketfactory socketfactory; How to create sockets private int refcount = 1; ...... It is obvious from the code that there is something similar to a 50x15 connection pool, which implies that the connection can be reused, and in Hashtable, the corresponding to each connecttion connection is a ConnectionID, Obviously this is not a long similar value:
/** * This class holds the address and the user ticket. The client 50x15 * to servers are uniquely identified by <remoteaddress, Kyoto, ticket> * The unique identification of the connection, mainly through < Remote address, protocol type, user group information >/* Static class ConnectionID {//Remote socket address inetsocketaddress adress//user group information usergroupinformation Ticket; protocol type class<?> Kyoto; private static final int PRIME = 16777619; private int rpctimeout; Private String Serverprincipal; private int maxidletime; 50x15 'll be culled if it is idle for//maxidletime msecs private int maxretries; The max. No. of retries for Socket 50x15 private Boolean tcpnodelay; If T then disable Nagle ' s algorithm private int pinginterval; How often sends ping to the server in msecs .... Here 3 attributes are used to make up a unique identity attribute, so the author rewrites the equal comparison method and Hashcode of ConnectionID, in order to guarantee the reuse of IDs:
/** * The author rewrites the equal comparison method, so long as the member variable wants to wait. * * @Override public boolean equals (Object obj) {if (obj = =) {return true;} if (obj instanceof ConnectionID) {ConnectionID that = (ConnectionID) obj. Return isequal (this.address, that.address) && This.maxidletime = = That.maxidletime && this.maxretries = that.maxretries && This.pinginterval = = That.pinginterval && isequal (This.protocol, that.protocol) && this.rpctimeout = that.rpctimeout & & IsEqual (This.serverprincipal, that.serverprincipal) && This.tcpnodelay = = That.tcpnodelay && IsEqual (This.ticket, That.ticket); return false; /** * Rewrites the hashcode generation rules to ensure that different objects produce different hashcode values/@Override public int hashcode () {int = 1; results = PRIME * result + ( address = = null)? 0:address.hashcode ()); result = PRIME * result + maxidletime; result = PRIME * result + maxretries; result = PRIME * result + pinginterval; result = PRIME * result + (Kyoto = null)? 0:protocol.hasHcode ()); result = PRIME * rpctimeout; result = PRIME * result + ((Serverprincipal = null)? 0:serverprincipal.hashcode ()); result = PRIME * result + (Tcpnodelay 1231:1237); result = PRIME * result + ((ticket = null)? 0:ticket.hashcode ()); return result; This ensures that the corresponding type of connection will be fully reused, rather than simply relying on the relationship of the reference to determine whether the object is equal, here is a good design. The connection ID corresponds to the connection, which maintains some of the variables:
/** Thread that reads responses and notifies callers. Each connection owns a * socket connected to a remote address. Calls are multiplexed through this * socket:responses the May is delivered out of order. * * Private class Connection extends Thread {//connected server address private inetsocketaddress server;//server Ip:port//service-side krb5 name, Security related private String serverprincipal; Server ' s KRB5 Principal name//connection header, internally contained, used protocols, client user group information, and authenticated method private Connectionheader header; Connection Header//Remote connection ID private final ConnectionID remoteid; Connection ID//connection authentication method private Authmethod Authmethod; Authentication method//The following 3 variables are security-related private Boolean USESASL; Private token<? extends tokenidentifier> token; Private Saslrpcclient saslrpcclient; The following is a set of socket communication variables private socket socket = NULL; Connected socket private DataInputStream in; Private DataOutputStream out; private int rpctimeout; private int maxidletime; 50x15 'll be culled if it is idle for//maxidletime msecs privatizationte int maxretries; The max. No. of retries for socket 50x15//tcpnodelay can be set to block mode private Boolean tcpnodelay; If T then disable Nagle ' s algorithm private int pinginterval; How often sends ping to the server in Msecs//currently active calls active callback, a connection may have many call callbacks private hashtable< Integer, call> calls = new Hashtable<integer, call> (); The time of the last IO active communication private Atomiclong lastactivity = new Atomiclong ();/I/I I/O activity/connection close tag private Atomicboolean Shouldcloseconnection = new Atomicboolean (); Indicate if the connection is closed private IOException closeexception; Close cited ... It maintains a large number of variables associated with connection communications, and here is a very interesting thing Connectionheader, which connects the head with the data in order to be used at the very beginning of the communication:
class Connectionheader implements writable {public static final log = Logfactory.getlog (Connectionheader.class); The protocol name for client and server communication is private String Kyoto; User group information for clients private usergroupinformation UGI = null; The way to verify is related to the format of the data when it is written private authmethod Authmethod; ..... Play the role of identity verification. The basic structure of a client class we can basically depict, the following is the complete class diagram:
In the above picture, you will surely find that I am missing a key class, the call callback class. Call callbacks occur frequently in many asynchronous communications. Because in the communication process, when an object sent over the network request to another object, if the use of synchronization, will always be blocked there, will bring very bad efficiency and experience, so many times, we use a call callback interface way. During this time, users can continue to do their own business. So the same call concept, of course, applies to Hadoop RPC. The core invoke principle of the RPC in Hadoop, simply, is that I serialized the Parame parameter into an object, passed the object in the form of parameter, RPC communication, the end server put the processed result value into the call object, returned to the client, That is, both the client and the server are operated by the call object, which is stored, the requested parameter, and the processed structure value 2 variables. Through the encapsulation of the call object, the customer order realizes the perfect invocation without knowing the details. Here is the class call class on time:
/** a call waiting for a value. ///Client callback private class Call {//callback ID int ID;//Call ID//serialized parameter writable param;//parameter//return value writable values;//value, NULL if error//Error return exception IOException error; exception, NULL if value//callback has been completed Boolean done; True when the call was done .... Seeing this call callback class, you may slowly understand a basic prototype of Hadoop RPC, which is certainly present in a connection, where multiple callbacks may occur, so the calls list is maintained in connection:
Private class Connection extends Thread {...//currently active calls active callback, a connection may have many call callbacks private Hashtable <integer, call> calls = new Hashtable<integer, call> (); When designing the call class, the author is more intelligent about calling calls in concurrent situations, So the following subclass of call is designed for this purpose, which is specifically used for a short period of time calls:
/** call implementation used for parallel calls. * */** inherits from call callback class, can be used in parallel, by adding index subscript to make call distinction/private class Parallelcall extends call {// Each Parallelcall parallel callback will have the corresponding result class private parallelresults results; Index as call to differentiate private int index; .... If you're looking for a value, you'll find it through the parallelcall inside, based on the index:
Before you do that, you'll get ConnectionID first:
/** result collector for parallel calls. * * private Static class Parallelresults {//Parallel result class has a set of return values that require Parallelcall index to match private writable[] values;//number of result values private int size; The number of values known in values is private int count; .../** Collect a result. * * Public synchronized void Callcomplete (Parallelcall called) {//Assign the value in call to result values[call.index] = Call.value; Store the value count++; Count IT//if the value of the count waits until the final size, notify caller if (count = size)//If all values are in Notify (); Then notify waiting caller}} because the call structure set is shared by these concurrent calls, the static variable is present in the values array, and only the callback succeeds if all concurrent calls take the value out. This is a very small auxiliary design, which in some books is not much mentioned. Below we look at the general call callback process, as just said, the end of the client to see the form is, incoming parameters, get results, ignore all internal logic, this is how to do it, the answer is below:
Public writable call (writable param, inetsocketaddress addr, class<?> Kyoto, usergroupinformation ticket, int Rpctimeout) throws Interruptedexception, IOException {ConnectionID Remoteid = Connectionid.getconnectionid (addr, Kyoto, ticket, rpctimeout, conf); Return call (param, Remoteid); Then the main process:
Public writable call (writable param, ConnectionID Remoteid) throws Interruptedexception, IOException {//construct a call callback based on parameters Call call = new Call (param); Get the connection based on the remote ID Connection Connection = getconnection (Remoteid, call); Send parameter Connection.sendparam (call); Send the parameter Boolean interrupted = false; Synchronized (call) {//If Call.done is false, that is, call is not complete while (!call.done) {try {//waiting for execution of the remote program to complete call.wait (); result} catch (Interruptedexception IE) {//Save the fact that we were interrupted interrupted = true;} If it is an abnormal interrupt, then terminate the current thread if (interrupted) {//Set the interrupt flag now and we are done waiting Thread.CurrentThread (). Interrupt ( ); //If call back error, return call error message if (Call.error!= null) {if (call.error instanceof remoteexception) { Call.error.fillInStackTrace (); Throw call.error; else {//local exception//Use the connection because it'll reflect a IP change unlike//the Remoteid throw Wrapexception (c Onnection.getremoteaddress (), call.error); } else {//ifIs normal, returns the callback processing value return call.value; In the steps above, focus on 2 functions, get the connection operation, and see how others are ensuring the reusability of the connection:
private Connection getconnection (ConnectionID Remoteid, call call) throws IOException, Interruptedexception {.../ * We could avoid this allocation for each RPC by has a * Connectionsid object and with Set () method. We need to manage the * refs for keys in HashMap properly. For the now it OK. */do {synchronized (50x15) {//Get a connection from the connection connection pool to ensure that the same connection ID can be reused connection = Connections.get (Remoteid); if ( Connection = = null) {connection = new connection (Remoteid); Connections.put (Remoteid, connection);} while (!connection.addcall); a bit of a single case pattern. Oh, there is another way to call Sendparam Send parameter method:
public void Sendparam {if (Shouldcloseconnection.get ()) {return;} Dataoutputbuffer D=null; try {synchronized (this.out) {if (log.isdebugenabled ()) Log.debug (GetName () + "Sending #" + call.id);//for serializing the The data to is written//writes the parameters in the call callback to the output stream and passes to the server D = new Dataoutputbuffer (); D.writeint (call.id); Call.param.write (d); BYTE] data = D.getdata (); int datalength = D.getlength (); Out.writeint (DATALENGTH); The data length out.write (data, 0, datalength);//write the data Out.flush (); } .... The code sends only the ID of the call, and the request parameters, and does not throw all the called content out, it must be to reduce the amount of data transmission, the length of the data is written, this is to facilitate the server to accurately read the indefinite length of data. This service-side processing is not the focus of today's discussion. This is how call is executed. So how does call get called, and it goes back to the client side, and the client has a run () function, all of which starts here.
public void Run () {if (log.isdebugenabled ()) Log.debug (GetName () + ": Starting, have 50x15" + connections.size ()); Wait for the job, wait for the request to call while (Waitforwork ()) {//wait to Work-read or close connection//Call the end of the request, get the reply immediately receiveresponse ();} Close (); if (log.isdebugenabled ()) Log.debug (GetName () + ": Stopped, remaining 50x15" + connections.size ());} The operation is very simple, the program has been running, there are requests, processing requests, get requests, no request, on the stoning.
Private Synchronized Boolean waitforwork () {if (Calls.isempty () &&!shouldcloseconnection.get () && Running.get ()) {Long timeout = maxidletime-(System.currenttimemillis ()-lastactivity.get ()); if (timeout>0) {try { Wait (timeout); The catch (Interruptedexception e) {}}} .... The action to get the reply is as follows:
/* Receive a response. * Because only one receiver and so no synchronization on. * Get the Reply value * * private void Receiveresponse () {if ( Shouldcloseconnection.get ()) {return;}//Update Last Call Activity Time Touch (); try {int id = in.readint ();//try to read a ID if (log.isdebugenabled ()) Log.debug (GetName () + "Got value #" + ID);//Get from Call The corresponding call call call = Calls.get (ID); Judge the result status int state = In.readint (); Read call status if (state = = Status.SUCCESS.state) {Writable value = Reflectionutils.newinstance (Valueclass, conf); Value.readfields (in); Read value call.setvalue (value); Calls.remove (ID); else if (state = = Status.ERROR.state) {call.setexception (new remoteexception (in), Writableutils.readstring (in)); Calls.remove (ID); else if (state = = Status.FATAL.state) {//Close the connection markclosed (new RemoteException writableutils.readstring ( IN), writableutils.readstring (in)); } ..... } catch (IOException e) {markclosed (e);}} Remove from the previously maintained call list and make a judgment. Execution flow of the client itselfThe simplicity of the process comparison:
Hadoop RPC Client Communication module is the part of the above process, in fact, I also ignored a lot of details, we learn, for the source will help to better understand, Hadoop RPC Server implementation of the server is more complex, so it is recommended to use the module of learning may be better.
