Code structure analysis for Hadoop IPC

The IPC-related code is under the ORG.APACHE.HADOOP.IPC package. A total of seven files, 4 of which are helper classes:

RemoteException

Status

Versionedprotocol

Connectionheader

The main implementation of the Class 3:

Client

Server

Rpc

Client clients:

Such as:

Related to IPC connections

• Client.connection
• Client.connectionid
• Connectionheader

Associated with calling call remotely.

• Client.call
• Client.parallelcall
• Client.parallelresults

Server-side server:

Related to IPC connections

• Server.connection
• Connectionheader

Associated with calling call remotely.

• Server.call
• Server.responder
• Server.listener
• Server.handler

Rpc

RPC is the implementation of the Hadoop IPC on the basis of server and client.

Client-related features:

• Rpc. Clientcache
• Rpc. Invoker (inheriting Java.lang.reflect.InvocationHandler)
• Rpc. Invocation

Features related to the service side:

• Rpc. Server

Connection

The client and server side are different from the abstraction of the connection, so there are server.connection and client.connection. Hadoop remote invocation uses TCP protocol communication.

1) Client Client.connectionid

Connection multiplexing: When multiple IPC clients are connectionid together, they share an IPC connection. Connection multiplexing reduces the resource usage of Hadoop Server, client, and the IPC connection time.

2) Connectionheader

The first message that is exchanged after the TCP connection between the server and the client is established, including the Connectionid.ticket (usergroupinformation) user information and the IPC interface information, to verify that the IPC interface is implemented, And whether the user has permission to use the interface.

Pager

After the connection is established, the remote procedure Call service can be made, that is, the invocation of the IPC interface method, the source code is abstracted as call.

Remote invocation of the Client.call object and the Server.call object, which is generated by an IPC call, exists in the IPC client (stub) and the IPC service side (skeleton) of the entity.

After the Client.call object is connected to the server through the IPC, it naturally forms the corresponding Server.call object.

When and how does client.call occur?

As shown in the process:

1. User initiated remote interface call
2. Dynamic Proxy, RPC. Invoker call handle captures a remote call
3. Generates a Rpc.invocation object based on the input parameter of Invoke method and args
4. And calling Client.call,call will serialize the invocation object from the previous step and send it to the server via an IPC connection. Client.call will wait for the result returned by the server.
5. The server-side listener listens for connection requests and data requests from the client and invokes the server-side Connection object.
6. The connection object receives a remote call request frame, deserializes it, and places the request in the blocking queue, which is handled by handler.
7. The handler invokes the corresponding IPC interface implementation class, completes the procedure call, and serializes the result.
8. If the answer queue for the connection is empty at this point, it is returned to the client.
9. Otherwise, the client is busy, the answering queue is not empty, handler the result into the response queue, and sent by Responser to the client via IPC.

IPC Connection

• Connection creation
• Data read and write on the connection
• Connection Maintenance
• Connection Close

The IPC connection code on the server side is dispersed in listener and server.connection.

Listener.run () implements the selector loop in NiO. The following code:

//Listener constructor Function PublicListener ()throwsIOException {Address=Newinetsocketaddress (bindaddress, Port); //Create A new server socket and set to non blocking modeAcceptchannel =Serversocketchannel.open (); Acceptchannel.configureblocking (false); //Bind The server socket to the local host and Portbind (Acceptchannel.socket (), address, backloglength); Port= Acceptchannel.socket (). Getlocalport ();//Could is an ephemeral port//Create a selector;Selector=Selector.open (); Readers=NewReader[readthreads]; Readpool=Executors.newfixedthreadpool (readthreads);  for(inti = 0; i < readthreads; i++) {Selector readselector=Selector.open (); Reader Reader=NewReader (Readselector); Readers[i]=reader;      Readpool.execute (reader); }

Listener.run () turns on the selector loop and processes the accept request as follows:

//Listener Run function Public voidrun () {Log.info (GetName ()+ ": Starting"); Server.set (Server. This);  while(running) {Selectionkey key=NULL; Try{selector.select (); Iterator<SelectionKey> iter =Selector.selectedkeys (). iterator ();  while(Iter.hasnext ()) {Key=Iter.next ();            Iter.remove (); Try {              if(Key.isvalid ()) {if(Key.isacceptable ()) doaccept (key); }            } Catch(IOException e) {} key=NULL; }        } Catch(OutOfMemoryError e) {//we can run out of the memory if we have too many threads//log the event and sleep for a minute and give//some thread (s) a chance to finishLog.warn ("Out of Memory in server select", E);          Closecurrentconnection (key, E); Cleanupconnections (true); Try{Thread.Sleep (60000);}Catch(Exception IE) {}}Catch(Exception e) {closecurrentconnection (key, E); } cleanupconnections (false); } log.info ("Stopping" + This. GetName ()); synchronized( This) {        Try{acceptchannel.close ();        Selector.close (); } Catch(IOException e) {} selector=NULL; Acceptchannel=NULL; //Clean up all connections         while(!Connectionlist.isempty ()) {CloseConnection (Connectionlist.remove (0)); }      }    }

Doaccept () Gets the Socketchannel through Server.accpet and gets a reader object that contains a selector:readerselector, With Reader.registerchannel, register Socketchannel under Readerselector. and create a new Connection object.

//do_acceptvoidDoaccept (Selectionkey key)throwsIOException, OutOfMemoryError {Connection C=NULL; Serversocketchannel Server=(Serversocketchannel) Key.channel ();      Socketchannel Channel;  while(channel = Server.accept ())! =NULL) {channel.configureblocking (false);        Channel.socket (). Settcpnodelay (Tcpnodelay); Reader Reader=Getreader (); Try{reader.startadd (); Selectionkey ReadKey=Reader.registerchannel (channel); C=NewConnection (ReadKey, Channel, System.currenttimemillis ());          Readkey.attach (c); synchronized(connectionlist) {Connectionlist.add (numconnections, C); Numconnections++; }          if(log.isdebugenabled ()) Log.debug ("Server connection from" + c.tostring () + "; # Active Connections: "+ numconnections +"; # Queued Calls: "+callqueue.size ()); } finally{reader.finishadd (); }      }    }

 Public synchronized Selectionkey RegisterChannel (Socketchannel channel)                                                           throws IOException {          return  channel.register (Readselector, selectionkey.op_read);      }

Code structure analysis for Hadoop IPC