Spring+netty+protostuff+zookeeper Implementing lightweight RPC Services (i) __java

Reprint Address: https://my.oschina.net/Listening/blog/682124

Reprint Address: http://www.jb51.net/article/87079.htm

To get a better understanding, refer to two posts, so just post the addresses of two posts.

To avoid the length of the article, I wrote the full source section in the next article, Spring+netty+protostuff+zookeeper to implement lightweight RPC services (ii)

RPC, remote Procedure call, which is generally speaking, calls a service on a remote computer as if it were a local service.

RPC can be based on HTTP or TCP protocol, WEB service is RPC based on HTTP protocol, it has good cross-platform, but its performance is inferior to RPC based on TCP protocol. Two aspects will directly affect the performance of RPC, one is the transmission mode, the second is serialization.

As we all know, TCP is the Transport Layer protocol, HTTP is the application layer protocol, and the transport layer is lower than the application layer, in the data transmission, the lower the faster, so, in general, TCP must be faster than HTTP. In the case of serialization, Java provides the default serialization method, but in high concurrency, this approach will bring some performance bottlenecks, so there are a series of excellent serialization of the framework, such as: Protobuf, Kryo, Hession, Jackson and so on, They can take the place of Java default serialization, providing more efficient performance.

In order to support high concurrency, traditional blocking IO is obviously not appropriate, so we need asynchronous Io, or NIO. Java provides a solution for NIO, and Java 7 provides better nio.2 support, and using Java to implement NIO is not a remote thing, but requires us to be familiar with the technical details of NIO.

We need to deploy the services to different nodes in the distributed environment, to enable clients to automatically discover and invoke services that are currently available through the way the service is registered. This requires a service Registry component that registers the service addresses (including host names and port numbers) in the distributed environment.

The relationship between the application, service, and service registry is as follows:

There are multiple service releases on each server, which share a host and port and provide Server common service in a distributed environment. In addition, to prevent a single point of failure in Service Registry, it needs to be built into a clustered environment.

This article will show you the process of developing a lightweight distributed RPC framework based on the TCP protocol that provides NIO features, provides efficient serialization, and also the ability to register and discover services.

According to the above technical requirements, we can use the following technology selection: Spring: It is the most powerful dependency injection framework, but also the industry's authoritative standard Netty: It makes NIO programming easier, masking the Java low-level NIO details Protostuff: It is based on the PROTOBUF serialization framework , for POJO, no need to write. proto file Zookeeper: Provide service registration and discovery function, develop the necessary choice of distributed system, at the same time it also has the innate cluster ability.

To introduce the general steps of this example, the relevant source code will be detailed in the article listed later in This example steps to introduce the first step: Writing Service Interface

Public interface HelloService {
  string hello (string name);
}

Place the interface in a separate client jar package for use by the application. Step Two: Write the implementation class for the service interface

@RpcService (Helloservice.class)//Specify the remote interface public
class Helloserviceimpl implements HelloService {
  @Override Public
  String, hello (String name) {return
    ' hello! ' + name;}
}

Defining the implementation class for the service interface using the Rpcservice annotation requires specifying a remote interface for the implementation class, because the implementation class may implement multiple interfaces, and be sure to tell the framework that the remote interface is the one.

The Rpcservice code is as follows:

@Target ({elementtype.type})
@Retention (retentionpolicy.runtime)
@Component//indicates that Spring can be scanned for public
@ Interface Rpcservice {
  class<?> value ();
}

This annotation has the characteristics of spring's Component annotation and can be scanned by spring.
The implementation class is placed in the server-side jar package, which also provides some server-side configuration files and boot programs to start the service. Step Three: Configure the service side

The service-side Spring configuration file is named Applicationcontext.xml and reads as follows:

<beans ...>
  <context:component-scan base-package= "Com.xxx.server"/> <context

  : Property-placeholder location= "Classpath:config.properties"/>

  <!--Configure Service Registration component--> <bean id=
  " Serviceregistry "class=" Com.xxx.registry.ServiceRegistry ">
    <constructor-arg name=" Registryaddress " Value= "${registry.address}"/>
  </bean>

  <!--configure RPC server-->
  <bean id= "Rpcserver" Com.xxx.server.RpcServer ">
    <constructor-arg name=" serveraddress "value=" ${server.address} "/>
    <constructor-arg name= "Serviceregistry" ref= "Serviceregistry"/>
  </bean>
</beans>

The specific configuration parameters are in the Config.properties file, which reads as follows:

# Zookeeper Server
registry.address=127.0.0.1:2181

# RPC Server
server.address=127.0.0.1:8000

The above configuration indicates that the local zookeeper server is connected and the RPC service is published on Port 8000. Step Fourth: Start the server-side Publishing Service

To load the Spring configuration file to publish the service, you only need to write a bootstrapper:

public class Rpcbootstrap {public

  static void Main (string[] args) {
    new Classpathxmlapplicationcontext (" Applicationcontext.xml ");
  }
}

Run the main method of the Rpcbootstrap class to start the server, but there are two important components that are not implemented, namely, Serviceregistry and Rpcserver, which give specific implementation details. Fifth Step: Implement service registration

The service registration function can be easily implemented using the Zookeeper customer list, Serviceregistry code is as follows:

public class Serviceregistry {private static final Logger Logger = Loggerfactory.getlogger (Serviceregistry.class);

  Private Countdownlatch latch = new Countdownlatch (1);

  Private String registryaddress;
  Public Serviceregistry (String registryaddress) {this.registryaddress = registryaddress;
      public void register (String data) {if (data!= null) {Zookeeper ZK = ConnectServer ();
      if (ZK!= null) {CreateNode (ZK, data);
    }} Private Zookeeper ConnectServer () {zookeeper ZK = null; try {ZK = new zookeeper (registryaddress, Constant.zk_session_timeout, New Watcher () {@Override Pub LIC void Process (Watchedevent event) {if (event.getstate () = = Event.KeeperState.SyncConnected) {LA
          Tch.countdown ();
      }
        }
      });
    Latch.await (); catch (IOException |
    Interruptedexception e) {logger.error ("", e);
  } return to ZK; } private void CreateNode (ZoOkeeper ZK, String data) {try {byte[] bytes = Data.getbytes ();
      String path = zk.create (Constant.zk_data_path, Bytes, ZooDefs.Ids.OPEN_ACL_UNSAFE, createmode.ephemeral_sequential);
    Logger.debug ("Create Zookeeper node ({} => {})", Path, data); catch (Keeperexception |
    Interruptedexception e) {logger.error ("", e); }
  }
}

Where all the constants are configured via Constant:

Public interface Constant {

  int zk_session_timeout = 5000;

  String Zk_registry_path = "/registry";
  String Zk_data_path = Zk_registry_path + "/data";
}

Note: First you need to create a/registry permanent node using the Zookeeper Client command line to hold all the service temp nodes. Step Sixth: Implementing the RPC server

Using Netty to implement an RPC server that supports NIO, you need to use Serviceregistry to register the service address, Rpcserver code as follows:

public class Rpcserver implements Applicationcontextaware, Initializingbean {private static final Logger Logger = Logg

  Erfactory.getlogger (Rpcserver.class);
  Private String serveraddress;

  Private Serviceregistry serviceregistry; Private map<string, object> handlermap = new hashmap<> ();
  Holds the mapping relationship between the interface name and the service object public rpcserver (String serveraddress) {this.serveraddress = serveraddress;
    Public Rpcserver (String serveraddress, Serviceregistry serviceregistry) {this.serveraddress = serveraddress;
  This.serviceregistry = Serviceregistry; @Override public void Setapplicationcontext (ApplicationContext ctx) throws Beansexception {map<string, obje ct> Servicebeanmap = ctx.getbeanswithannotation (Rpcservice.class); Gets all Spring Bean if Maputils.isnotempty (servicebeanmap) with Rpcservice annotations {for Object Servicebean:servic Ebeanmap.values ()) {String InterfaceName = Servicebean.getclass (). Getannotation (Rpcservice.class). Value (). GetName ();
      Handlermap.put (InterfaceName, Servicebean); @Override public void Afterpropertiesset () throws Exception {Eventloopgroup Bossgroup = new Nioevent
    Loopgroup ();
    Eventloopgroup Workergroup = new Nioeventloopgroup ();
      try {serverbootstrap bootstrap = new Serverbootstrap (); Bootstrap.group (Bossgroup, Workergroup). Channel (Nioserversocketchannel.class). Childhandler (New ChannelInitialize 
            R<socketchannel> () {@Override public void Initchannel (Socketchannel channel) throws Exception {
              Channel.pipeline (). AddLast (New Rpcdecoder (Rpcrequest.class))//decoding RPC requests (for processing requests) . AddLast (New Rpcencoder (Rpcresponse.class))//encodes the RPC response (in order to return the response). AddLast (New Rpchandler (HANDLERMAP) ); Processing RPC Request}}. Option (Channeloption.so_backlog, 128). Childoption (Channeloption.so_k

      Eepalive, True); string[] array = SERveraddress.split (":");
      String host = array[0];

      int port = integer.parseint (array[1]);
      Channelfuture future = Bootstrap.bind (host, port). sync ();

      Logger.debug ("server started on port {}", port); if (serviceregistry!= null) {serviceregistry.register (serveraddress);//Register service address} future.channel ().
    Closefuture (). sync ();
      finally {workergroup.shutdowngracefully ();
    Bossgroup.shutdowngracefully (); }
  }
}

In the above code, there are two important POJO to describe, they are rpcrequest and rpcresponse respectively.
Use Rpcrequest to encapsulate RPC requests with the following code:

public class Rpcrequest {

  private String RequestID;
  Private String className;
  Private String methodname;
  Private class<?>[] parametertypes;
  Private object[] Parameters;

  Getter/setter ...
}

The RPC response is encapsulated using the Rpcresponse code as follows:

public class Rpcresponse {

  private String RequestID;
  Private Throwable error;
  Private Object result;

  Getter/setter ...
}

To provide RPC decoding using Rpcdecoder, simply extend the Decode method of the Netty Bytetomessagedecoder abstract class, as follows:

public class Rpcdecoder extends Bytetomessagedecoder {

  private class<?> genericclass;

  Public Rpcdecoder (class<?> genericclass) {
    this.genericclass = Genericclass;
  }

  @Override public
  void Decode (Channelhandlercontext ctx, bytebuf in, list<object> out) throws Exception {
    if (In.readablebytes () < 4) {return
      ;
    }
    In.markreaderindex ();
    int datalength = In.readint ();
    if (Datalength < 0) {
      ctx.close ();
    }
    if (In.readablebytes () < datalength) {
      in.resetreaderindex ();
      return;
    }
    byte[] data = new Byte[datalength];
    In.readbytes (data);

    Object obj = serializationutil.deserialize (data, genericclass);
    Out.add (obj);
  }

Using Rpcencoder to provide RPC encoding, you only need to extend the Encode method of the Netty Messagetobyteencoder abstract class, the code reads as follows:

public class Rpcencoder extends Messagetobyteencoder {

  private class<?> genericclass;

  Public Rpcencoder (class<?> genericclass) {
    this.genericclass = Genericclass;
  }

  @Override public
  void Encode (Channelhandlercontext ctx, Object in, bytebuf out) throws Exception {
    if (GENERICCLA Ss.isinstance (in)) {
      byte[] data = Serializationutil.serialize (in);
      Out.writeint (data.length);
      Out.writebytes (data);}}

Write a Serializationutil tool class that uses Protostuff to implement serialization:

public class Serializationutil {private static map<class<?>, schema<?>> cachedschema = new Concurren

  Thashmap<> ();

  private static Objenesis objenesis = new OBJENESISSTD (true); Private Serializationutil () {} @SuppressWarnings ("Unchecked") private static <T> schema<t> GetSchema (C
    Lass<t> cls) {schema<t> Schema = (schema<t>) cachedschema.get (CLS);
      if (schema = = null) {schema = Runtimeschema.createfrom (CLS);
      if (schema!= null) {Cachedschema.put (CLS, schema);
  } return schema; @SuppressWarnings ("unchecked") public static <T> byte[] Serialize (T obj) {class<t> cls = (class&lt ;
    t>) Obj.getclass ();
    Linkedbuffer buffer = linkedbuffer.allocate (linkedbuffer.default_buffer_size);
      try {schema<t> Schema = GetSchema (CLS);
    return Protostuffioutil.tobytearray (obj, schema, buffer); catch (Exception e) {throw new IllegalstaTeexception (E.getmessage (), E);
    finally {buffer.clear (); } public static <T> T deserialize (byte[] data, class<t> CLS) {try {t-message = (t) objenesi
      S.newinstance (CLS);
      schema<t> schema = GetSchema (CLS);
      Protostuffioutil.mergefrom (data, message, schema);
    return message;
    catch (Exception e) {throw new IllegalStateException (E.getmessage (), E); }
  }
}

The above uses the Objenesis to instantiate the object, which is more powerful than the Java reflection.
Note: If you need to replace other serialization frames, simply modify the Serializationutil. Of course, the better way to do this is to provide configuration items to determine which serialization method to use.

To process RPC requests using Rpchandler, simply extend the Netty Simplechannelinboundhandler abstract class, as follows:

public class Rpchandler extends simplechannelinboundhandler<rpcrequest> {private static final Logger Logger = Lo

  Ggerfactory.getlogger (Rpchandler.class);

  Private final map<string, object> Handlermap;
  Public Rpchandler (map<string, object> handlermap) {this.handlermap = Handlermap; @Override public void channelRead0 (final channelhandlercontext CTX, rpcrequest request) throws Exception {RPCR
    Esponse response = new Rpcresponse ();
    Response.setrequestid (Request.getrequestid ());
      try {Object result = handle (request);
    Response.setresult (result);
    catch (Throwable t) {response.seterror (t);
  } ctx.writeandflush (response). AddListener (Channelfuturelistener.close);
    Private Object handle (Rpcrequest request) throws Throwable {String className = Request.getclassname ();

    Object Servicebean = Handlermap.get (className);
    class<?> ServiceClass = Servicebean.getclass (); String methodname = RequeSt.getmethodname ();
    class<?>[] parametertypes = Request.getparametertypes ();

    object[] Parameters = Request.getparameters ();
    /*method method = Serviceclass.getmethod (methodname, parametertypes);
    Method.setaccessible (TRUE);
    Return Method.invoke (Servicebean, parameters); * Fastclass Servicefastclass = fastclass.create (ServiceClass);
    Fastmethod Servicefastmethod = Servicefastclass.getmethod (methodname, parametertypes);
  Return Servicefastmethod.invoke (Servicebean, parameters);  @Override public void Exceptioncaught (Channelhandlercontext ctx, throwable cause) {logger.error ("server caught
    Exception ", cause);
  Ctx.close (); }
}

To avoid performance problems with Java reflection, we can use the reflection APIs provided by Cglib, such as the Fastclass and Fastmethod used above. Step Seventh: Configure the client

Also using the Spring configuration file to configure the RPC client, the Applicationcontext.xml code is as follows:

<beans ...>
  <context:property-placeholder location= "classpath:config.properties"/>

  <!-- Configure Service Discovery Component-->
  <bean id= "servicediscovery" class= "Com.xxx.registry.ServiceDiscovery" >
    < Constructor-arg name= "registryaddress" value= "${registry.address}"/>
  </bean>

  <!--configure RPC proxy- >
  <bean id= "RpcProxy" class= "Com.xxx.client.RpcProxy" >
    <constructor-arg name= " Servicediscovery "ref=" Servicediscovery "/>
  </bean>
</beans>

The config.properties provides a specific configuration:

# Zookeeper Server
registry.address=127.0.0.1:2181

Eighth step: realize Service Discovery

Also use Zookerper to implement the service discovery function, see the following code:

public class Servicediscovery {private static final Logger Logger = Loggerfactory.getlogger (Servicediscovery.class);

  Private Countdownlatch latch = new Countdownlatch (1);

  Private volatile list<string> dataList = new arraylist<> ();

  Private String registryaddress;

    Public Servicediscovery (String registryaddress) {this.registryaddress = registryaddress;
    Zookeeper ZK = ConnectServer ();
    if (ZK!= null) {Watchnode (ZK);
    The public string discover () {string data = null;
    int size = Datalist.size ();
        if (Size > 0) {if (size = = 1) {data = Datalist.get (0);
      Logger.debug ("Using only data: {}", data);
        else {data = Datalist.get (Threadlocalrandom.current (). Nextint (size));
      Logger.debug ("Using random data: {}", data);
  } return data;
    Private Zookeeper ConnectServer () {zookeeper ZK = null; try {ZK = new zookeeper (registryaddress, constant.zk_session_timeout, New Watcher () {@Override public void process (Watchedevent event) {if (event.getstate
          () = = Event.KeeperState.SyncConnected) {latch.countdown ();
      }
        }
      });
    Latch.await (); catch (IOException |
    Interruptedexception e) {logger.error ("", e);
  } return to ZK; } private void Watchnode (Final zookeeper ZK) {try {list<string> nodelist = Zk.getchildren (constant.zk _registry_path, New Watcher () {@Override public void process (Watchedevent event) {if (event.ge
          Ttype () = = Event.EventType.NodeChildrenChanged) {watchnode (ZK);
      }
        }
      });
      list<string> dataList = new arraylist<> ();
        for (String node:nodelist) {byte[] bytes = zk.getdata (Constant.zk_registry_path + "/" + node, false, NULL);
      Datalist.add (New String (bytes)); } logger.debug ("node data: {}&quo