Development of Netty program based on Protostuff codec technology: Transmitting Pojo Object

[TOC]

Development of Netty program based on Protostuff codec technology: The preface of Transmitting Pojo object

The development of the Netty program is mainly the transmission of Java objects in the network, the transmission of objects not only limited to strings, but also can be customized other types of objects.

The previous use of Protostuff are relatively simple to use, to carry out a few simple tests, the following to complete the example of the function is not complex, but relatively use will be more comprehensive. The development of the serialization tool class, the development of the codec, and then apply it to our Netty program.

The Netty program is developed to transmit Pojo objects in preparation for the development of the remote Procedure call framework later, because the returned result is an object when the remote call is made, depending on the method being called, and the type of object returned. Therefore, transferring Pojo objects through the Netty program is only a small part of the development of a custom RPC framework, but it is also very important.

Also note that the use of the Netty framework itself is important, and how to develop a serialization tool class (that is, how to develop an enabling serialization tool through Protostuff), how to develop Netty encoder and decoder based on the serialization tool class, How to use custom developed encoders and decoders in Netty this knowledge is 10 important and must be mastered.

The code has been written in the comments, the program is able to run directly, dependent on the relevant package, because the use of MAVEN project, so in the back will also give the contents of the Pom.xml file.

Development of Protostuff Serialization tool class

The tool class development process can refer to the previous article based on the Protostuff serialization tool class development, the following directly gives the code of the serialization tool class with caching capabilities:

Serializationutil2.java

Package Cn.xpleaf.protostuff.netty.utils;import Java.util.map;import Java.util.concurrent.ConcurrentHashMap; Import Com.dyuproject.protostuff.linkedbuffer;import Com.dyuproject.protostuff.protostuffioutil;import com.dyuproject.protostuff.runtime.runtimeschema;/** * Caching-enabled serialization tool class based on protostuff implementation (based on Google protobuf) * * @author Yeyonghao * */public class SerializationUtil2 {//Cache schema Object Map private static map<class<?>, Runtimeschem    a<?>> Cachedschema = new Concurrenthashmap<class<?>, runtimeschema<?>> ();     /** * According to the schema method of obtaining the corresponding type * * @param clazz * @return */@SuppressWarnings ({"Unchecked", "unused"})        Private <T> runtimeschema<t> GetSchema (class<t> clazz) {//try to get the appropriate type of schema from the cache schema map first        runtimeschema<t> schema = (runtimeschema<t>) cachedschema.get (clazz); If the corresponding schema is not obtained, create a schema of that type//add it to the schema map at the same time if (schema = = null) {SCHema = Runtimeschema.createfrom (Clazz);            if (schema! = null) {Cachedschema.put (clazz, schema);    }}//Return schema object return schema; }/** * Serialization method that serializes an object into a byte array (object---> Byte array) * * @param obj * @return */@SuppressWarnings ("Unche cked ") public static <T> byte[] Serialize (T obj) {//Get type of generic object class<t> Clazz = (class<t&        gt;) Obj.getclass ();        Create a schema object for a generic object runtimeschema<t> schema = Runtimeschema.createfrom (Clazz);        Create Linkedbuffer object Linkedbuffer buffer = linkedbuffer.allocate (linkedbuffer.default_buffer_size);        Serialized byte[] Array = Protostuffioutil.tobytearray (obj, schema, buffer);    Returns the serialized object return array; }/** * Deserialization method to deserialize a byte array into an object (byte array---> Object) * * @param data * @param clazz * @return */PU Blic static <T> T deserialize (byte[] data, class<t> clazz) {//GenThe schema object of the generic object runtimeschema<t> schema = Runtimeschema.createfrom (Clazz);        Instantiate object based on schema T message = Schema.newmessage ();        Deserializes the data in a byte array into a Message object Protostuffioutil.mergefrom (data, message, schema);    Returns the deserialized object return message; }}

Encoder and Decoder Development Echoencoder.java

package cn.xpleaf.protostuff.netty.utils;import io.netty.buffer.ByteBuf;import io.netty.channel.ChannelHandlerContext;import io.netty.handler.codec.MessageToByteEncoder;/** * PojoEncoder继承自Netty中的MessageToByteEncoder类， * 并重写抽象方法encode(ChannelHandlerContext ctx, Object msg, ByteBuf out) * 它负责将Object类型的POJO对象编码为byte数组，然后写入到ByteBuf中 *  * @author yeyonghao * */public class EchoEncoder extends MessageToByteEncoder<Object> {    @Override    protected void encode(ChannelHandlerContext ctx, Object msg, ByteBuf out) throws Exception {        // 直接生成序列化对象        // 需要注意的是，使用protostuff序列化时，不需要知道pojo对象的具体类型也可以进行序列化时        // 在反序列化时，只要提供序列化后的字节数组和原来pojo对象的类型即可完成反序列化        byte[] array = SerializationUtil2.serialize(msg);        out.writeBytes(array);    }}

Echodecoder.java

Package Cn.xpleaf.protostuff.netty.utils;import Java.util.list;import Io.netty.buffer.bytebuf;import Io.netty.channel.channelhandlercontext;import io.netty.handler.codec.messagetomessagedecoder;/** * Pojodecoder inherits from the Messagetomessagedecoder class in Netty, * and overrides the abstract method decode (Channelhandlercontext ctx, Bytebuf msg, list< Object> out) * First gets the byte array that needs to be decoded from the datagram msg (the generic type that is populated when the data type depends on inheriting messagetomessagedecoder) *  The call is then deserialized (decoded) using the serialization tool class to add the decoded object to the decoded list out, which completes the decoding operation * * @author Yeyonghao * */public class Echodecoder extends    messagetomessagedecoder<bytebuf> {//requires the type to which the deserialization object belongs private class<?> genericclass;    Constructs a method, passing in the type that needs to deserialize the object public Echodecoder (class<?> genericclass) {this.genericclass = Genericclass;         } @Override protected void decode (Channelhandlercontext ctx, Bytebuf msg, list<object> out) throws Exception {        Length of bytebuf int length = Msg.readablebytes ();        Constructs a byte array of length lengths byte[] array = new Byte[length]; //Copy the BYTEBUF data into a byte array msg.readbytes (array);        Deserializes Object obj = serializationutil2.deserialize (array, this.genericclass);    Added to the list of deserialized object results out.add (obj); }}

Netty Service-side program development Echoserver.java

Package Cn.xpleaf.protostuff.netty.echoservice;import Cn.xpleaf.protostuff.netty.pojo.echorequest;import Cn.xpleaf.protostuff.netty.utils.echodecoder;import Cn.xpleaf.protostuff.netty.utils.echoencoder;import Io.netty.bootstrap.serverbootstrap;import Io.netty.channel.channelfuture;import Io.netty.channel.channelinitializer;import Io.netty.channel.channeloption;import Io.netty.channel.EventLoopGroup ; Import Io.netty.channel.nio.nioeventloopgroup;import Io.netty.channel.socket.socketchannel;import Io.netty.channel.socket.nio.nioserversocketchannel;public class Echoserver {public void bind (int port) throws Exceptio        n {//config service-side NIO thread Group Eventloopgroup bossgroup = new Nioeventloopgroup ();        Eventloopgroup Workergroup = new Nioeventloopgroup ();            try {serverbootstrap b = new Serverbootstrap (); B.group (Bossgroup, Workergroup). Channel (nioserversocketchannel.class). Option (channeloption .              So_backlog, 1024)  . Childhandler (New channelinitializer<socketchannel> () {@Override protected void Initchannel (Socketchannel ch) throws Exception {//Add encoder ch.pipeline ()                        . AddLast (New Echodecoder (Echorequest.class));                        Add Decoder Ch.pipeline (). AddLast (New Echoencoder ());                    Add Business Processing Handler ch.pipeline (). AddLast (New Echoserverhandler ());            }                });            Bind port, the synchronization waits successfully, the method is synchronous blocking, the binding succeeds and returns a channelfuture channelfuture f = b.bind (port). sync ();        Wait for the service-side listener port to close, block, wait for the service-side link to close after the main function exits F.channel (). Closefuture (). sync ();            } finally {//gracefully exits, releasing thread pool resource bossgroup.shutdowngracefully ();        Workergroup.shutdowngracefully ();        }} public static void Main (string[] args) throws Exception {int port = 8080; if (args! = null &Amp;& args.length > 0) {try {port = integer.valueof (port); } catch (NumberFormatException e) {//Todo:handle exception}} new Echoserver ()    . bind (port); }}

Echoserverhandler.java

Package Cn.xpleaf.protostuff.netty.echoservice;import Java.util.uuid;import Cn.xpleaf.protostuff.netty.pojo.echorequest;import Cn.xpleaf.protostuff.netty.pojo.echoresponse;import Cn.xpleaf.protostuff.netty.pojo.user;import Io.netty.channel.channelhandlercontext;import    Io.netty.channel.channelinboundhandleradapter;public class Echoserverhandler extends Channelinboundhandleradapter { @Override public void Channelread (Channelhandlercontext ctx, Object msg) throws Exception {//The type of object received is Echoreq        Uest echorequest req = (echorequest) msg;        System.out.println (Req.getrequestid () + ":" + req.getrequestobj ());        Create a user object that needs to be transferred by user user = new user ();        User.setname ("server");        User.setage (10);        Create a transport User object carrier Echorequest object Echoresponse resp = new Echoresponse ();        Set Responseid Resp.setresponseid (Uuid.randomuuid (). toString ());        Set the object to be transferred resp.setresponseobj (user); Sets the type of object that needs to be transferred RESP.SEtresponseobjclass (Resp.getresponseobj (). GetClass ());    Call Writeandflush to send data to Socketchannel Ctx.writeandflush (RESP);    } @Override public void Channelreadcomplete (Channelhandlercontext ctx) throws Exception {Ctx.flush ();    } @Override public void Exceptioncaught (Channelhandlercontext ctx, throwable cause) {ctx.close (); }}

Netty Client program Development Echoclient.java

Package Cn.xpleaf.protostuff.netty.echoservice;import Cn.xpleaf.protostuff.netty.pojo.echoresponse;import Cn.xpleaf.protostuff.netty.utils.echodecoder;import Cn.xpleaf.protostuff.netty.utils.echoencoder;import Io.netty.bootstrap.bootstrap;import Io.netty.channel.channelfuture;import Io.netty.channel.ChannelInitializer; Import Io.netty.channel.channeloption;import Io.netty.channel.eventloopgroup;import Io.netty.channel.nio.nioeventloopgroup;import Io.netty.channel.socket.socketchannel;import Io.netty.channel.socket.nio.niosocketchannel;public class Echoclient {public void connect (int port, String host) throw        s Exception {//Configure client NIO thread Group Eventloopgroup group = new Nioeventloopgroup ();            try {Bootstrap b = new Bootstrap (); B.group (Group). Channel (niosocketchannel.class). Option (Channeloption.tcp_nodelay, True)//Set Timeout for TCP connections. Option (Channeloption.connect_timeout_millis, +). Handler (NEW channelinitializer<socketchannel> () {@Override protected void Initchannel (S Ocketchannel ch) throws Exception {//Add decoder ch.pipeline (). AddLast (New Echod                        Ecoder (Echoresponse.class));                        Add Encoder Ch.pipeline (). AddLast (New Echoencoder ());                    Add Business Processing Handler ch.pipeline (). AddLast (New Echoclienthandler ());            }                });            Initiates an asynchronous connection operation (Note that the server is bind and the client needs connect) Channelfuture F = b.connect (host, port). sync ();        Wait for the client link to close F.channel (). Closefuture (). sync ();        } finally {//gracefully exits, releasing the NIO thread Group group.shutdowngracefully ();        }} public static void Main (string[] args) throws Exception {int port = 8080; if (args! = null && args.length > 0) {try {port = integer.valueof (port);            } catch (NumberFormatException e) {//Takes default}} new Echoclient (). Conn    ECT (port, "localhost"); }}

Echoclienthandler.java

Package Cn.xpleaf.protostuff.netty.echoservice;import Java.util.uuid;import Cn.xpleaf.protostuff.netty.pojo.echorequest;import Cn.xpleaf.protostuff.netty.pojo.echoresponse;import Cn.xpleaf.protostuff.netty.pojo.user;import Io.netty.channel.channelhandlercontext;import    Io.netty.channel.channelinboundhandleradapter;public class Echoclienthandler extends Channelinboundhandleradapter {         @Override public void channelactive (Channelhandlercontext ctx) {//Create user object to transfer user user = new user ();        User.setname ("Client");        User.setage (10);        Create a transport User object carrier Echorequest object Echorequest req = new Echorequest ();        Set RequestID Req.setrequestid (Uuid.randomuuid (). toString ());        Set the object to be transferred req.setrequestobj (user);        Sets the type of object to be transferred Req.setrequestobjclass (Req.getrequestobj (). GetClass ());    Call Writeandflush to send data to Socketchannel Ctx.writeandflush (req); } @Override public void Channelread (ChannelhandlerconText CTx, Object msg) throws Exception {//The type of object received is Echoresponse echoresponse resp = (echoresponse) msg;    System.out.println (Resp.getresponseid () + ":" + resp.getresponseobj ());    } @Override public void Channelreadcomplete (Channelhandlercontext ctx) throws Exception {Ctx.flush (); } @Override public void Exceptioncaught (Channelhandlercontext ctx, Throwable cause) throws Exception {ctx.cl    OSE (); }}

Pojoechorequest.java

package cn.xpleaf.protostuff.netty.pojo;/** * EchoRequest是client向server端发送数据的传输载体，将需要进行传输的pojo对象统一封装到EchoRequest对象中， * 这样会为编解码工作带来很大的方便性和统一性，同时也可以携带其它信息， 对于后面对程序进行扩展会有非常大的帮助 *  * @author yeyonghao * */public class EchoRequest {    private String requestId;    private Object requestObj;    private Class<?> requestObjClass;    public String getRequestId() {        return requestId;    }    public void setRequestId(String requestId) {        this.requestId = requestId;    }    public Object getRequestObj() {        return requestObj;    }    public void setRequestObj(Object requestObj) {        this.requestObj = requestObj;    }    public Class<?> getRequestObjClass() {        return requestObjClass;    }    public void setRequestObjClass(Class<?> requestObjClass) {        this.requestObjClass = requestObjClass;    }}

Echoresponse.java

package cn.xpleaf.protostuff.netty.pojo;/** * EchoResponse是server向client端发送数据的传输载体，将需要进行传输的pojo对象统一封装到EchoResponse对象中， * 这样会为编解码工作带来很大的方便性和统一性，同时也可以携带其它信息， 对于后面对程序进行扩展会有非常大的帮助 *  * @author yeyonghao * */public class EchoResponse {    private String responseId;    private Object responseObj;    private Class<?> responseObjClass;    public String getResponseId() {        return responseId;    }    public void setResponseId(String responseId) {        this.responseId = responseId;    }    public Object getResponseObj() {        return responseObj;    }    public void setResponseObj(Object responseObj) {        this.responseObj = responseObj;    }    public Class<?> getResponseObjClass() {        return responseObjClass;    }    public void setResponseObjClass(Class<?> responseObjClass) {        this.responseObjClass = responseObjClass;    }}

User.java

package cn.xpleaf.protostuff.netty.pojo;public class User {    private String name;    private int age;    public User() {    }    public User(String name, int age) {        this.name = name;        this.age = age;    }    public String getName() {        return name;    }    public void setName(String name) {        this.name = name;    }    public int getAge() {        return age;    }    public void setAge(int age) {        this.age = age;    }    @Override    public String toString() {        return "User [name=" + name + ", age=" + age + "]";    }}

Test

Execute separately EchoServer.java EchoClient.java , and the server and client outputs are as follows:

Service side:

4b76d70d-7a31-4738-8daa-ca4f40483e7e : User [name=client, age=10]

Client:

e40b6e34-33a3-485e-bb8f-7157ee324e97 : User [name=server, age=10]

Appendix: Pom.xml

<project xmlns= "http://maven.apache.org/POM/4.0.0" xmlns:xsi= "Http://www.w3.org/2001/XMLSchema-instance" xsi: schemalocation= "http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd" > < Modelversion>4.0.0</modelversion> <groupId>cn.xpleaf</groupId> <artifactid>chapter08 </artifactId> <version>0.0.1-SNAPSHOT</version> <dependencies> <!--https://mvnrep Ository.com/artifact/com.google.protobuf/protobuf-java-<dependency> <groupid>com.goog Le.protobuf</groupid> <artifactId>protobuf-java</artifactId> <version>3.5.1&        Lt;/version> </dependency> <!--https://mvnrepository.com/artifact/io.netty/netty-all-- <dependency> <groupId>io.netty</groupId> <artifactid>netty-all</artif Actid> <version>4.1.21.Final</version> </dependency> <!--Https://mvnrepository.com/artifact/com.dyuproject.protostuff/protos            Tuff-core-<dependency> <groupId>com.dyuproject.protostuff</groupId> <artifactId>protostuff-core</artifactId> <version>1.1.3</version> </dependenc Y> <!--https://mvnrepository.com/artifact/com.dyuproject.protostuff/protostuff-runtime-&LT;DEP Endency> <groupId>com.dyuproject.protostuff</groupId> <artifactid>protostuff-ru ntime</artifactid> <version>1.1.3</version> </dependency> &LT;/DEPENDENCIES&G    T <build> <plugins> <!--java Compiler plugin--<plugin> <group                Id>org.apache.maven.plugins</groupid> <artifactId>maven-compiler-plugin</artifactId> <Version>3.2</version> <configuration> <source>1.8</source>                <target>1.8</target> <encoding>UTF-8</encoding> </configuration> </plugin> </plugins> </build></project>

Existing problems and solutions

The above actually transmits the object is EchoRequest and EchoResponse , although it encapsulates the returned object, but because its defined type is the object type, in the above example, it is actually a user object, in this simple example can be transformed by the type of downward transformation, When actually used, the encapsulated object object is not necessarily a user object, but it needs to do a downward transformation, how to solve the problem? This problem can be solved by dynamic agent technology, which is further improved by this example to make it more versatile.

Development of Netty program based on Protostuff codec technology: Transmitting Pojo Object