Mina, Netty, twisted Study Together (v): Integration Protobuf

PROTOBUF is the abbreviation for Google's protocol buffers, which is used to convert between structured data and bytecode (serialization, deserialization), which is generally applied to network transmission and can support multiple programming languages.

Protobuf How to use here no longer introduced, this article mainly introduces in Mina, Netty, twisted How to use PROTOBUF, do not know protobuf students can refer to my other blog post.

In a previous blog post, there is a message segmentation method that uses a fixed 4-byte prefix header to specify the number of bytes in the body, which is also used here. Only the contents of the body are no longer strings, but protobuf bytecode.

When dealing with business logic, you certainly do not want to serialize and deserialize the data, but instead want to manipulate an object directly, then you need to have the appropriate encoder and decoder to write the serialized and deserialized logic in the encoder and decoder. The implementation of encoders and decoders is described in the previous blog post.

The Netty package already has its own encoder and decoder for protobuf, so you don't have to implement it yourself. And Mina, twisted also need to implement PROTOBUF encoder and decoder.

This defines a PROTOBUF data structure that describes a student's information, saved as a Studentmsg.proto file:

Message Student {    //ID    required Int32 id = 1;      Name    Required String name = 2;    Email    Optional String email = 3;    Friend    repeated string friends = 4;}

Use Studentmsg.proto to generate Java and Python code separately, and add the code to the corresponding project. The generated code is no longer posted.

The following describes how to use PROTOBUF to transmit student information in Netty, MINA, and twisted.

Netty:

The Netty comes with PROTOBUF encoder and decoder, respectively Protobufencoder and Protobufdecoder. It is important to note that Protobufencoder and Protobufdecoder are only responsible for the serialization and deserialization of Protobuf, Lengthfieldbasedframedecoder and Lengthfieldprepender are also required to handle message header prefixes and message segmentation. The Lengthfieldbasedframedecoder is used to parse the message header prefix, intercepting the body based on the number of body bytes specified in the header. Lengthfieldprepender is used to specify the number of body bytes by adding a header prefix in front of the message when Wirte messages.

public class TCPServer {public static void main (string[] args) throws Interruptedexception {Eventloopgroup Bos        Sgroup = new Nioeventloopgroup ();        Eventloopgroup Workergroup = new Nioeventloopgroup ();            try {serverbootstrap b = new Serverbootstrap (); B.group (Bossgroup, Workergroup). Channel (Nioserversocketchannel.class). Childhandler (New channelinitializer<socketchannel> () {@Override public void INITC Hannel (Socketchannel ch) throws Exception {Channelpipeline pipe  line = Ch.pipeline (); Responsible for using the body length specified by the 4-byte header to cut the message pipeline.addlast ("Framedecoder", New Lengthfieldbasedframedecoder (1048576, 0, 4, 0, 4)) ;//is responsible for the framedecoder processing of a complete message protobuf bytecode to student object Pipeline.addlast ("Protobufdecoder", New Protobufdecoder ( StudentMsg.Student.getDefaultInstance ()));//responsible for specifying the body length by adding a 4-byte header prefix to the written byte code Pipeline.addlasT ("Frameencoder", New Lengthfieldprepender (4));//responsible for turning student object into Protobuf bytecode pipeline.addlast ("Protobufencoder", new                            Protobufencoder ());                        Pipeline.addlast (New Tcpserverhandler ());            }                    });            Channelfuture f = b.bind (8080). sync ();        F.channel (). Closefuture (). sync ();            } finally {workergroup.shutdowngracefully ();        Bossgroup.shutdowngracefully (); }    }}

When handling events, the parameters that are received and sent are directly student objects:

public class Tcpserverhandler extends Channelinboundhandleradapter {@Override public void Channelread (ChannelHandle        Rcontext ctx, Object msg) {//Read Student object passed by client studentmsg.student Student = (studentmsg.student) msg;        System.out.println ("ID:" + Student.getid ());        System.out.println ("Name:" + student.getname ());        System.out.println ("Email:" + student.getemail ());        System.out.println ("Friends:");        List<string> friends = Student.getfriendslist ();        for (String friend:friends) {System.out.println (friend);        }//Create a new Student object to the client StudentMsg.Student.Builder Builder = StudentMsg.Student.newBuilder ();        Builder.setid (9);        Builder.setname ("server");        Builder.setemail ("[email protected]");        Builder.addfriends ("X");        Builder.addfriends ("Y");        Studentmsg.student Student2 = Builder.build ();    Ctx.writeandflush (Student2); } @Override public void ExceptioncAught (Channelhandlercontext ctx, throwable cause) {cause.printstacktrace ();    Ctx.close (); }}

MINA:

There is no encoder and decoder for protobuf in Mina, but you can implement a function and Netty the same encoder and decoder.

Encoder:

public class Minaprotobufencoder extends Protocolencoderadapter {    @Override public    void Encode (iosession Session, Object message,            protocolencoderoutput out) throws Exception {    studentmsg.student Student = ( studentmsg.student) message;        byte[] bytes = Student.tobytearray (); Student object to protobuf byte code        int length = bytes.length;                Iobuffer buffer = iobuffer.allocate (length + 4);        Buffer.putint (length); Write header        buffer.put (bytes);//write Body        buffer.flip ();        Out.write (buffer);}    }

Decoder:

public class Minaprotobufdecoder extends Cumulativeprotocoldecoder {@Overrideprotected Boolean dodecode (iosession Session, Iobuffer In,protocoldecoderoutput out) throws Exception {//If the header portion (4 bytes) is not received, return directly to Falseif (In.remaining () & Lt 4) {return false;} else {//Mark start position, if a message does not transmit complete then return to this position in.mark ();//Read header section, get body length int bodylength = In.getint ();// If body is not received intact, return directly to Falseif (In.remaining () < Bodylength) {in.reset ();//Iobuffer position return to original marked place return false;} else {byte[] bodybytes = new Byte[bodylength];in.get (bodybytes);//Read body part studentmsg.student Student = StudentMsg.Student.parseFrom (bodybytes); Convert the Protobuf byte code in body into student object Out.write (student); Parse out a message return true;}}}

The Mina server joins the PROTOBUF encoder and decoder:

public class TCPServer {public    static void Main (string[] args) throws IOException {        Ioacceptor acceptor = new Nio Socketacceptor ();        Specifies the PROTOBUF encoder and Decoder        acceptor.getfilterchain (). AddLast ("Codec",                new Protocolcodecfilter (new Minaprotobufencoder (), New Minaprotobufdecoder ());        Acceptor.sethandler (New Tcpserverhandle ());        Acceptor.bind (new inetsocketaddress (8080));}    }

Thus, when dealing with business logic, it is the same as Netty:

public class Tcpserverhandle extends Iohandleradapter {@Override public void Exceptioncaught (iosession session, THR    Owable cause) throws Exception {cause.printstacktrace (); } @Override public void messagereceived (iosession session, Object message) throws Exception {//Read        The Student object passed by the client studentmsg.student Student = (studentmsg.student) message;        System.out.println ("ID:" + Student.getid ());        System.out.println ("Name:" + student.getname ());        System.out.println ("Email:" + student.getemail ());        System.out.println ("Friends:");        List<string> friends = Student.getfriendslist ();        for (String friend:friends) {System.out.println (friend);        }//Create a new Student object to the client StudentMsg.Student.Builder Builder = StudentMsg.Student.newBuilder ();        Builder.setid (9);        Builder.setname ("server");        Builder.setemail ("[email protected]");    Builder.addfriends ("X");    Builder.addfriends ("Y");        Studentmsg.student Student2 = Builder.build ();    Session.write (Student2); }}

Twisted:

In twisted, you first define a Protobufprotocol class that inherits the Protocol class and acts as an encoder and decoder. The Tcpserverhandle class that processes the business logic inherits the Protobufprotocol class, invoking or overriding the methods provided by the Protobufprotocol.

#-*-coding:utf-8–*-from struct import pack, Unpackfrom twisted.internet.protocol import Factoryfrom TWISTED.INTERNET.P Rotocol Import protocolfrom twisted.internet import reactorimport studentmsg_pb2# protobuf encoding, decoder class Protobufprotocol (Protocol): # for temporary storage of received data _buffer = B "" Def datareceived (Self, data): # The last unhandled data plus the data received this time Self._b Uffer = self._buffer + data # keeps looping until the new message does not receive full while True: # If the header receives the full if Len (self.                 _buffer) >= 4: # header section, press the large byte order to int, get body length lengths, = Unpack (">i", Self._buffer[0:4])                     # If body receives full if Len (Self._buffer) >= 4 + Length: # body part, Protobuf bytecode                    Packet = self._buffer[4:4 + length] # protobuf byte code to student object Student = Studentmsg_pb2. Student () Student. Parsefromstring (packet) # call ProtoBufreceived incoming Student Object self.protobufreceived (student) # Remove _buffe            The message part that has been processed in R self._buffer = self._buffer[4 + length:] else:break;    Else:break; def protobufreceived (self, Student): Raise Notimplementederror def sendprotobuf (self, Student): # Student The object is converted to PROTOBUF byte code data = student. Serializetostring () # Add header prefix specify Protobuf bytecode length Self.transport.write (Pack (">i", Len (data) + data) # logic code C         Lass Tcpserverhandle (Protobufprotocol): # implements the Protobufreceived def protobufreceived (self, student) provided by Protobufprotocol: # will receive student output print ' ID: ' + str (student.id) print ' Name: ' + student.name print ' Email: ' + Student.email print ' Friends: ' for friend in Student.friends:print friend # creates a student concurrency Send to client Student2 = Studentmsg_pb2. Student () Student2.id = 9 student2.name = ' server '. Decode (' UTF-8 ') # Chinese needs to be converted to UTF-8 string student2.email = ' [email protected] ' s Tudent2.friends.append (' X ') student2.friends.append (' Y ') self.sendprotobuf (student2) factory = Factory () facto Ry.protocol = tcpserverhandlereactor.listentcp (8080, Factory) Reactor.run ()

Here is a client-side test program written in Java:

public class TcpClient {public static void main (string[] args) throws IOException {socket socket = NULL;        DataOutputStream out = null;                DataInputStream in = null;            try {socket = new socket ("localhost", 8080);            out = new DataOutputStream (Socket.getoutputstream ());                        in = new DataInputStream (Socket.getinputstream ());            Create a Student pass to the server StudentMsg.Student.Builder Builder = StudentMsg.Student.newBuilder ();            Builder.setid (1);            Builder.setname ("Client");            Builder.setemail ("[email protected]");            Builder.addfriends ("A");            Builder.addfriends ("B");            Studentmsg.student Student = Builder.build (); byte[] outputbytes = Student.tobytearray (); Student to byte code out.writeint (Outputbytes.length); Write Header Out.write (outputbytes);                        Write body Out.flush (); Get the server sent over the Student int bodylength = In.readint ();            Read header byte[] bodybytes = new Byte[bodylength];  In.readfully (bodybytes); Read body studentmsg.student Student2 = StudentMsg.Student.parseFrom (bodybytes);            The body byte code is parsed into student System.out.println ("Header:" + bodylength);            System.out.println ("Body:");            System.out.println ("ID:" + Student2.getid ());            System.out.println ("Name:" + student2.getname ());            System.out.println ("Email:" + student2.getemail ());            System.out.println ("Friends:");            List<string> friends = Student2.getfriendslist ();            for (String friend:friends) {System.out.println (friend);            }} finally {//close connection in.close ();            Out.close ();        Socket.close (); }    }}

Use the client to test the above three TCP servers separately:

Server output:

Id:1
Name: Client
Email:[email protected]
Friends:
A
B

Client output:

Header:32
Body:
Id:9
Name: Server
Email:[email protected]
Friends:
X
Y

Fork Brother reproduced please indicate the source: http://blog.csdn.net/xiao__gui/article/details/38864961

Mina, Netty, twisted Study Together (v): Integration Protobuf