Mina, Netty, and Twisted (4): Customize your own protocol, nettytwisted

Mina, Netty, and Twisted (4): Customize your own protocol, nettytwisted

In the previous blog, I introduced some message splitting schemes and APIs provided by MINA, Netty, and Twisted for these schemes. For example, MINA's TextLineCodecFactory, PrefixedStringCodecFactory, Netty's LineBasedFrameDecoder, LengthFieldBasedFrameDecoder, Twisted's LineOnlyReceiver, Int32StringReceiver.

In addition to these solutions, there are many other solutions that can be defined by yourself. Here, we customize our own solution and use MINA, Netty, and Twisted to parse and assemble such messages, that is, encoding and decoding.

The previous blog article introduced a usageSpecifies the number of bytes of the Body based on the Header of the number of characters.Message splitting scheme, where the Header part is commonBig-Endian)Is a 4-byte integer. In this article, we will slightly modify this scheme and change the Header of the fixed number of cellsLittle-Endian)Is a 4-byte integer.

If the general large byte order represents a number, it is quite human-like to store the Low Bit Of the number with a high byte. In contrast, the small byte order and the large byte order use the high byte to store the high number.

In PythonStruct moduleSupports pack and unpack in byte order. in Java, you can use the following two methods to convert a small byte array to an int or an int to a small byte array, these two methods will be used in the following Java program:

Public class LittleEndian {/*** converts int to a 4-byte ordered byte array */public static byte [] toLittleEndian (int I) {byte [] bytes = new byte [4]; bytes [0] = (byte) I; bytes [1] = (byte) (I >>> 8 ); bytes [2] = (byte) (I >>> 16); bytes [3] = (byte) (I >>> 24); return bytes ;} /*** convert the 4-byte array in the small byte order to int */public static int getLittleEndianInt (byte [] bytes) {int b0 = bytes [0] & 0xFF; int b1 = bytes [1] & 0xFF; int b2 = bytes [2] & 0xFF; int b3 = bytes [3] & 0xFF; return b0 + (b1 <8) + (b2 <16) + (b3 <24 );}}

Whether MINA, Netty, or Twisted, the message encoding, decoding, and applicable code should be separated from the business logic code, which is conducive to code development, reuse, and maintenance. Similarly, in MINA and Netty, encoding and decoding must be inherited to implement the corresponding Encoder and Decoder, while in Twisted, Protocol is inherited to implement encoding and decoding. Although the implementation methods are different, their functions are the same:

1. Fit the message according to certain rules, such as fixed-length messages, specifying the Body length by line, separator, fixed-length headers, etc;

2. convert a suitable message from bytecode to the desired type. For example, convert IoBuffer into a String in MINA, so that the message Parameter received by messageReceived is of the String type;

3. You can input custom parameters during write, which are encoded by the encoder.

The following uses MINA, Netty, and Twisted to encode and decode messages with the specified body length in a 4-byte int.

MINA:

Connect to the received message in MINA for splitting and decoding. Generally, a decoder class is defined and inherited from the abstract class CumulativeProtocolDecoder to implement the doDecode method:

Public class MyMinaDecoder extends CumulativeProtocolDecoder {@ Override protected boolean doDecode (IoSession session, IoBuffer in, ProtocolDecoderOutput out) throws Exception {// If the Header part is not received (4 bytes ), returns false if (in. remaining () <4) {return false;} else {// mark the start position. If a message is not transmitted completely, it is returned to this position in. mark (); byte [] bytes = new byte [4]; in. get (bytes); // read the 4-byte Header int bodyLength = LittleEndian. getLittleEndianInt (bytes); // convert the value to int in the byte order. // if the body does not receive the complete value, false if (in. remaining () <bodyLength) {in. reset (); // return the position of IoBuffer to the place where it was originally marked; return false;} else {byte [] bodyBytes = new byte [bodyLength]; in. get (bodyBytes); String body = new String (bodyBytes, "UTF-8"); out. write (body); // parse a message return true ;}}}}

In addition, data encoding is required during session. write. An Encoder must be defined to inherit from the abstract class ProtocolEncoderAdapter to implement the encode method:

Public class MyMinaEncoder extends ProtocolEncoderAdapter {@ Override public void encode (IoSession session, Object message, ProtocolEncoderOutput out) throws Exception {String msg = (String) message; byte [] bytes = msg. getBytes ("UTF-8"); int length = bytes. length; byte [] header = LittleEndian. toLittleEndian (length); // converts it into a byte array in the small byte order, IoBuffer buffer = IoBuffer. allocate (length + 4); buffer. put (header); // header buffer. put (bytes); // body buffer. flip (); out. write (buffer );}}

Add the corresponding encoder and decoder when the server is started:

Public class TcpServer {public static void main (String [] args) throws IOException {IoAcceptor acceptor = new NioSocketAcceptor (); // specify the acceptor of the decoder. getFilterChain (). addLast ("codec", new ProtocolCodecFilter (new MyMinaEncoder (), new MyMinaDecoder (); acceptor. setHandler (new TcpServerHandle (); acceptor. bind (new InetSocketAddress (8080 ));}}

The following is the business logic code:

Public class TcpServerHandle extends IoHandlerAdapter {@ Override public void exceptionCaught (IoSession session, Throwable cause) throws Exception {cause. printStackTrace ();} // receives new data @ Override public void messageReceived (IoSession session, Object message) throws Exception {// MyMinaDecoder converts the received data from IoBuffer to String msg = (String) message; System. out. println ("messageReceived:" + msg); // MyMinaEncoder adds a small byte Header to the write string and converts it into a bytecode session. write ("received ");}}

Netty:

The decoder in Netty is similar to MINA. the decoder inherits the abstract class ByteToMessageDecoder and implements the decode method:

Public class MyNettyDecoder extends ByteToMessageDecoder {@ Override protected void decode (ChannelHandlerContext ctx, ByteBuf in, List <Object> out) throws Exception {// If the Header part is not received (4 bytes ), directly exit this method if (in. readableBytes ()> = 4) {// mark the start position. If a message is not transmitted, it is returned to this position in. markReaderIndex (); byte [] bytes = new byte [4]; in. readBytes (bytes); // read the 4-byte Header int bodyLength = LittleEndian. getLittleEndianInt (bytes); // the header is converted to int in the byte order. // if the body does not receive the complete if (in. readableBytes () <bodyLength) {in. resetReaderIndex (); // ByteBuf returns to the tag position} else {byte [] bodyBytes = new byte [bodyLength]; in. readBytes (bodyBytes); String body = new String (bodyBytes, "UTF-8"); out. add (body); // parse a message }}}}

The following is the encode method inherited from the abstract class MessageToByteEncoder:

Public class MyNettyEncoder extends MessageToByteEncoder <String >{@ Override protected void encode (ChannelHandlerContext ctx, String msg, ByteBuf out) throws Exception {byte [] bytes = msg. getBytes ("UTF-8"); int length = bytes. length; byte [] header = LittleEndian. toLittleEndian (length); // int is converted to the byte array out in the byte order. writeBytes (header); // write header out. writeBytes (bytes); // write body }}

Add the corresponding encoder and decoder:

Public class TcpServer {public static void main (String [] args) throws InterruptedException {EventLoopGroup bossGroup = new NioEventLoopGroup (); EventLoopGroup workerGroup = new NioEventLoopGroup (); try {ServerBootstrap B = new ServerBootstrap (); B. group (bossGroup, workerGroup ). channel (NioServerSocketChannel. class ). childHandler (new ChannelInitializer <SocketChannel> () {@ Override public void initChannel (SocketChannel ch) throws Exception {ChannelPipeline pipeline = ch. pipeline (); // Add your own Encoder and Decoder pipeline. addLast (new MyNettyDecoder (); pipeline. addLast (new MyNettyEncoder (); pipeline. addLast (new TcpServerHandler () ;}}); ChannelFuture f = B. bind (8080 ). sync (); f. channel (). closeFuture (). sync ();} finally {workerGroup. shutdownGracefully (); bossGroup. shutdownGracefully ();}}}

Business logic processing class:

Public class TcpServerHandler extends ChannelInboundHandlerAdapter {// receives new data @ Override public void channelRead (ChannelHandlerContext ctx, Object msg) {// MyNettyDecoder converts the received data from ByteBuf to String message = (String) msg; System. out. println ("channelRead:" + message); // MyNettyEncoder adds a small byte Header to the write string and converts it into a bytecode ctx. writeAndFlush ("received") ;}@ Override public void exceptionCaught (ChannelHandlerContext ctx, Throwable cause) {cause. printStackTrace (); ctx. close ();}}

Twisted:

The implementation of Twisted is not the same as that of MINA and Netty. The implementation method is more primitive, but the more primitive it is, the closer it is to the underlying principle.

First, define a MyProtocol class inherited from Protocol to act as an encoding and decoder similar to MINA and Netty. The class TcpServerHandle that processes the business logic inherits MyProtocol and overwrites or calls some methods provided by MyProtocol.

#-*-Coding: UTF-8-*-from struct import pack, unpack from twisted. internet. protocol import Factory from twisted. internet. protocol import Protocol from twisted. internet import reactor # encoding and decoder class MyProtocol (Protocol): # used to temporarily store received data _ buffer = B "" def dataReceived (self, data ): # The Last unprocessed data plus the received data self. _ buffer = self. _ buffer + data # keep repeating until the new message does not receive the complete while True: # if the header receives the complete if len (self. _ buffer)> = 4: # convert int length, = unpack ("<I", self. _ buffer [0: 4]) # if the body receives the complete if len (self. _ buffer)> = 4 + length: # body packet = self. _ buffer [4: 4 + length] # After receiving and decoding a new message, call stringReceived self. stringReceived (packet) # Remove the processed message part self in _ buffer. _ buffer = self. _ buffer [4 + length:] else: break; def stringinclued (self, data): raise NotImplementedError def sendString (self, string): self. transport. write (pack ("<I", len (string) + string) # logic code class TcpServerHandle (MyProtocol): # implement the stringReceived provided by MyProtocol instead of dataReceived, otherwise, def stringinclued (self, data): # print 'stringinclued: '+ data # Call sendString instead of self. transport. write. Otherwise, you cannot encode self. sendString ("received") factory = Factory () factory. protocol = TcpServerHandle reactor. listenTCP (8080, factory) reactor. run ()

The following is a client test program written in Java:

Public class TcpClient {public static void main (String [] args) throws IOException {Socket socket = null; OutputStream out = null; InputStream in = null; try {socket = new Socket ("localhost", 8080); out = socket. getOutputStream (); in = socket. getInputStream (); // request Server String data = "I Am a client"; byte [] outputBytes = data. getBytes ("UTF-8"); out. write (LittleEndian. toLittleEndian (outputBytes. length); // write header out. write (outputBytes); // write body out. flush (); // get the response byte [] inputBytes = new byte [1024]; int length = in. read (inputBytes); if (length> = 4) {int bodyLength = LittleEndian. getLittleEndianInt (inputBytes); if (length> = 4 + bodyLength) {byte [] bodyBytes = Arrays. copyOfRange (inputBytes, 4, 4 + bodyLength); System. out. println ("Header:" + bodyLength); System. out. println ("Body:" + new String (bodyBytes, "UTf-8") ;}} finally {// close the connection in. close (); out. close (); socket. close ();}}}

Use the client to test the preceding three TCP servers:

MINA server output result:

MessageReceived: I am a client.

Netty server output result:

ChannelRead: I am a client.

Twisted server output result:

ChannelRead: I am a client.

The output results of the client test on the three servers are:

Header: 6
Body: Received

Because a Chinese character generally occupies three bytes, the Header corresponding to the two Chinese characters is 6.

Who has spring integrated hibernate project, cannot use struts framework, if there is a servlet to provide data through mina or netty Communication

Sent to your mailbox