NIO framework-mina source code parsing (4): processing, encoding, and decoding of stick packets and broken packets

NIO framework-mina source code analysis (1): Background

NIO framework-mina source code analysis (2): Mina Core Engine
NIO framework-mina source code analysis (III): underlying communication and responsibility chain model application

1. Stick package and segment package

Stick package: In the TCP protocol, several packets sent by the sender are pasted into a packet upon receiving by the receiver. From the receiving buffer, the header of the last packet is followed by the end of the data of the previous packet.
Possible causes:

The sender needs to wait until the buffer is full to send out, resulting in sticking packets.

The receiver does not receive packets in the buffer zone in time, resulting in receiving multiple packets.

Packet disconnection: The data is incomplete. For example, if the package is too large, the package is divided into multiple packets and sent multiple times, resulting in incomplete data received each time.

2. Message Transmission Format

The message length, header, and body are the first n Bytes used to store the message length and determine when the current message ends.

The message header + message body is a fixed-length message. The first few bytes are the message header followed by the message header.

What is used in Mina is

The message length + the first four bytes are used to store the message length and determine when the current message ends.

3. encoding and decoding

In the network, information is transmitted in bytes. When we write our own code, it is a specific object, if we want our objects to be transmitted over the network, we need to encode and decode them.

Encoding: encode our messages in binary format and transmit them in bytes over the network.

Decoding: decodes the received bytes into the objects in our code.

In Mina, object encoding and decoding are implemented using the objectoutputstream provided by JDK.

4. Message Processing implementation in Mina

For receiving and processing messages, we usually use the TCP protocol, which is sharded. In the following code, the specific function is to read data cyclically from the channel until no data is readable, if the buffer is full, the received data is sent to the decoding factory for processing.

4.1 receive messages

// Class abstractpollingioprocessorprivate void read (s session) {iosessionconfig = session. getconfig (); int buffersize = config. getreadbuffersize (); iobuffer Buf = iobuffer. allocate (buffersize); Final Boolean hasfragmentation = session. gettransportmetadata (). hasfragmentation (); try {int readbytes = 0; int ret; try {// check whether there are fragments in TCP transmission, that is, the large message is divided into multiple small messages and then transmitted if (hasfragmentation) {// The read method is not blocked and is not read 0 while (ret = read (Session, Buf)> 0) {readbytes + = ret; // If (! Buf. hasremaining () {break ;}} else {ret = read (Session, Buf); If (Ret> 0) {readbytes = RET ;}} finally {Buf. flip ();} If (readbytes> 0) {iofilterchain filterchain = session. getfilterchain (); // process the message filterchain. firemessagereceived (BUF); Buf = NULL; If (hasfragmentation) {If (readbytes <1 <config. getreadbuffersize () {session. decreasereadbuffersize ();} else if (readbytes = config. Getreadbuffersize () {session. increasereadbuffersize () ;}} if (Ret <0) {scheduleremove (Session) ;}} catch (throwable e) {If (E instanceof ioexception) {If (! (E instanceof portunreachableexception) |! Abstractpolicramsessionconfig. class. isassignablefrom (config. getclass () | (abstract‑ramsessionconfig) config ). iscloseonportunreachable () {scheduleremove (Session) ;}} iofilterchain filterchain = session. getfilterchain (); filterchain. fireexceptioncaught (e );}}

4.2 decoding and encoding

// Class implements actiobuffer public object GetObject (final classloader) throws classnotfoundexception {// first checks whether the message length in the current buffer is complete. If the message length is incomplete, if (! Prefixeddataavailable (4) {Throw new bufferunderflowexception ();} // message length int length = getint (); If (length <= 4) {Throw new bufferdataexception ("Object length shocould be greater than 4:" + length);} int oldlimit = limit (); // limit to the end of the message limit (position () + length); try {objectinputstream in = new objectinputstream (asinputstream () {@ override protected objectstreamclass readclassdescriptor () thro WS ioexception, classnotfoundexception {int type = read (); If (type <0) {Throw new eofexception ();} switch (type) {Case 0: // non-serializable class or primitive types return Super. readclassdescriptor (); Case 1: // serializable class string classname = readutf (); Class <?> Clazz = Class. forname (classname, true, classloader); Return objectstreamclass. lookup (clazz); default: Throw new streamcorruptedexception ("unexpected class descriptor type:" + type) ;}}@ override protected class <?> Resolveclass (objectstreamclass DESC) throws ioexception, classnotfoundexception {string name = DESC. getname (); try {return class. forname (name, false, classloader);} catch (classnotfoundexception ex) {return Super. resolveclass (DESC) ;}}; return in. readobject ();} catch (ioexception e) {Throw new bufferdataexception (E);} finally {limit (oldlimit) ;}// judge whether the current message is complete public Boolean prefixeddata Available (INT prefixlength, int maxdatalength) {If (remaining () <prefixlength) {return false;} int datalength; Switch (prefixlength) {Case 1: datalength = getunsigned (position (); break; Case 2: datalength = getunsignedshort (position (); break; Case 4: datalength = getint (position (); break; default: Throw new illegalargumentexception ("prefixlength:" + prefixlength);} If (datalength <0 | Datalength> maxdatalength) {Throw new bufferdataexception ("datalength:" + datalength);} // determines whether the current message is complete. Return remaining ()-prefixlength >= datalength ;} // code public iobuffer putobject (Object O) {int oldpos = position (); Skip (4); // make a room for the length field. four bytes are reserved for storing the message length. Try {objectoutputstream out = new objectoutputstream (asoutputstream () {@ override protected void writeclassdescripto R (objectstreamclass DESC) throws ioexception {try {class <?> Clz = Class. forname (DESC. getname (); If (! Serializable. class. isassignablefrom (clz) {// non-serializable class write (0); super. writeclassdescriptor (DESC);} else {// serializable class write (1); writeutf (DESC. getname () ;}} catch (classnotfoundexception ex) {// primitive types write (0); super. writeclassdescriptor (DESC) ;}}; out. writeobject (o); out. flush ();} catch (ioexception e) {Throw new bufferdataexception (E);} // fill the length field int newpos = position (); position (oldpos ); // store the message length putint (newpos-oldpos-4); position (newpos); return this ;}

4.3 handling of broken packets and stick packets

// Class cumulativeprotocoldecoder public void decode (iosession session, iobuffer in, protocoldecoderoutput out) throws exception {// whether there is a shard, TCP has a shard if (! Session. gettransportmetadata (). hasfragmentation () {While (in. hasremaining () {If (! Dodecode (Session, in, out) {break ;}return ;}// 1. process the broken packet // 2. process the sticking packet Boolean usingsessionbuffer = true; // whether there is a packet disconnection condition in the session (after the previous processing), the packet is saved to the iobuffer Buf = (iobuffer) session in the session. getattribute (buffer); // if we have a session buffer, append data to that; otherwise // use the buffer read from the network directly. if (BUF! = NULL) {// if there is a broken packet, the current package is spliced into the broken packet. boolean appended = false; // make sure that the buffer is auto-expanded. if (BUF. isautoexpand () {try {Buf. put (in); appended = true;} catch (illegalstateexception e) {// a user called derivation method (e.g. slice (), // which disables auto-expansion of the parent buffer .} catch (indexoutofboundsexception e) {// a user disabled auto-expansion .}} if (appended) {B Uf. flip ();} else {// reallocate the buffer if append operation failed due to // Derivation or disabled auto-expansion. buf. flip (); iobuffer newbuf = iobuffer. allocate (BUF. remaining () + in. remaining ()). setautoexpand (true); newbuf. order (BUF. order (); newbuf. put (BUF); newbuf. put (in); newbuf. flip (); Buf = newbuf; // update the session attribute. session. setattribute (buffer, Buf) ;}} else {Buf = in; Usingsessionbuffer = false;} // 2 sticky packet processing. There may be multiple messages in the buffer, and each message needs to be processed (decoded) multiple times until the message is processed completely, or the remaining message is not a complete message or the buffer has no data for (;) {int oldpos = Buf. position (); Boolean decoded = dodecode (Session, Buf, out); If (decoded) {// decoded if (BUF. position () = oldpos) {Throw new illegalstateexception ("dodecode () Can't return true when buffer is not consumed. ");} // If (! Buf. hasremaining () {// The buffer has no data break;} else {// the rest of the message is not a complete message, and the packet is broken with a break ;}} // if there is any data left that cannot be decoded, we store // it in a buffer in the session and next time this decoder is // invoked the session buffer gets appended to if (BUF. hasremaining () {// The remaining message is not a complete message. If the packet is disconnected, the buffer is updated. If no, save the remaining broken packets to the IF (usingsessionbuffer & Buf. isautoexpand () {Buf. compact () ;}else {storeremaininginsession (BUF, session) ;}} else {If (usingsessionbuffer) {removesessionbuffer (Session );}}}

// Class objectserializationdecoder protected Boolean dodecode (iosession session, iobuffer in, protocoldecoderoutput out) throws exception {// first checks whether the message length in the current buffer is complete. If it is incomplete, an IF (! In. prefixeddataavailable (4, maxobjectsize) {return false;} Out. Write (in. GetObject (classloader); Return true ;}

??

NIO framework-mina source code parsing (4): processing, encoding, and decoding of stick packets and broken packets