Packet and unpacking of network communication

for the TCP-based development of the communication program, there is a very important problem to solve, that is, packet and unpacking.

I. Why TCP-based communication programs require packets and unpacking.

TCP is a "flow" protocol, the so-called flow, is a string of data without bounds. You can think of the river water, is connected into a piece, in the meantime there is no dividing line. However, the general communication program development needs to define a separate packet, such as a data packet for logging in, for logging off the packet. Because the TCP "stream "and the network condition, there are several situations in which data transfer occurs.
Suppose we call two times in a row send two pieces of data data1 and data2, at the receiving end there are the following kinds of reception (of course, not only these cases, here are only a representative case).
A. Receive the DATA1 first, then receive the DATA2.
B. Receive some of the DATA1 data first, then receive the remainder of data1 and all of the data2.
C. Received all the data of data1 and some data of data2, then received the remaining data of data2.
D. All data of data1 and Data2 are received at once.

For a This situation is exactly what we need, no more discussion. For b,c,d the situation is that we often say "sticky bag", we need to take the received data to be dismantled, split into a separate packet. In order to split the package, the packet must be marshaled on the sending side.

Another: For UDP, there is no problem of unpacking, because UDP is a "packet" protocol, that is, two of the data is bounded, at the receiving end of either receive data or receive a complete piece of data, no less receive or receive more.

Two. Why there is a b.c.d situation.
"Sticky packs" can occur on the sending side and can occur at the receiving end.
1. Sticky packets from the sending end caused by the Nagle algorithm: The Nagle algorithm is a kind of algorithm to improve the network transmission efficiency. Simply put, when we submit a piece of data to TCP for sending, TCP does not send this data immediately, but instead waits for a short period of time to see if there is any data to be sent during the wait. If any, these two pieces of data will be sent out at once. This is a simple explanation of the Nagle algorithm, please read the relevant books. The case of C and D is probably caused by the Nagle algorithm.
2. Receive end-of-time receive packets: TCP will present the received data in its own buffer, and then notify the application layer to fetch the data. When the application layer is unable to get the TCP data out in time for some reason, it will cause the TCP buffer to hold several pieces of data.

three. How to package and unpacking.
Package:
Packet is to a piece of data with Baotou, so that the packet is divided into Baotou and the package of two parts of the content (later, the packet will be added to filter illegal packets "packet tail" content). Baotou is actually a fixed-size structure, which has a struct member variable represents the length of the package body, this is a very important variable, Other struct members can be defined on their own terms. Depending on the length of the Baotou and the variable containing the length of the packet in the Baotou, a complete packet can be split correctly.

Equivalent to: two times the logical business packet received, first receive its own definition of a logical header, like the need to deal with the logical business package how long, and then receive the package body. For non-blocking receive, if one receive is not completed, the received data can be saved in a buffer, waiting for the next packet to be processed when the entire logical business package is received. Note: TCP is used for streaming data sending, so a business package and another business package are arrived in sequence (in the order of your contract), not the first business package nested inside the other business package content (unless you intentionally send the sender), so please be assured to pick up the package.

Dynamic buffer Staging and unpacking method:

The buffer is dynamic because the buffer length is increased when the length of the data that needs buffering exceeds the length of the buffer.
The approximate process is described as follows:
A, a buffer is dynamically allocated for each connection, and the buffer is associated with the socket, which is commonly associated with a struct.
B, when the data is received, the data is first stored in the buffer.
C, determine whether the data length in the buffer is sufficient for the length of a baotou, if not enough, do not do the unpacking operation.
D, according to the Baotou data to parse out the inside represents the length of the package body variables.
E, determine whether the buffer in addition to the length of the packet outside the Baotou is enough to the length of a package, if not enough, do not do the unpacking operation.
F, remove the entire packet. Here the "take" means not only to copy the packet from the buffer, but also to remove the packet from the cache. The way to delete this packet is to move the data behind the package to the start address of the buffer.
There are two drawbacks to this approach. 1. Dynamically allocating a buffer for each connection increases memory usage. 2. Three places need to copy data, one place is to store the data in the buffer, one place is to take the complete packet out of the buffer, One place is to remove the packet from the buffer. This improved method of unpacking will solve and perfect some shortcomings.
The relevant code is given below.

First look at Baotou structure definition

#pragma pack (push,1)//start defining the packet in byte alignment/*----------------------Baotou---------------------*/typedef struct TAGPACKAG       ehead {BYTE Version;       WORD Command;  WORD Ndatalen;   The length of the package body is}package_head; #pragma pack (POP)//end definition packet to restore the original alignment

Then look at the storage data and the "fetch" data functions.

[C-sharp]  View Plain Copy/*****************************************************************************  Description: Add data to cache   input:pbuff[in]-data to be added; nlen[in]-to add data length   return:  Returns False if the current buffer does not have enough space to hold pbuff, otherwise returns   TRUE.   ******************************************************************************/   BOOL  Cdatabufferpool::addbuff ( char *pBuff, int nLen )    {        m_cs. Lock ();///Critical area lock        if  ( nLen < 0 )        {           m_cs. Unlock ();           return FALSE;        }       if  ( nlen <= getfreesize ()   )///Determine if the remaining space is sufficient to store nlen Long data        {           memcpy (M_pbuff + m_noffset, pbuff, nlen);            m_nOffset += nLen;       }       else///if not enough, expand the original space        {            char *p = m_pBuff;            m_nsize += nlen*2;//per growth 2*nlen            m_pBuff = new char[m_nSize];            memcpy (m_pbuff,p,m_noffset);           delete []p;            memcpy (M_pbuff + m_noffset, pbuff,  nlen);           m_nOffset += nLen;      &Nbsp;     m_cs. Unlock ();           return FALSE;