"Troublesome" Data Buffer

. Net 4.0 getting started with network development --

"Troublesome" Data Buffer

Note:

This is
Beginners in the Network Development Field
A series of articles, which can be used as an introduction to. Net 4.0 Object-Oriented Programming



The expanded reading of this book. During the writing process, I assume that the reader can read the relevant chapters of this book without wasting any effort to introduce the relevant content.

For other types of readers, unless you have the corresponding. Net technical background and certain development experience, you may encounter difficulties in reading.

I hope this series of articles will give readers a taste of the charm of network development!

In addition, these articles are original articles. Please respect the work of the author. I allow everyone to freely repost these articles and related examples for the purpose of knowledge sharing, but without my permission, please do not use it for commercial profit purposes.

If any error occurs in this article, please reply and correct it.

Thank you!

 

Jin xuliang

========================================================== ==========

Click the following link to read the previous articles in this series:

 

1 《

Opening Speech --
Invincible

2 《
IP address knowledge

3. I am in the center of "network"
"

4 "first appointment with socket"
"

5 "meet again" with socket"
"

========================================================== ==========

 

1 Guide


In the previous two lectures, you have learned a lot about how to use socket to transmit string information between two computers, and developed a "one-question-one-answer" network application.
I don't know if you have noticed that the program we have developed so far must work correctly. There are two assumptions:
(1) The data to be sent must not exceed the data buffer size given by the application (in the preceding example, the data buffer is mostly composed of 1024 bytes of byte [] arrays)
(2) The socket. Send () method called by the client must be strictly matched with the socket. Receive () method called by the server.
If the above two requirements are not met, we are unfortunately stuck in the swamp. Both issues are directly or indirectly related to the data buffer zone.
In this article, let's talk about the "Swamp"-data buffer size.

2 Data Buffer Zone Test

 

Open the sample solution tcpbufferdemo.
The text file textfile1.txt contains an ancient poem. The client tcpclientapp needs to send it to the server.

Night Calls
There is an infinite period of heaven and earth, and there is a poor period of life. One day is less than one day if you go to the next day;
Wealth has a fixed number, while learning has no fixed number. If you want to get one point, you will get one point.
Exit

Note that the last one in the above data is the "exit" command, which notifies the server that "data transmission is complete" to disconnect. The following is the client code:

// Send the content in the text file to the server by line
Foreach (string userinput in file. readlines ("textfile1.txt "))
{
Byte [] sentbytes = encoding. utf8.getbytes (userinput );
Server. Send (sentbytes); // send to server
Thread. Sleep (1000 );


}

Note that the above "thread. Sleep" sentence will be interesting, and it is by no means an irrelevant "small role" in the entire experiment ".
The tcpserverapp on the server receives data from the client in an infinite loop. The following is a code snippet.

Byte [] DATA = new byte [Buffersize

]; // Buffersize = 1024
//......
While (true)
{
// Thread. Sleep (4000 );


Recv = client. Receive (data); // receives data
Stringsentbyclient = encoding. utf8.getstring (data, 0, Recv );
Console. writeline ("client sent: {0}", stringsentbyclient );
If (stringsentbyclient = "exit ")
Client. Close ();
Break;
}

Note that there is also a "thread. Sleep" in the above Code, which is commented out first. In addition, buffersize is a constant, which is currently defined as 1024 bytes.
The following is the screen for "Everything works:

 
Figure 1

Read the client and server code carefully to understand the meaning of each sentence.
Now, a fun experiment begins.

Experiment 1: Change the buffersize from 1024 to 10.

 

Figure 2

Yo, there's a bunch of "?", Chinese characters cannot be correctly decoded on the server!

Experiment 2: Change the buffer settings used by the server to 1024 bytes for receiving data, and comment out the "thread. Sleep (1000)" Statement on the client.

 
Figure 3

The server is "dead!

Experiment 4: cancel the "thread. comment the sleep (1000) statement to cancel the "thread. comment on the sleep (4000) statement to delay receiving data from the client:

Figure 4

Through these four experiments, we can see how fragile such network applications are, just like flowers in the greenhouse, which cannot withstand any storms. The key points can be summarized as follows:

• When using TCP to transmit data between two computers, ensure that the two sides "send" and "receive" pairing, and the data buffer size set by both parties should be able to put a complete message. Otherwise, it will cause a lot of trouble to send and receive data.
• In addition, the data processing speed of the Data sender and receiver should be "matched". Otherwise, it will also cause troubles.

We can use the terms "Sticky package" and "packet loss" to express the mismatch between the size of the data sending and receiving buffer caused by various reasons.

3 intuitive display of "package" and "packet loss"

 

Let's take a deeper look at the "Sticky package" and "packet loss" issues.
Both the send and receive methods of the socket have a special form of overload. Data in multiple buffers can be sent at a time:

Public int send (Ilist <arraysegment <byte>

Buffers );
Public int receive (Ilist <arraysegment <byte>

Buffers );

In the above Code, arraysegment <byte> represents the "segment" of a one-dimensional byte array. Generally speaking, it is a "part" of a byte array ".
The above overload form of the send method will send the data contained by all arraysegment objects in a set (which must implement the ilist Interface) to "distant" in turn ". Correspondingly, the receive method is responsible for receiving the data and "Filling" the received data to various arraysegment objects whose original values are "null.

See the example solution testarraysegment.
The client clientapp has three arraysegment objects. Each arraysegment object contains arraysegmentsize bytes, which are "1", "2", and "3" respectively, and then sends the object by calling the send method.
Similarly, the server serverapp has prepared three arraysegment objects in advance, which are filled by the receive method.
Experiment 1: If the size of the byte array list on the client and server is the same, arraysegmentsize = 10

 
Figure 5

Experiment 2: the size of the byte array list between the client and the server is different. The server arraysegmentsize = 10, and the client arraysegmentsize = 5

 
Figure 6

Experiment 3: the size of the byte array list on the client and server is different. The size of arraysegmentsize on the server is 5, and the size of the client arraysegmentsize is 10.

Figure 7

4. Summary of buffer issues:


 

After two rounds of experiments, although these examples are highly simplified, readers must have an intuitive experience with the data buffer in TCP network applications, in addition, network application development is far more complex and difficult than desktop applications.
In fact, TCP network applications developed based on socket have two buffer zones:

• One is the data buffer set by the application, which can be called"Application TCP/IP data buffer
  
  This type of buffer is involved in this article.
• The other is the dedicated data buffer provided by the operating system for TCP/IP.TCP/IP System Buffer
  
  ".

The send method of socket is called in blocking mode. It only ensures that the data has been successfully sent to the buffer set for TCP/IP in the operating system, it is still unknown whether the data is immediately sent to the network by the operating system and received by the other party.
By default, the size of the data buffer provided by the operating system for TCP is 8 K. We can adjust the following attributes of the socket object.

Public int receivebuffersize {Get; set ;}
Public int sendbuffersize {Get; set ;}

The size of the System Buffer will affect the performance of network applications. For example, if a large file needs to be transferred between two computers, a large buffer (such as 8 K) is a relatively small buffer (such as 1 K) high data transmission efficiency.

All in all, TCP only guarantees that "all data to be sent is submitted to the receiver in order", but does not maintain the boundary of the message to be sent, it is entirely possible that the sender connects to call the send method multiple times, and the receiver only calls the receive once to receive all the results. In addition, the network may be congested, and the sending end and the receiving end process data at different speeds ,......, The situation is even more complicated.

Therefore, we must carefully consider various situations and adopt appropriate strategies to deal with them. The following three methods are usually used:
(1) only messages of a fixed size are transmitted.
(2) enclose the size information of a message at the beginning of the message.
(3) one question and one answer

We will introduce the specific examples of these three policies in the next section.

If you are interested, try these three strategies first. Can you implement them by yourself?

========================================================== =

Click to download the sample source code.

(Blog garden download link: http://files.cnblogs.com/bitfan/TCPBufferSourceCode.rar)