How should I avoid packet loss when a large amount of data is continuously transmitted over TCP?

How should I avoid packet loss when a large amount of data is continuously transmitted over TCP?

For example, sending a file. I remember someone mentioned the possible stack overflow. How can this problem be avoided? Can I send a confirmation packet after receiving the data? After receiving the confirmation packet, I will continue to send it. Or sleep after the sender sends some data.

Also, we all know that confirmation is required when UDP is used to send packets, but connection-oriented reliable transmission is required for TCP. Is it true that the sent packets can be received without confirmation?

Netsys2 (incoming call!) on the first floor !) The score is 0 at 13:31:44, for example, the file is sent. I remember someone mentioned the possible stack overflow. How can this problem be avoided?
-------> Sends messages in segments and receives messages within a fixed length. After receiving the messages correctly, the system returns to the foreground.
For example:
Front-end --> back-end: Start the file name, length, and other information,
Background --> Foreground: OK, 0 (this 0 indicates the file offset to be received)
..
Front-end --> back-end: Send a paragraph
Background --> Foreground: OK, the next offset.
..
.. To the end of sending

Confirmation is required when UDP is used to send packets, but connection-oriented reliable transmission is required for TCP. Is it true that the sent packets can be received without confirmation?
---> Not necessarily, although TCP ensures your data transmission, if the receiver suddenly disconnects or the intermediate switching device is offline, the data may not reach the background, so you have to send a response packet for confirmation.

On the 2nd floor, shilong (Yanyu www.ylog.net) replied to the 13:44:04 score 0 packet loss because when the sender sends the packet too fast, the receiver will fold the package behind it with the previous package. For example
Sending end: sending Package 1, sending package 2
The receiving end only receives one packet. The content is: Package 1 + package 2

I also encountered this problem before. Later, I used the following method to add a packet header for each packet, and the receiving buffer is large enough, then, package obtained from the receiving buffer is placed in a message queue.

PS: it is best to mark the length of the entire package in the packet header when sending the packet to facilitate unpacking.
The queue can be solved using the deque container in STL.

Back to shilong (Yanyu www.ylog.net) on the third floor, with a score of 0 at 13:47:50
If the TCP sending function returns a successful message, it must have been successfully sent. This is the feature of the TCP protocol. After the disconnection, the sending function will certainly return an error.

On the 4th floor, youngwhz (sunbird) replied with a score of 20 at 13:58:19
For example, sending a file. I remember someone mentioned what Stack Overflow might happen .???
TCP transmission can ensure the reliability of data exchange, but this means that a host on one side correctly transmits data to the target machine. The stack of the protocol stack on the target machine is limited, if the received data is not processed in time on the target machine, the stack may overflow! This overflow is not due to the TCP protocol itself, but to the buffer overflow of the system's IP protocol stack!

The fifth floor navy125 (Navy) Replies to 18:40:42 score 0 should be said to be avoided, in the sending end need to add the select () option, wait for select () the next package will be sent only after the result is returned.
Select () is used to determine whether the current network is readable or writable. functions that are not readable or writable will be blocked.
Although this may affect some speed, it generally does not cause stack overflow.

On the 6th floor, fly1980 (FEI) replied with a score of 0 at 22:38:29.
Youngwhz (sunbird) is very right.
Recommended solution:
Send the file in multiple parts. Each time the receiving end receives a piece of data, it sends a receipt confirmation message to the sender (including the length of the received data)
The sender sends the next piece of data only after receiving the "receive confirmation" sent by the receiver ....
Until the entire file is sent.
Each piece of data has a packet header, which can contain some flag information, such as whether all data has been sent (that is, the last package)
Top

Back to ypos (Ye Kai) on the 7 th floor, with a score of 0 from 15:13:16
When TCP connections are continuously enabled, no data is lost.
The buffer overflow of the system's IP protocol stack is not called overflow. It can only be said that the packet is discarded. It is a normal phenomenon, but it only affects the performance, and the TCP has its own traffic control. You do not need to consider it when doing TCP.
As for "receive confirmation", it is completely unnecessary. In order to deal with the disconnection, each package can carry a serial number. When the connection is disconnected, the sender resends the unsent packet.

The 8-floor wwwdfq1977 (qswl) was replied to the 23:58:18 score of 30.
I think many people do not understand the concept of TCP/IP hierarchy.
When the TCP protocol is used, the send function is used to send data successfully. This means that the application layer successfully transmits data to the TCP layer of the local machine, not that the data has been confirmed by the TCP layer of the other party.

If the connection is normal, the data you pass to the TCP layer of the local machine can be correctly received by the other TCP layer. This is guaranteed by the TCP protocol, otherwise the reliability of TCP is meaningless. however, if the network is suddenly interrupted, unsent data in the TCP protocol stack of the local machine may be discarded. Therefore, if you re-connect to send data after the connection is interrupted, the data you send cannot be determined by the amount of data you send to the TCP layer of the Local Machine. Instead, you should ask the other party how much data the application layer has received.

If yourProgramThe data to be sent must be received by the recipient without missing a word, so you must save the sent data as a file and then send it. The sending rules are described above. In addition, confirmation should also be made when the file is uploaded, but you do not need to confirm each package.

If this is not necessary, you do not need to confirm

Zihan (zi Han) on the 9 th Floor replied to the score of 0 at 22:55:06 and agreed to the upstairs. indeed, TCPIP will help you confirm your package information, but if it is caused by a network exception, TCPIP may not be able to help you solve it in some cases. Therefore, it is recommended to save things as a file, then send messages in multiple parts, and add the header and tail information by yourself,

However, this is not necessary in general.

Asklxf (Xuefeng) on the 10th floor, with a score of 30 at 23:32:48
TCP will not cause packet loss at any time, because:
In the TCP/IP model, the IP layer is responsible for sending packets but does not guarantee correct receipt, while the TCP layer is on the IP layer to ensure that each packet is correctly received.
In the application, if a piece of data is sent by sending a socket, as long as the function returns OK, the other party will certainly receive the data correctly.

When using TCP to send data, you do not need to worry about whether the data is correctly received (which is guaranteed by TCP), or you do not need to write the response mechanism yourself (TCP responds to each packet). Otherwise, it will become UDP.

If a network fault occurs, the socket will report an error and the TCP connection will be disconnected. However, the sent data must have been correctly sent. All you need is to try to reconnect, then, the unsent data is sent.

On the 11th floor, wwwdfq1977 (qswl) replied with a score of 0 at 01:48:11.
Upstairs: I do not agree with your statement about the send function.
First, you must specify what the send function has done. It is responsible for transmitting data to the local TCP layer and returning the data. It is also responsible for obtaining the data transmitted at the application layer to the receiver's TCP layer for confirmation.

If the latter is true, but in fact it is not, it is because if so, then NagleAlgorithmIt is useless, because this algorithm is to splice the small data passed into the send Function Multiple times into a large package and then send it again.

It can be seen that even if the send function is OK, the other party may not be able to accept it because the TCP protocol only fulfills its obligations on the TCP layer, the send function only plays the role of passing data to the TCP layer at the application layer. In addition, it has nothing to do with the TCP layer.

There is another special case, although rare, but not impossible: If the receiver's TCP layer does receive the data and sends the confirmation information to the sender, however, when he transmits data to the application layer, the application layer suddenly crashes, so the receiver's application cannot accept the data, the sender's TCP layer obtains confirmation information that the data has been successfully sent. This is an extreme example in the book. If you don't believe it, you can read this book.

In short, do not mix the send function with the TCP protocol. The send function is not something in the TCP protocol. The send function cannot guarantee anything, and these guarantees are completed by TCP.

On the 12th floor, wwwdfq1977 (qswl) replied with a score of 0 at 01:51:36. Again, TCP will not lose packets and the send Return OK receiver will certainly receive no relationship. The former does not mean that the latter

On the 13th floor, netsys2 (incoming call !) The score is 0 at 10:20:47.
Without any function, the protocol can ensure no packet loss at all,
TCP is much better, because it has a connection and a protocol for re-sending errors.
In extreme cases (the communication line is suddenly interrupted), TCP may report an error, and the application can establish a connection and re-transmit it.

However, if there is a third-party attack (such as eavesdroppers or line faults) in the middle, TCP cannot guarantee that your data is completely connected to the other party. Therefore, in key data interaction scenarios (such as online transactions ), it must be controlled through the application layer protocol.

Asklxf (Xuefeng) on the 14th floor replied to 23:32:22 score 0
The send functions are different in each language.
If it is an asynchronous call, it is not guaranteed to be sent correctly
But I use java.net. socket.
As long as no ioexception occurs, it must be correctly sent out.

 

Awjx (La la) on the 15th floor replied with a score of 0 at 16:30:23
I went to school on TCP/IP. How is it different from what I went upstairs?

16 floor xlfrd (highlighted. net) reply to 19:54:37 score 0 if it is in blocking mode, send should be returned after the other party's TCP protocol is confirmed, instead of simply getting the response from the local TCP module, otherwise send will always return success, you can test it as follows. After the connection is established, the physical line is disconnected, and the operating system does not immediately know that the physical link is disconnected (such as dialing), and the program then calls send, if the return fails, it indicates that send is returned only after confirmation. If the return is successful, it means that send only needs to be sent to the local TCP to return.
,
Ypos (Ye Kai) on the 17 th Floor replied to-10-13 08:59:41, score 0
Implement another window control on TCP, buffer, and confirm how to send?

Deva (zhyi) on the 18 th Floor, with a score of 0 from 18:07:23
TCP is used for reliability propagation.
However, if there is a large amount of data and the network will soon repeat sequence
This is the main problem !!

Esdn (easy) on the 19th floor replied to the score of 10 at 11:32:10
I would like to express my views on wwwdfq1977 (qswl) and asklxf (Xuefeng.

First, after the TCP send function is called, the data in the application must be transmitted to the TCP sending buffer occupied by the TCP/IP protocol.
Then, when the send function returns to the application, whether the data is sent depends on whether the Nagle algorithm is enabled.
1. If the Nagle algorithm is not enabled, the sender's TCP/IP protocol occupies the sending action.
2. if enabled, check whether the sending conditions are met (for example, whether the buffer zone has an MSS data ).
If yes, it is sent and then returned.
If the recipient receives the message and sends a further confirmation message, it cannot be determined.
The basic principle is as follows. The specific system implementation may also have a mechanism.

Asklxf (Xuefeng) on the 20th floor, with a score of 0 at 18:40:24
I think the main problem is: Should I use the Q & A mechanism to ensure no packet loss?
However, the purpose of the TCP protocol design is to use the answer mechanism to ensure reliable transmission. TCP has already implemented this mechanism at the transport layer. What is the significance if it is implemented at the application layer? It is better to use UDP instead.
TCP itself uses a Sliding Window model, which confirms each packet. Using a TCP connection is concerned with establishing a connection, closing the connection, and exceptions caused by connection interruption, you should transfer the data to TCP.
In school movies, a file is often 600-700 m, and the transmission speed is 500 KB-2 Mb/s. If TCP is unreliable, how can I ensure that several g downloads at a time are not faulty? The FTP protocol is based on TCP.

Xiaohyy (drunken ELE. Me) on the 21st floor replied to the question about the send function with a score of 0 at 20:34:02. Sending should be returned only after the recipient receives the confirmation. It is very simple. From the error message returned by the send function, we can see that here are a few wsaetimedout: this error is caused by network failure or remote host crash or wsaeconnreset error. This error is caused by the forced exit or death of programs on the remote host. How can I detect these problems if wwwdfq1977 (qswl) only sends data to the local transmission layer?
You can also set breakpoints in the send function. If the buffer is large enough, it is not returned immediately, but there is still a certain latency. This also indicates that the send function has done some underlying work and received confirmation information from the other party.

On the 22nd floor, wwwdfq1977 (qswl) replied to-10-17 21:08:58, with a score of 0.
Can I see that the error is in conflict with my statement?
The send function does not just pass data to the transport layer, but does it have to wait for the response from the other party to return the data? Is there an error that cannot be returned during the next call?
The buffer zone is too large and there is latency. What is the relationship between it and the confirmation information it receives, because it is entirely possible to do other work.
Note that the thread where the TCP protocol is located is no longer in the same thread as the application. If you do not send a response to TCP, nothing will be done, during this period, he may do some work such as setting errors or normal state, so that the program will return these errors the next time it calls send or closesocket.

I think you should take a good look at all the posts above.

On the 23rd floor, pc200300 (just want to mix bowl meals...) replied to 22:05:49 score 10 I 'd like to express my opinion:

(1) If the network cable or other hardware fails, for example, if the network cable is disconnected or the socket of one party does not reflect any fault, a heartbeat test is usually used to check whether a connection is normal;
(2) The send () function only writes the data you want to send to your socket buffer, and does not guarantee that the recipient can receive the data correctly;
(3) TCP is a reliable connection. It has its own security mechanism, but it cannot guarantee that the peer can receive it correctly;
(4) My solution: define a set of data transmission protocols at the application layer, and define a set of good security protocols by looking at the methods provided by a few digits.

On the 24th floor, chengwei_xj (ylxz.blogbus.com) replied to 13:41:31, with a score of 0. I'm glad that my post has aroused wide and in-depth discussions. Thank you.