TCP Series 36-Window Management & Flow Control-10, Linux exception message series receive

In this article we look at the processing of the server side when it receives the exception data series, the main purpose is to have an intuitive understanding of the processing of these exception data series through the Wireshark example, the interested self-reading the relevant code and protocol, which is no longer described in detail

Before you perform the following tests, first set the relevant parameters, where the window parameter specifies the maximum receive window for the TCP connection to 127.0.0.2.

 
   
  

  
    
   
[email protected]:/home/******/tcp12# ip route change local 127.0.0.2 dev lo window 40
 
   
  

One, wireshark example

1, a packet part of the data after RCV_NXT

As shown, after the client establishes a connection with the server, the first issue of the NO4 packet, the serial number corresponding to [1,10], the text column in the TCP packet represents the actual transmission of data, No4 len=10, the application layer actually transmits the data content is "0123456789". The client then transmits the packet with the serial number [6,15] in the NO6 packet, and the content of the transmission is also "0123456789", the same length is 10bytes.

Can see here No4 and No6 two packet part serial number repeats, NO4 packet serial number [6,10] corresponds to the string "56789", and NO6 packet serial number [6,10] corresponds to the string "01234". Server side for NO6 Packet Normal reply to an Ack message, ack=16, that is, confirm the serial number 16 before the data, while you can see NO7 this Ack confirmation packet also comes with a dsack block, indicating that the sending side received a duplicate message.

The final server-side application layer actually reads the 15bytes data, the content is "012345678956789", the server side of the NO6 message serial number [6,10] corresponding data is discarded, but retains the serial number [11,15] content.

2, a disorderly order package completely outside the receiving window

As shown, after the client establishes a connection with the server, the NO4 message is sent first, and then the NO6 message is sent. From No5 This confirmation package can be seen ack+win=51, that is, the data starting from the serial number 51 is to receive outside the window data (including serial number 51 corresponding to the byte). The serial number corresponding to the NO6 message is [51,60] exactly outside the receiving window, so the server side loses the NO6 packet. The server-side application layer eventually reads 10bytes of data, which is "0123456789".

3, a disorderly order packet part in the receiving window outside

This example differs from the previous one in that the NO6 series number range is [46,55], that is, the packet [46,50] serial number is in the Receive window, and [51,55] is outside the receive window.

Then see No7,no7 through the sack complete confirmation of the NO6 packet, the outside of the receiving window of the [51,55] serial number is also fully confirmed, note this example and the above comparison. When the sequence number falls outside the receiving window, Linux receives the packet normally and completely, including data that falls outside the receiving window. And when the packet falls completely outside the receiving window, Linux does not receive the packet.

It is also noteworthy that no8-no11 four times the process of waving and closing the connection, No8 carried the sack information, No9 replied to an ACK, note No10 This fin packet starting serial number for the 11,server end of the normal receive this packet, This means that the server will discard the incoming NO6 messages, and the application layer cannot be read successfully.

4. Continuous packet falling outside the receiving window under the fast path

Linux is divided into a fast path and a slow path for the received packet processing process, which enters a slow processing path in scenarios such as receiving an emergency pointer, a change in the window size in the TCP header, and a slower path that performs more check processing than the fast path. Let's take a look at the case where a continuous packet falls outside the receiving window under the fast path. In order not to let the server side update the Receive window, we secure the server-side receive cache through the SO_RCVBUF option, and we have the server side enter the delay ACK mode through a specific packet. Cancel the window parameter set through the routing table before performing the following tests.

The related data interaction is as follows, the NO4 packet is used to trigger the server side to quickly enter the delay ACK mode, after the server replies NO5 This confirmation packet will enter the delay ACK mode. Refer to the previous article for delay ACK related content. At the same time note the NO5 message notification win=2, that is, the receiving window only 2bytes left.

We then see that the NO6-NO32 message sent by the client does not trigger the server-side reply ACK packet. Where the NO6 message part falls outside the receiving window, and the NO7-NO32 message is a packet that falls completely outside the receiving window. From No34 This confirmation package can see that the server side has received the No6 that partially fell inside the receiving window and the no7-no32 that fell completely outside the receiving window. For No33, it is not received, and then it is not received on the server side of the sending NO35 packet.

The reason NO6-NO32 can be properly received here is that TCP in Linux reserves a certain amount of cache and does not perform strict checks when it is in fast processing mode, as long as the newly received packet consumes more cache than the reserved cache will normally receive. The NO6-NO32 does not exceed the reserved cache, so the server side is receiving normally. However, when the server side receives the NO34, the reserved cache is insufficient, it will discard the packet directly, and enter the Quick ACK mode to reply to an ACK immediately, you can see that the No34 is a 0-window ACK message, so the server side also exits the fast processing mode of the packet. Then, when the No35 is received, it enters the packet slow processing process, and when the check is performed, it is found that the receiving window is 0,no35 to the outside of the receiving window, so it will discard the packet and immediately reply to an ACK acknowledgement packet.

5. Continuous packet falling outside the receiving window under slow path

In fact, the last example of No35 processing has been processed under a slow path. We are looking at another similar example. The processing of the NO1-NO15 message in the example is similar to the example above, except that the win for this packet is updated No16. Server side when receiving NO16 this packet, found that No16 updated window size will exit the fast path processing into a slow path to perform more complex inspection and processing. In the slow path, the server side discovers that the No16 falls outside the receiving window, so drop the packet directly and enter the Quick ACK mode to reply to a No17 confirmation packet immediately, reply to No17 confirm packet, find Win=0,server side TCP exits the packet fast processing mode , and then directly into the slow path processing when new packets are received.

6. Partial overlap of the contents of the disorderly sequence package

As shown, the client sends 5 packets in turn, with the associated TCP interaction as shown

For the sake of comparison, I have five packets corresponding to do as shown, where the first line represents the serial number bit, the second row is the server side of the application layer actually read the data stream, the remaining 5 lines represent the client sent 5 packets, where the red part of the packet indicates that the server side is eventually discarded , the green portion of the packet indicates that it is eventually received by the server side. It can be seen that once a newly received message overlaps with the previous message, if the front end of the new receiving message overlaps, the front-end content of the new receive message is discarded, and if the backend overlaps, the contents of the previously received message will be discarded.

Additional notes:

1, Linux related function packet receive processing: tcp_rcv_established, Tcp_data_queue, Tcp_data_queue_ofo

2, slow path fast path and other related functions: Tcp_fast_path_check, the focus is TP->PRED_FLAGS this flag variable in different places of the update

From for notes (Wiz)

TCP Series 36-Window Management & Flow Control-10, Linux exception message series receive