Sock_stream & Sock_dgram

Brain Map: Socket

From the length of the UDP datagram.

UDP belongs to the transport layer in the network model. Let's look at the next step from the bottom to the previous step:

Theoretically, the maximum IP packet (packet) allowed by the IP protocol is 2^16=65535 (the total IP packet length is 16 bits):

But! The length of the Ethernet (Ethernet) data frame must be between 46-1500 bytes, which is determined by the physical characteristics of the Ethernet. This 1500 byte is called the MTU of the link layer (the Maximum transmission unit). But this does not mean that the length of the link layer is limited to 1500 bytes, in fact, this MTU refers to the data area of the link layer, not including the link layer header and the tail of the 18 bytes. Therefore, 1500 bytes is the length limit of the network layer IP datagram.

Because the header of the IP datagram is 20 bytes, the data for the IP datagram has a maximum of 1480 bytes. And this 1480 bytes is used to put TCP packets sent from the TCP packet or UDP transmitted UDP datagram.

Because of the first 8 bytes of the UDP datagram, the data area of the UDP datagram has a maximum length of 1472 bytes. 1472 Bytes is the number of bytes that a UDP datagram can use.

So the question is, what if the application layer transmits more than 1472 bytes of data?

This means that the IP datagram is larger than 1500 bytes and is greater than the MTU. At this point the sender IP layer needs to be fragmented (fragmentation)-dividing the datagram into slices so that each piece is smaller than the MTU. The receiver IP layer is required to reorganize the datagram. This enables the fragmentation and reassembly process to be transparent to the transport layer (TCP/UDP)--TCP/UDP protocol without paying attention to whether the data is too long.

Although the IP shard process appears transparent, for the TCP protocol , there is a flaw: the entire datagram is re-transmitted even if only one piece of data is lost. Why? Because the IP layer itself does not have a time-out retransmission mechanism-a higher layer (TCP) is responsible for timeouts and retransmissions. When a piece from a TCP packet is lost, TCP will re-send the entire TCP segment after the timeout, which corresponds to an IP datagram (not a shard), and there is no way to retransmit only one of the data shards in the datagram.

The udp--protocol itself does not manage the reliability of the data, and there is no retransmission mechanism. Because of the characteristics of UDP, when a piece of data transmission is lost. The receiver will not be able to reorganize the message, causing the entire UDP message to be discarded. Therefore, in the LAN environment, it is generally recommended to control UDP packets below 1472byte is advisable. Internet programming is different-because routers on the Internet may set the MTU to a different value. If we assume that the MTU is sending data to 1500来, and that the MTU value of a network passing through is less than 1500 bytes, then the system will use a series of mechanisms to adjust the MTU value, so that the datagram can reach the destination smoothly, so that there are many unnecessary operations. Because the standard MTU value on the Internet is 576 bytes, it is recommended that UDP data-length controls be within 576-8-20=548 bytes for UDP programming on the Internet.

MSS (maxitum Segment Size)

In fact, the use of TCP protocol for data transmission will not cause IP fragmentation, because once the TCP data is too large, over the MSS, the transport layer will fragment TCP packets. Since MSS generally less than MTU,IP layer for TCP fragment data will not have to be fragmented.

In order to achieve the best transmission performance, the TCP protocol usually negotiates the MSS value of both sides when establishing the connection. This value TCP protocol at the time of implementation is often replaced with the MTU value (need to subtract the IP packet header size 20Bytes and TCP data Segment header 20Bytes) so often MSS for 1460. Both sides of the communication will determine the maximum MSS value for this connection based on the MSS value provided by both parties.

So how do you segment it? In fact, TCP does not matter fragmentation, because each TCP datagram in the composition before its size has been limited by the MSS, so the length of the TCP datagram is not more than the MSS.

Summary : A fragment or shard occurs when the message is large in length. Fragmentation occurs in the transport layer of the TCP protocol, where shards occur at the network layer of the IP protocol.

About Kernel buffers

sock_dgram Types of socket features:

1. There is only one receive buffer, and no send buffer exists. The data is sent directly, regardless of whether the peer receives it properly, or whether the peer receive buffer is full.
2. Since UDP is not traffic-controlled, the sender can easily overwhelm the receiver (slower), causing the receiver's UDP drop datagram.
3. It is also important to note that UDP packets are not guaranteed in order, so the messages in the receive buffer need to be sorted manually (if necessary);
4. The good news is that UDP does not need to be manually disassembled-datagram-oriented messages are not automatically merged in the buffer.

For TCP sockets:

1. The kernel has a send buffer and a receive buffer--tcp full-duplex mode of operation, and the TCP sliding window is dependent on the two independent buufer and buffer fill state.
2. The data is sent to the end, and the kernel is stored in the receive buffer. The buffer data is retained until the application layer read () takes the data, and in this process the TCP/IP stack continues to execute, populating the new packet data with the receive buffer until it fills up.
3. The next action is to notify the window in the peer TCP protocol to close. This is the implementation of the sliding window to ensure that the TCP socket receive buffer does not overflow, so that TCP is reliable transmission. Because the other party is not allowed to emit more than the advertised window size data. This is the traffic control of TCP, and if the other party ignores the window size and emits more data than the window size, the receiver TCP discards it.

About design strategies and issues for TCP Why TCP uses byte-stream protocol

In fact, this difference is determined by the characteristics of TCP and UDP. TCP is connection-oriented, that is, in the process of continuous connection, the data received in the socket is issued by the same host (hijacking what is not considered), so long as the data is ordered to arrive on the line, as for each read how much data themselves to do. (Q: Why not just assign a packet to the protocol, do it?) ）
UDP is a non-connected protocol, which means that any host can send data to the receiving end as long as it knows the IP and port of the receiving end and the network is accessible. At this point, if you can read more than one message at a time, the data will be confused. For example, host A to send a message P1, Host B sent a message P2, if can read more than one message data, then will be P1 and P2 data merge together, such data is meaningless.

Adhesive bags, packaging and unpacking

Since the TCP message is stream-oriented and does not have any boundary records, all messages will be read in conjunction with the parameter (buffer_length) when read (), and must be unpacked by hand;

"Sticky packs" can occur on the sending side and can occur at the receiving end:

• The sticky packet of the sending end caused by the Nagle algorithm: The Nagle algorithm is an algorithm to improve the network transmission efficiency. Simply put, when we submit a piece of data to TCP to send, TCP does not send this data immediately, but instead waits for a short period of time to see if there is any data to be sent during the wait, and if so, sends the two pieces of data at once.
• Receive end-of-time receive-side sticky packet: TCP will present the received data in its own buffer, and then notify the application layer to fetch the data. When the application layer is unable to get the TCP data out in time for some reason, it will cause the TCP buffer to hold more than a few pieces of data.

Packet is to give a piece of data with Baotou, so that the packet is divided into Baotou and the package two parts of the content (filtering illegal packets will be added "packet tail" content). Baotou is actually a fixed-size structure, which has a struct member variable to represent the length of the package, which is a very important variable. Other struct members can be defined themselves as needed. Depending on the length of the Baotou and the variable containing the length of the packet in the Baotou, a complete packet can be split correctly.

Traffic control and congestion control for TCP

Due to the receiver cache limit, the Send window cannot be larger than the receiver receive window. There is a field in the header of the message section called the window (Rwnd), which is used to tell the other side of the receiving window, the size of the visible window can be changed.

　　

As a summary, TCP traffic and congestion control adopts the strategy of "slow start", "additive increase" and "multiplicative reduction".

• Slow start: The initial window value is small, but gradually increases exponentially, until the slow start threshold (Ssthresh) is reached.
• Additive increase: When the window value reaches the slow start threshold, each message segment is sent, and the window value increases by one unit amount.
• Multiplicative subtraction: Whenever a timeout occurs, the window value is reduced by half.

Thinking

1. Why does the TCP packet not automatically implement the packet? Or, in the software world, for example, C + + defaults to a stream as an external representation of a string, which seems to be a consideration of some kind of advantage. However, languages such as Python specify string lengths (including the processing of strings in ZMQ, as well as standard processing by specifying the length) by encapsulating the length attribute. What are the latter considerations?
2. What is the comparison between "byte stream" and "datagram" transfer efficiency?

Sock_stream & Sock_dgram