Linux NIC buffer size

Reference interception part: 7764461

Under Linux You can modify the protocol stack to change the TCP buffer related parameters:

Modify the system socket buffer

echo 65536 >/proc/sys/net/core/rmem_max
echo 256960 >/proc/sys/net/core/wmem_max
echo 65536 >/proc/sys/net/core/wmen_default

Modify TCP receive/Send buffers
echo "4096 32768 65536" >/proc/sys/net/ipv4/tcp_rmem
echo "4096 65536 256960" >/proc/sys/net/ipv4/tcp_wmem

Modify network device Receive queue
echo >/proc/sys/net/core/netdev_max_backlog

Retransmission times
Echo 5 >/proc/sys/net/ipv4/tcp_retries2

found that the above parameters are small, since the big time video comparison card, change small will be good? The first problem is that the larger the buffer, the better? If the machine does not handle the TCP traffic, then no matter how big the buffer, the early evening overflow, which leads to the application layer know that TCP did not receive a relatively late, because in the buffer for a period of time, and the retransmission of the data is also large, will be a large network burden, in fact, it does not seem to benefit the entire network. So what's the best way to change the buffer zone? The buffer is large enough to handle bursts of large flow data, not when the picture changes, that is, the sudden increase in traffic when the buffer full. That comes back to see this problem, since the big time video card, then changed to small, let the application layer early know that TCP did not receive it, the whole network is also a bit of traffic, on the real-time video whether the card has little impact, their own analysis, to be verified.

If you suspect a machine problem, or a TCP configuration problem, you can switch to a machine (better configured) to see if it is not processing the delay caused by the request, or to use Wireshark to capture packets to count the traffic.

The following is from the network, the original source has not been found.

How much memory does the computer need? Of course the bigger the better, this is the user's idea. But the designer of the computer must compromise the cost, difficulty, and pleasing the customer, and choose the best balance. For the computer, its main basis is the market positioning of the product, high-end business PC with at least 2G of memory, low-end student machine with 256M is enough. If you use a 256M RAM student machine for complex large-scale FPGA simulation, you may find that the hard drive lights are always on, which means that the memory is not enough, the operating system is constantly exchanging data between the memory and the hard disk, with large capacity of low-speed hard disk to compensate for the lack of memory too small, But the cost is that the calculation time has been extended many times. Is the router also the same as the PC, mainly based on the price to determine the size of the memory configuration? Is it possible that the larger the memory, the better? What factors does the designer of the router depend on to determine the size of the memory configuration? Generally speaking, the memory of routers is mainly used for these aspects:

(1) for the storage of router software instructions and static data, the router and the PC is different, the PC is only the current running program loaded into RAM, but most routers are all installed into the RAM in a single boot, generally speaking, the router program is not small (a few megabytes to a few 10 trillion) ; (Note: This is primarily a control plane program, which is the routing engine for Cisco and Juniper)

(2) used to store dynamic data such as routing tables, OSPF link state databases, and so on. If a router needs to support up to 100,000 routes and 256 bytes per route, then approximately 200M of memory is required.

(3) For buffering data packets, the router works by storing and forwarding. In extreme cases, each interface of the router needs to buffer at least one message, otherwise the router will not work at all. The following focuses on this issue.

In general, the router configures a message buffer that has more than one message. Because this means that when a new message arrives, if the previous message is being sent, the message buffer is not yet idle, and the new message is bound to be discarded. When the previous message is sent out, the link is idle, but because the message has just been discarded, the link idle state is not available. If the discarded message is a TCP message, then the host is bound to re-transmit the message (two times the same message on a line in front of the router), and reduce its own sending window, reducing the rate of TCP connections.

That is, if the message buffer of the interface is too small, it will result in high packet loss rate, low data link utilization and low TCP transmission efficiency. So is the message buffer bigger the better? It is not, because the message buffer is large to a certain extent, it can not continue to improve data link utilization and reduce packet loss rate. If this router is in a congested state, the rate of receiving messages is much larger than the transmit bandwidth of the interface, no matter how large the message buffer will be filled, and the message buffer is large, then it means that the delay time of the message forwarding is very long. A message with too long delay is meaningless to the receiver. In the case of TCP connection, when the message is larger than the sender's retransmission time, the sender will retransmit the message, that is, the message that is greater than the arrival of the retransmission time of TCP is meaningless. For VoIP and other applications, the network delay is more sensitive.

In general, the size of the interface buffer of a router has a rule of thumb (rule-of-thumb): B = C * Rtt,c is the link rate, and RTT is the average packet-to-round. As to where this rule of thumb comes from, I am not seriously textual. But the main basis of this rule of thumb is to maximize TCP efficiency and maximize network interface bandwidth utilization. If the router is designed according to this rule, the problem is not very big for mid-low-end routers. But for high-end routers, there are challenges. Generally on the high-end Internet backbone routers will assume that the RTT is 250ms, then for a 10GE interface, the required memory is (10G bit/s * 0.25 s)/8 is about 300MB. Perhaps everyone will say, 300M is not big, but it can be foreseen that the capacity of the core router in the last two years will be developed to a single slot 80-160g, which means that the single need to 2.5g-5g memory. It's not completely achievable, but it's still a certain difficulty. As can be seen from a white paper by juniper (characteristics of switches and routers), Juniper is also designed in accordance with this rule of thumb.

But some recent research has suggested that (sizing router buffers, Guido Appenzler, Isaac Keslassy, Nick McKeown), the routers do not need that much memory, and each port only needs to buffer dozens of packets to be sufficient, This is sufficient with NPS or ASIC embedded RAM, without the need to configure external RAM. His main basis is that the previous rule of thumb is based on a single TCP flow, the author believes that this model is not correct, the actual backbone router is a lot of TCP flow, so should be based on B = C * RTT/SQRT (n) to calculate, n is the number of TCP flows. But other studies argue that this is not the case, that routers should not only consider TCP, but also many voice and video applications that are eager for UDP. There is still no consensus among the professors about the problem. The engineer is no longer arguing about the problem, and is designed in accordance with the B = C * RTT, is cost-acceptable, and is more secure.

To correct network problems, it is sometimes necessary to reconfigure the default IP packet size of the NIC. The maximum IP packet size that often occurs for routing is smaller than that of the NIC. The TCP protocol can be adaptive, but the UDP protocol does not (it causes the Shard to drop packets and then retransmit). So NFS over UDP is especially important to set the MTU size. You can use the command *tracepath* to see the MTU value on the network, with the Ifconfig command to see the MTU value of the NIC, to match the two. (most networks are 1500, unless a large package is set up), the IP packet is fragmented when it is transmitted with the UDP protocol. A large number of IP packet shards consume a large amount of CPU resources at both ends of the network, and can also cause network traffic to become more volatile (because the full RPC will have to retransmit the entire RPC in case any packet of UDP shards is lost). In the 2.2 and 2.4 cores, the default socket read Cache Rmem_default is 64k, and the write buffer wmem_default is 8k. These two values are important for situations where there is a lot of read and write load.

Linux NIC buffer size