Bulk Network packet transmission

Bulk Network packet transmission

Translation zhangzl2013
Link: http://www.cnblogs.com/zhangzl2013/p/bulk_network_packet_transmission.html
Original Jonathan Corbet
Original link: Bulk network packet transmission
This article may be reproduced, resulting in the fragmentation of comment messages. Students who want to participate in comments and discussions, please find the original address of the original or the translation, and discuss with the original or the author.

Batch processing is one of the key factors in the performance of modern systems--doing as much as possible within a relatively intrinsic overhead. For example, if a subsystem has to acquire a lock when it does something, it can reduce the total cost of the system by getting it done as much as possible when the lock is acquired. There has been a lot of work done in recent years to improve performance by maximizing batch processing operations. The recent addition of batch processing to the network subsystem has also allowed it to improve its performance.

Each time a packet is transmitted over the network, some column operations are required. This includes getting the lock of the send queue, passing the packet to the driver, inserting the packet into the device transmission queue, and notifying the device to start the transmission. There are some operations that must be targeted for each packet, but there is no. For example, the operation to get a queue lock is not, and the overhead of notifying the device to start the transfer is significant. It involves hardware errors and, on some systems, the invocation of privileged instructions.

When a packet is transmitted, there are usually other packets in the queue, and the network traffic itself can be transmitted in bundles. So the network protocol stack can try to average the intrinsic overhead to multiple packets. Segmentation offload technology will do similar batch processing optimizations. But in the current kernel, if there is a series of packets to be sent in the network stack, it is also an inefficient way to send them every time.

This situation changes after merging a small set of patches in 3.18. Look at some of the following functions for sending packets to the driver export:

netdev_tx_t    (*ndo_start_xmit) (structstruct net_device *dev);

This function takes a SKB pointer and sends it out through the dev device. Each invocation is a separate operation with a fixed overhead. 3.18, plan to add a new function to the driver:

void (*ndo_xmit_flush) (struct net_device *dev, U16 queue);

If the driver provides this function, it means that the network stack can be transferred in bulk. The network stack can accept multiple ndo_start_xmit () function calls, but the driver does not start sending. After a series of calls, call the Ndo_xmit_flush () function to get the driver to start a real transfer.

Given the added overhead of adding a function call to the transfer path, the function is almost just added to the code base and removed. Instead, a new Boolean variable Xmit_more is added to the sk_buff structure. If it is true, it means there are more packets to pass, and the driver can postpone the transfer of the hardware. This variable allows you to provide enough information to the driver without having to introduce a new function.

This mechanism, added by David Miller, enables bulk operations. Some of the drivers were supported, but David did not modify the network stack himself, but was implemented by Jesper Dangaard Brouer adding "bulk dequeue" patches in 3.18. At the moment, it's a small scope, just for a single transmission queue.

Jesper changes are small: When a packet is to be transmitted, the network stack tries to send more packets at once after the lock is acquired. The "byte queue limit" mechanism limits the maximum number of pending data. Once this limit is reached, the Skb->xmit_more will be set to false, thus starting the real transmission.

Eric Dumazet found that this patch set can be further improved by confirming that the work of the packet transfer can be placed entirely outside the queue lock, thereby increasing the concurrency of the system. Eric's patch works great: Even without segmentation offload technology, it can reach speeds of 40Gb per second. There is no doubt that this patch has been merged into 3.18.

Although the changes are small, small changes can have a big impact. This small change allows the 3.18 kernel to have the fastest network stack available.

--End--

Bulk Network packet transmission