Using RSS to improve the performance of DPDK apps

This article describes RSS and how to configure RSS in DPDK to achieve performance improvement and unified distribution.

DPDK 1.8.0

What is RSS

RSS (Receive Side Scaling) is a network card driver technology that allows receiving messages to be efficiently distributed across multiple CPUs in a multiprocessor system.

• The network card is connected to the received message and gets the IP address, protocol and port five tuple information.
• The network card calculates the hash value based on the five-tuple information by the configured hash function, or it can be calculated according to the two or three or four-tuple.
• The low number of hash values (this specific NIC may be different) as the index of the Reta (redirection table)
• distributed to the corresponding CPU according to the values stored in the Reta

Describes the complete process flow:

Based on the RSS technology program, the data stream can be distributed between multiple CPUs via hardware, and dynamic load balancing can be achieved by modifying the Reta.

Configuring RSS in DPDK

DPDK supports setting static hash values and configuring Reta.
However, RSS in DPDK is based on the port and is distributed according to the receiving queue of the port.
For example, we configured 3 receive queues (0,1,2) on one port and started RSS, so
That's the way it is:

{0,1,2,0,1,2,0..}

Applications running on different CPUs receive messages from different receive queues, thus achieving the effect of message distribution.

The RSS feature is enabled in the DPDK by setting rte_eth_conf The fields in the mq_mode
rx_mode.mq_mode = ETH_MQ_RX_RSS。

When the RSS function is turned on, the rte_pktmbuf hash value of the RSS calculation will be stored in the corresponding message, which can be pktmbuf.hash.rss accessed by.
This value can be used directly in subsequent message processing without the need to recalculate hash values, such as fast forwarding, identity message flow, etc.

The Reta is configurable at runtime so that the application can dynamically adjust the message distribution by changing the CPU's corresponding receive queue.
Specifically through the PMD module driver configuration, such as ixgbe_dev_rss_reta_update and ixgbe_dev_rss_reta_query .

Symmetric RSS

In a network application, if the same two-way message is distributed to the same CPU after RSS is turned on, this RSS is called symmetric RSS.
For network applications that need to hold some information for a connection, symmetric RSS can be a great help for performance improvements.
If the same two-way message is distributed to different CPUs, then the information that is shared between the two CPUs will involve locks, and the lock will obviously affect performance.

RSS generally uses the Toeplitz hashing algorithm, which has two inputs: a default hash key and five-tuple information extracted from the message.
DPDK uses the default hash key is recommended by Microsoft, the specific definition see lib/librte_pmd_e1000/igb_rxtx.c:1539,
The hash value calculated using this default value in different directions of the same connection is not the same.

Specifically, {src:1.1.1.1, dst:2.2.2.2, srcport:123, dstport:456} and {src:2.2.2.2, dst:1.1.1.1, srcport:456, dstport:123}
The calculated hash value is not the same, the hash value will cause two directions of the message is distributed to different receive queues, processed by different CPUs.

If you want to achieve the effect of the symmetric RSS, then you need to use other hash key to replace the DPDK currently used.
In the paper "Scalable TCP Session monitoring with symmetric receive-side Scaling" mentioned a hash key can be used to do the symmetric RSS

This gives the value of the hash key, the specific principle can refer to the paper.

static uint8_t rss_intel_key[40] = { 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, };

Using RSS to improve the performance of DPDK apps