Introduction to Receive Side Scaling (RSS)
Overview
A network interface card (NIC) can be configured with multiple receive queue (Rx). This is done to improve the throughput (i.e, the number of packets that can be processed per second) of the device and to achieve higher throughput, more than one CPU is used to poll packets from the device independently. Each CPU polls for its assigned Rx queue, but still we need a strategy to distribute the packets over all the queues, otherwise all the packets will go to one queue and only one CPU will be utilised.
Few Terminologies
RSS Input Fields
The RSS input fields plays a vital role in deciding the criteria for distributing the packets over multiple receive queue. There are lot of hash input fields available like source mac address, destination mac address, source IP, destination IP etc.
Note : Support of above hash functions depends on the vendor of the NIC.
RSS Hash Key
The RSS hash is a 40/52 byte key used to randomize the distribution of packets over the receive queues. The size of the hash key depends on the hardware of the NIC.
Hash Algorithm
Once the hash function and hash key is configured, the hash algorithm is used to determine the receive queue index using the input fields and RSS Key.
The hash functions which are generally used are:
- Simple XOR Hash Function
- Toeplitz Hash Function
Working of RSS
When a NIC is started with multiple Rx queues then based on the configured RSS configurations like Input Fields, RSS and number of Rx queues, an indirection table is generated. An indirection table represents mapping between the hash value and the queue index.
When a packet is received by the NIC, all the values from the configured fields are extracted and those values are fed into the hash function. The generated hash value is used to do a lookup in the indirection table in order to get the destination queue index.
There are two main advantages of using RSS:
1. Higher throughput as more number of CPUs can be used independently.
2. The computational cost of load balancing is offloaded to the NIC’s hardware instead of doing software based load-balancing.
Note that the the implementation of RSS varies from vendor to vendor but the basic fundamental remains the same.
References
- https://docs.microsoft.com/en-us/windows-hardware/drivers/network/introduction-to-receive-side-scaling
- http://galsagie.github.io/2015/02/26/dpdk-tips-1/
- http://www.mellanox.com/related-docs/user_manuals/Ethernet_Adapters_Programming_Manual.pdf
- https://www.intel.com/content/www/us/en/support/articles/000006703/network-and-i-o/ethernet-products.html