[Chinese-English control] Introduction to Remote Direct Memory Access (RDMA) | RDMA Overview

Foreword: What is RDMA? In short,RDMA means that data is not transmitted over the network through the operating system (OS) kernel and the TCP/IP protocol stack , so latency (latency) is very low and CPU consumption is minimal. Here is a brief introduction to the RDMA of the article in English-controlled translation.

Introduction to Remote Direct Memory Access (RDMA) | RDMA Overview

1. What is RDMA? | What is RDMA

Direct Memory Access (DMA) is an ability of a device to access host memory directly, without the intervention of the CPU (s ).

RDMA (Remote DMA) is the ability of accessing (i.e. reading from or writing to) memory on a Remote machine without interru Pting the processing of the CPU (s) on the that system.

Xxxx

2. So? Why are this good? | Why is it so good?

Using RDMA has the following major advantages:

• Zero-copy-applications can perform data transfer without the network software stack involvement and data is being send R Eceived directly to the buffers without being copied between the network layers.
• Kernel bypass-applications can perform data transfer directly from userspace without the need to perform context Switche S.
• No CPU Involvement-applications can access remote memory without consuming any CPU on the remote machine. The remote memory machine is being read without any intervention of the remote process (or processor). The caches in the remote CPU (s) won ' t is filled with the accessed memory content.
• Message based transactions-the data is handled as discrete messages and not as a stream, which eliminates the need of th e application to separate the stream into different messages/transactions.
• Scatter/gather entries SUPPORT-RDMA supports natively working with multiple Scatter/gather entries i.e. reading multiple Memory buffers and sending them as one stream or getting one stream and writing it to multiple memory buffers.

Xxxx

3. Where can I find RDMA? | Where does RDMA need to be used?

You can find RDMA in industries this need at least one the following:

• Low Latency -for EXAMPLE:HPC, financial services, Web 2.0
• High Bandwidth -for EXAMPLE:HPC, medical appliances, storage and backup systems, cloud computing
• Small CPU Footprint -for EXAMPLE:HPC, cloud computing

And in Many-many more other industries ...

Xxxx

4. Which network protocols support RDMA? | What are the RDMA-capable network protocols?

Today, there is several network protocols which support RDMA:

• InfiniBand (IB) -A New Generation network protocol which supports RDMA natively from the beginning. Since This was a new network technology, it requires NICs and switches which supports this technology.
• RDMA over converged Ethernet (RoCE) – A network protocol which allows performing RDMA over Ethernet network. Its lower network headers is Ethernet headers and its upper network headers (including the data) is InfiniBand headers. This allows using RDMA-over-standard Ethernet infrastructure (switches). Only the NICs should is special and support RoCE.
• Internet Wide Area RDMA Protocol (IWARP) -a network Protocol which allows performing RDMA over TCP. There is features that exist in IB and RoCE and aren ' t supported in IWARP. This allows using RDMA-over-standard Ethernet infrastructure (switches). Only the NICs should is special and support IWARP (if CPU offloads is used) otherwise, all IWARP stacks can is implemente D in SW and loosing most of the RDMA performance advantages.

Xxxx

5. Does it mean that I need to learn several programming APIs? | Need to learn the programming API?

No. Luckily, the same API (i.e. verbs) can be used for all the above-mentioned RDMA enabled network protocols. In *nix it was libibverbs and kernel verbs and in Windows it was Network Direct (ND).

Xxxx

6. Is those network protocols interoperable? | Can network protocols interoperate?

Since those is different network protocols, their packets is completely different and they cannot send/receive messages Directly without any router/gateway between them. However, the same code can support all of the them. Since all those network protocols support Libibverbs, the same binary can is used without even the need to recompile the S Ource code.

Xxx

7. Do I need to download special packages to use RDMA or are it part of the Operating System? | Do I need to pack extra packages with RDMA? Or is RDMA part of the OS?

For several Operating Systems, RDMA support is embedded within the kernel. For example, the Linux which supports RDMA natively and all major the Linux distributions support it. Other Operating Systems could need to download a package (such as OFED) to add RDMA support to it.

Xxxx

Recommended reading:

1. InfiniBand version evolution, basic concept, mass transfer speed
2. Rdma:https://en.wikipedia.org/wiki/remote_direct_memory_access
3. Ib:https://en.wikipedia.org/wiki/infiniband
4. Iser:https://en.wikipedia.org/wiki/iscsi_extensions_for_rdma
5. Iwarp:https://en.wikipedia.org/wiki/iwarp
6. Roce:https://en.wikipedia.org/wiki/rdma_over_converged_ethernet
7. Sdp:https://en.wikipedia.org/wiki/sockets_direct_protocol
8. Srp:https://en.wikipedia.org/wiki/scsi_rdma_protocol

[Chinese-English control] Introduction to Remote Direct Memory Access (RDMA) | RDMA Overview