Cost-effective RDMA implementation in the DB2 client server environment

LAN network devices such as host adapters and Ethernet switches have been greatly developed over the past few years, making it possible to achieve a throughput of up to 40 Gbps and a network latency below 1 microsecond. Computing power and LAN devices are no longer a bottleneck.

LAN network devices such as host adapters and Ethernet switches have been greatly developed over the past few years, making it possible to achieve a throughput of up to 40 Gbps and network latency below 1 microsecond. Computing power and LAN devices are no longer a bottleneck.

On the contrary, in this new era of high-performance hardware, protocols such as TCP have become a real bottleneck. To overcome TCP limitations, new protocols such as Remote Direct Memory Access (RDMA) have been developed. This article will introduce a cost-effective method that can be used to implement RDMA In the DB2 client environment without re-coding or re-compiling existing applications.

RDMA is a mechanism that allows computers to access memory locations on other computers by bypassing the Operating System (kernel and TCP stack. Compared with the traditional TCP-based hardware and software architecture, RDMA has several advantages. Kernel bypass means that the path between two applications is shortened and the CPU usage is reduced. Data is transmitted directly between the network adapter and the application memory (user space) to reduce TCP-specific protocol overhead. data does not need to be copied to the kernel space for caching.

To make full use of all the advantages provided by RDMA, you need to use RDMA semantics or upper-layer protocols, such as User-Level Direct Access Transport (uDAPL) or Message Passing Interface (MPI) to write applications. However, for RDMA, rewriting a TCP application may be very expensive, so an alternative solution is developed for this situation. This method is called Direct Socket Protocol (SDP) without re-coding any application.

SDP is a connection protocol used between RDMA-capable adapters and sockets. For this reason, SDP is transparent to the application, and the implementation of the standard stream socket does not need to be replaced by another API. DB2 applications and DB2 servers can run on SDP or TCP without modification. You only need to pre-load the SDP shared library to select the Protocol to use before executing the application. No modification is required for TCP settings, such as host name, IP address, and port.

For example, a Java application that uses TCP to connect to the server can also run on SDP using the same jdbc url. Once pre-loaded, the SDP library determines which Protocol to start based on a set of rules defined in/etc/libsdp. conf and the Protocols accepted by the server. The default rule specifies SDP as the first option. If the connection fails, the SDP library will return to TCP.

An application can only use SDP, TCP, or both. For example, you can configure an application to use SDP for DB2 database connection and use TCP for LDAP connection. The database and LDAP server can run on different physical machines, or run on the same machine but listen on different interfaces. This article will discuss various scenarios and how to apply rules later.

Hardware and software infrastructure

RDMA requires dedicated hardware and software infrastructure. This article will discuss SDP on Linux x86 platform.

• Host Adapter

  Two types of host adapters support RDMA: Infiniband adapter and RoCE adapter. The former requires an Infiniband Switch, while the latter requires an Ethernet switch.

  This example uses the Mellanox RoCE adapter. All commands provided in this article apply to the Infiniband Adapter. You only need to make slight modifications or do not need to modify them. RoCE (RDMA over Converged Ethernet) is an implementation of Infiniband over Ethernet protocol. The Converged Ethernet network allows different types of protocols to share the same media. The LAN Ethernet framework that periodically encapsulates IP packets, Infiniband over Ethernet, and Fiber Channel over Ethernet can coexist on one Ethernet line.

• Device Driver

  Here are a few device drivers and tools required to operate the Mellanox adapter. All available device drivers and tools are based on the OFED stack maintained by OpenFabrics Alliance.

• Operating System

  Red Hat Linux 6.2, kernel version 2.6.32-220.4.1.el6.x86 _ 64.

• Ethernet Switch

  10 Gbps RackSwitch G8264, operating system version 6.8.1.0.