RDMA Explained: The Backbone of High-Performance Computing

Enterprises, cloud providers, and data centers must meet high-performance networking and low-latency data transfer requirements for efficiency.
Traditional networking methods show limitations because they cannot handle the growing data demands of AI/ML workloads.
That’s where Remote Direct Memory Access (RDMA) comes into play. RDMA allows direct data transfers between two computers’ memories without involving the CPU, operating system, or most network stack components.

In this article, we will provide an overview of RDMA technology, including its functionality and operation, main protocols, comparisons with TCP/IP networks, practical applications, and common pitfalls. Systems engineers, cloud architects, and anyone passionate about internet technology will find this guide a valuable resource on how RDMA can revolutionize network infrastructure.

• RDMA enables ultra-fast, low-latency data transfers by allowing direct memory access between servers, bypassing the CPU and OS.
• RDMA achieves much lower latency, higher throughput, and CPU efficiency than traditional TCP/IP networking.
• For proper use of RDMA technology, specialized network interface cards, network configurations, and RDMA-aware applications are required.
• RDMA technology is used in various fields, such as high-performance computing, artificial intelligence, cloud storage, virtualization, and real-time analytics.
• Optimal performance with RDMA technology requires accurate configuration of hardware, software, and tuning processes.

RDMA enables direct data transfers from one computer’s memory to another without involving the remote system’s CPU or operating system.
Traditional networking systems, like TCP/IP, process data packets by moving them through each layer of the OS networking stack. This involves copying data between buffers at every stage and consuming CPU resources for each packet.

RDMA in Action: The “Zero-Copy” Concept

Traditional networking:

• Data transitions from user space into kernel space, travels through the network stack, and returns to user space on the receiving end.
• Each copy and stack traversal operation increases latency and consumes CPU processing power.

• Data moves directly between machines without additional copies, Zero-copy networking.
• Once the initial setup is complete, the operating system no longer handles operations.
• The CPU can direct its resources towards application logic.

The following describes the internal workflow of RDMA data transfer between two systems—Requester (initiator) and Responder (receiver):

Image source

• CPU: Manages the application logic and posts work queue entries (WQEs).
• PCIe: The interface that connects the CPU’s memory to the network interface card (RNIC).
• WQE Cache / DMA Engine: Offloads data transfer tasks from the CPU and directly interacts with memory to move data efficiently.
• ORT (Outstanding Request Table), Reordering Buffer, Bitmaps, Transport: Maintaining message order, ensuring reliable data transport, and reassembling data within the RNIC.
• Receiving Buffer / Basic NIC: The final storage for incoming data on the responder’s RNIC.

1. The user submits a work request to the send queue; the RNIC retrieves this request and temporarily stores it in a local cache.
2. Request processing continues with the RNIC parsing the request, converting the virtual address to a physical one using a memory translation table, and logging it in a table for quick retransmission if needed.
3. Incoming data is queued into the receiving buffer, where any out-of-order packets are identified and reordered as necessary.
4. Upon receipt of a data packet, the RNIC matches it to the corresponding received work request, fetches the destination buffer address, and transfers the reordered data directly into the host memory.
5. It then sends an acknowledgment (ACK) back to the requester and informs the user application that the data has been received.
6. Finally, when the ACK is received, the RNIC updates the completion queue to indicate that the original send request has been completed.

Summary

• The Requester posts a send request and the RNIC takes control of data movement (no longer needs to engage the CPU after setup).
• The Requester’s memory sends data straight to the Responder’s memory using zero-copy networking.
• This architecture uses hardware acceleration, skipping most OS and CPU layers for high-performance and low-latency data transfer.

Understanding RDMA protocols will allow you to choose the optimal solution for your environment. RDMA protocols offer unique benefits and are suited for specific operational requirements.

InfiniBand

InfiniBand protocol delivers high performance for RDMA protocols by offering low latency and high throughput. High-performance computing environments rely heavily on InfiniBand, which powers numerous leading supercomputers worldwide.

Key characteristics of InfiniBand:

• Native RDMA support with hardware-level reliability
• Extremely low latency (sub-microsecond)
• Achieves High bandwidth capabilities that exceed 200 Gbps
• Requires specialized switches and adapters.
• Higher cost compared to Ethernet-based solutions

RDMA over Converged Ethernet (RoCE)

RoCE delivers RDMA functionality over standard Ethernet networks. This results in lower costs and improved accessibility when compared to InfiniBand. There are two versions of RoCE:

• RoCEv1: RoCEv1 operates at Layer 2 (link layer) but can only work within one broadcast domain.
• RoCEv2: RoCEv2 functions at Layer 3 (network layer), allowing packets to be directed across subnets, and supports a broad network system.

RoCE provides RDMA benefits for existing Ethernet setups, which creates an attractive solution for data centers wanting to enhance their network performance without replacing their existing infrastructure.

iWARP

The Internet Wide Area RDMA Protocol (iWARP) provides RDMA functionality over standard Ethernet networks using TCP/IP. The protocol wraps RDMA traffic inside TCP/IP structures, which provides direct memory access capabilities without special hardware modifications.

Advantages of iWARP:

• Works with standard Ethernet switches
• Leverages existing TCP/IP infrastructure
• Provides reliability through TCP’s error recovery mechanisms
• iWARP demands more memory resources than alternative RDMA protocols.

RDMA differentiates itself from traditional TCP/IP networking through its performance features, making it ideal for high-performance applications that require low latency. The following table contrasts their main attributes:

Below are essential hardware components:

• RDMA NIC: RDMA deployments rely on network interface cards that support RDMA functionality. These cards must support InfiniBand, RoCE, or iWARP protocol. Most customers choose RDMA-capable network cards from NVIDIA (Mellanox), Intel, and Chelsio.
• Network Infrastructure: To deploy RoCE networking, you must use Ethernet switches that offer the necessary features to maintain high RDMA performance. InfiniBand requires specialized InfiniBand switches.
• Memory Requirements: System memory must be sufficient to handle buffer registration during RDMA operations. Users should properly configure the ulimit -l setting to enable memory locking.
• System Requirements: The highest RDMA performance in GPU-accelerated systems requires the RDMA adapter and GPU to operate under the same PCIe root complex. This configuration provides minimal latency while optimizing bandwidth consumption.

RDMA has broad OS support, with Linux leading in high-performance environments:

RDMA Implementation on Linux

Because Linux provides strong RDMA support, it is the operating system of choice for high-performance network environments. RDMA functionality is enabled through the rdma-core package and specific kernel modules that establish direct communication with RDMA hardware.

The Linux RDMA Stack

The Linux RDMA stack is built from these essential components:

• Kernel Modules: RDMA operation modules can be found in the directory /lib/modules/$(uname -r)/kernel/drivers/infiniband/. The directory includes:
  • Hardware Device Drivers (hw/): Modules for InfiniBand and RDMA-capable Ethernet NICs.
  • Software Drivers (sw/): The software driver simulates RDMA functionality using standard Ethernet hardware.
  • Upper Layer Protocol Modules (ulp/): These modules support RDMA protocols including iSER(iSCSI Extensions for RDMA), SRP(SCSI RDMA Protocol), and IPoIB(IP over InfiniBand).
• User Space Libraries: Developers can use powerful user space libraries like libibverbs and librdmacm to build high-performance applications that interact directly with RDMA hardware. These libraries provide standardized APIs, simplifying the process of developing RDMA-aware software without digging into complex kernel-level interactions.
• Configuration Files: The file /etc/rdma/rdma.conf allows system administrators to control the selection and behavior of RDMA protocols. System configurations enable administrators to decide which drivers and protocols will remain active.

Practical Steps: RDMA Setup in Linux

Implementing RDMA on Linux involves several essential stages.

In the following example, we will show how to establish RDMA communication between two Linux nodes. We will assume both servers (let’s call them nodeA and nodeB) have RDMA-capable network adapters and are running Ubuntu 22.04.

Step 1: Install RDMA Core Libraries and Drivers: We can install the RDMA libraries, drivers, and debugging utilities using our distribution’s package manager. These packages include drivers and tools to configure, manage, and test RDMA. For Ubuntu, this can be done by running:

sudo apt update
sudo apt install rdma-core ibverbs-utils infiniband-diags

Step 2: Enable and Start the RDMA Management Service: Next, enable and start the RDMA management service, which manages RDMA devices and is essential for RDMA operations:

sudo systemctl enable rdma
sudo systemctl start rdma

With these commands, RDMA services will also start automatically upon system reboot.

Step 3: Configure Memory Limits: This configuration will enable users in the RDMA group to lock as much memory as they want, which is required for stability and performance. For this purpose, you must increase the memory limit of users or groups involved in RDMA transfers by editing /etc/security/limits.d/rdma.conf:

@rdma   soft   memlock   unlimited
@rdma   hard   memlock   unlimited

Step 4: Configure RDMA Network Interfaces: Assign IP addresses to the RDMA-capable network interfaces on both nodes. For example, if the name of your InfiniBand or RoCE interface is ib0, run the following commands:

On nodeA:

sudo ip addr add 192.168.100.1/24 dev ib0
sudo ip link set ib0 up

On nodeB:

sudo ip addr add 192.168.100.2/24 dev ib0
sudo ip link set ib0 up

This step prepares the interface for RDMA traffic.

Step 5: Test and Validate RDMA Connectivity: Once the setup is complete, you must ensure RDMA is working as expected.

Check for available RDMA devices:

ibv_devinfo

This command displays information about RDMA devices present on the system.

Test basic connectivity with ping:

ping 192.168.100.2 # From nodeA to nodeB

This ensures there is basic network connectivity between the nodes.

Test RDMA communication with rping:

On nodeB (as server):

rping -s -a 192.168.100.2

On nodeA (as client):

rping -c -a 192.168.100.2

Successful completion confirms the RDMA path is operational and performing as expected.

Windows Server RDMA Support

Microsoft Windows Server 2012 and subsequent versions incorporate RDMA support. Windows uses the NetworkDirect interface for native RDMA support. SMB Direct is a feature of SMB 3.0 that uses RDMA for faster file transfers.
Features:

• Automatic detection of RDMA adapters
• Multiple RDMA connections per session for fault tolerance
• Integrates smoothly with your existing SMB infrastructure.

RDMA Integration in VMware

RDMA support was introduced in VMware vSphere 7.0 and above for virtualization with RoCE v2 (RDMA over Converged Ethernet v2) adapters and NVMe over RDMA storage solutions. This can allow you to use RDMA inside virtualized environments to:

• Connect to NVMe storage devices over RDMA fabrics
• Reduce latency for storage operations
• Increase virtualized workload performance

RDMA technology has improved performance, efficiency, and scalability in various fields and use cases. Here are the most common use cases where RDMA can provide tangible value:

Below is a table of the most common RDMA mistakes and misconceptions that occur during deployment, with a brief explanation for each one.

Knowledge of these mistakes can help you get the most out of your RDMA architecture. This way, you can avoid significant troubleshooting down the road.

What is RDMA used for?
RDMA technology is used in high-performance computing, cloud storage, AI/ML clusters, virtualization, and financial systems for ultra-low latency and high-performance data transfers.

How is RDMA different from TCP/IP?
Unlike traditional network protocols like TCP/IP, RDMA bypasses the standard network stack. Instead, it allows direct memory access, which speeds up data transfers between applications and network hardware.

Does RDMA require special hardware?
Yes, to use RDMA, you’ll need network interface cards (NICs) that support RDMA. For some protocols, you might also need switches that support lossless data transmission to maximize performance.

What OS supports RDMA?
RDMA support is available across multiple platforms, including Linux via rdma-core, Windows Server 2012, and later( via NetworkDirect, and various UNIX-based systems. Besides, VMware vSphere offers RDMA functionality for virtualized environments.

Is RDMA only for supercomputers?
No—While RDMA was originally developed for high-performance computing, it’s now widely adopted in cloud services, enterprise data centers, AI and machine learning workloads, and virtualization setups.

Enterprises, cloud providers, and data centers can use RDMA to satisfy the demands of AI, ML, and other data-intensive applications for low latency, high throughput, and CPU-efficient networking.

With RDMA, you can bypass the TCP/IP stack and instead use alternative protocols like InfiniBand or RoCE. This can lead to lower latency, higher throughput, and better CPU efficiency. RDMA can be used with Linux, Windows Server, and VMware platforms with solid hardware and network infrastructure compatibility. .
If your organization is considering a data center upgrade, it is essential to understand RDMA’s architecture, practical considerations, and use cases. You can further improve your understanding by exploring the following articles:

• Managed Caching for Valkey
• Types of Virtual Machines
• Bare Metal Hypervisors

Integrating RDMA with these solutions will ensure a scalable, future-ready network foundation for your most demanding workloads.

• Remote Direct Memory Access (RDMA)
• Implementing RDMA on Linux
• Understanding RDMA Microarchitecture Resources for Performance Isolation
• RDMA and Hardware Support
• A Quick Look at the Differences: RDMA vs TCP/IP
• Configuring RoCEv2: Step-by-Step Setup for Optimal Performance
• Common JSOR problems (Linux only)
• RDMA
• Improve Performance of a File Server with SMB Direct
• What is RoCE?