Introduction to RDMA (Remote Direct Memory Access)

What Is RDMA?

To begin with, what is RDMA? RDMA is the abbreviation of remote direct memory access. It is a technology that enables computers in a network to exchange data in the main memory without involving the processor, cache or operating system of either computer. Keep reading and you can find that this post from MiniTool offers you full information about RDMA.

Like local-based direct memory access (DMA), RDMA can free up resources so that it can increase throughput and performance. RDMA also helps increase data transfer rates and low-latency networking, which is particularly useful in massively parallel computer clusters. And it can be implemented for networking and storage applications.

How Does RDMA Work?

Then how does RDMA work? RDMA supports zero-copy networking by enabling the network adapter to transfer data directly from the wire to the application memory or directly from the application memory to the wire without copying data between application memory and the data buffers in the operating system.

Such transfers do not require the CPUs, caches, or context switches to do any work, and transfers occur in parallel with other system operations. This reduces latency in message transmission. What’s better, if both the sending and receiving devices support RDMA, the dialogue between the two will be completed faster than similar non-RDMA network systems.

If your applications require quick and massive parallel high-performance computing (HPC) clusters and data center networks, then you can find that RDMA is very useful. You can also find RDMA useful in analyzing big data, in supercomputing environments that process applications, and in machine learning that requires the absolute lowest latency and highest transfer rates.

However, this strategy has some issues related to the fact that the target node doesn’t receive notification of the requested completion (single-sided communication).

Network Protocols That Support RDMA

• RDMA on Converged Ethernet. RoCE is a network protocol that supports RDMA over Ethernet by defining the performance of RDMA in such an environment.
• Internet wide area RDMA protoco IWARP uses Transmission Control Protocol (TCP) or Stream Control Transmission Protocol (SCTP) to transmit data. It was developed by the Internet Engineering Task Force to allow applications on a server to directly read or write to applications that execute on another server without support from the operating system on either server.
• InfiniBand. For high-speed InfiniBand network connections, RDMA is a standard protocol. This RDMA network protocol is commonly used for inter-system communication and was first popular in high-performance computing environments. Because InfiniBand can quickly connect to large computer clusters, it has entered additional use cases, such as big data environments, databases, highly virtualized settings, and web applications with large resource requirements.

Products and Vendors That Support RDMA

As of 2018, RDMA has gained wider acceptance due to the implementation of enhanced functions, achieving better performance than ordinary network infrastructure, so which products and manufacturers support RDMA? Here are some products and vendors are listed below:

• Apache Hadoop and Apache Spark big data analysis
• Baidu Paddle (Parallel Distributed Deep LEarning) platform
• Broadcom and Emulex adapters
• Caffe deep learning framework
• Cavium FastLinQ 45000/41000 Series Ethernet NICs
• Ceph object storage platform
• ChainerMN Python-based deep learning open-source framework
• Chelsio Terminator 5 & 6 iWARP adapters
• Dell EMC PowerEdge servers
• FreeBSD operating system
• GlusterFS internetwork filesystem
• Intel Xeon Scalable processors and Platform Controller Hub
• Mellanox ConnectX family of network adapters and InfiniBand switches
• Microsoft Windows Server (2012 and higher) via SMB Direct supports RDMA-capable network adapters, Hyper-V virtual switch, and the Cognitive Toolkit.

You can click this link to get more products and vendors that support RDMA.

RDMA over Fabrics and Future Directions

RDMA on Fabric is a logical evolution of the existing shared storage architecture, benefiting from solid-state and flash memory, which improves performance access to shared data. An RDMA network adopts a protocol such as RoCE, iWARP, or InfiniBand to send data between the memory address space through the interface, which speeds up operations to increase the value of application, server, and storage investments.

The sixth generation of Fibre Channel storage networks – 32 Gigabits per second – and PCI Express support RDMA based on the Fabrics interface.

Bottom Line

This post has given you detailed information about what RDMA (remote direct memory access) is. Besides, you can also know products and vendors that support RDMA as well as the network protocols that support RDMA.