RDMA NIC: Features and How to Choose?

As a high-performance and low-latency data transmission method, RDMA((Remote Direct Memory Access) technology is increasingly gaining attention and application in the global Ethernet communication market. Initially applied in the high-performance computing field, it has gradually become prevalent in data centers. To address server data processing latency, RDMA NIC was introduced. This article will explore the key features of RDMA NIC, the distinctions between RDMA NICs and regular network cards without NIC, and a guide on how to make informed purchases.

Key Features of RDMA NIC

By embedding the RDMA protocol directly into NIC, the RDMA NIC enhances network performance and boosts application efficiency effectively. When compared to network interface cards without RDMA, RDMA NICs exhibit distinctive features in the following aspects:

Data Path: Simplified Pathways

In comparison to traditional network cards without RDMA, RDMA NICs exhibit more direct and efficient data movement. Figure 1 illustrates the traditional network card's transmission path vs RDMA NIC transmission path. Traditional network card follows the data communication as Application > System > Memory > CPU > Memory > Hard Disk > Memory > Network Card. In contrast, RDMA NICs streamline the process by bypassing the kernel networking stack in both computers: Application > Memory > Hard Disk > Memory > Network Card.

Figure 1: Regular NIC Data Communication vs RDMA NIC Data Communication

Transmission Performance: Enhanced Performance

RDMA NICs outperform non-RDMA NICs in terms of data transmission performance. As mentioned above, the transmission path of RDMA NICs is simplified than NICs without RDMA, which significantly reduces latency and CPU load during data transfer, enhancing efficiency and performance. Unlike regular network cards that rely on the operating system kernel, RDMA NICs can operate more efficiently by circumventing the kernel.

Software and Hardware Support: Specialized Requirements

The application of RDMA NICs necessitates support from both hardware and software, including RDMA-enabled hardware on the NIC, driver support, and operating system compatibility. NICs that don't support RDMA, on the other hand, do not require specialized hardware and software support and can be used directly in most operating systems.

Application Scenarios: High-Concurrency and High-Performance Application

In traditional network transmission paths, data goes through various steps involving device drivers, ultimately reaching the network structure. This process requires CPU cycles for data handling and transfer, posing a challenge in high-bandwidth networks because it takes a lot of time for CPU usage.

While RDMA NICs significantly improve network transfer efficiency, benefiting IO-intensive operations with lower latency and higher throughput. Common applications include distributed storage and distributed databases.

How to Select RDMA NIC?

After understanding the prominent features of RDMA NICs, consider the following points when making a purchase:

Compatibility and Interoperability

Ensuring good compatibility and interoperability is crucial for seamless collaboration in complex data center architectures, especially considering the hardware and software support required for RDMA NICs.

Application Needs Analysis

Analyze the requirements of different domains, such as data centers, storage systems, or high-performance computing, to determine the most suitable type of RDMA NIC.

Balancing Performance Needs and Budget

Strike a balance between high-performance hardware and cost-effectiveness, ensuring that the chosen RDMA NIC meets performance needs while aligning with budget requirements.

Selecting RDMA Technology

Based on the above considerations, last but not least, you should consider the specific RDMA technology. Currently, there are three main protocols - InfiniBand, RoCE, and iWARP. InfiniBand is a network protocol tailored for RDMA, RoCE runs RDMA over Ethernet, and iWARP allows RDMA over TCP.

• InfiniBand

  Designed specifically for RDMA, Infiniband technology ensures the reliability of data transmission at the hardware level. However, it requires the entire ecosystem to be equipped with specialized hardware. Among them, Mellanox (currently acquired by NVIDIA) is the leading representative in RDMA network card performance. As an Elite partner of NVIDIA, FS offers a variety of NVIDIA original network cards supporting RDMA for InfiniBand. The following InfiniBand NICs are available for purchase.

• iWARP

iWARP is a network protocol that enables RDMA over TCP, and its advantage lies in the ability to operate on today's standard TCP/IP networks. It only requires the purchase of iWARP-compatible network cards to implement RDMA. From this perspective, iWARP-compatible network cards offer higher ecosystem compatibility and cost-effectiveness compared to RoCE NICs. This makes them particularly suitable for small enterprises with slightly lower financial budgets. Intel iWARP NICs are typical iWARP-compatible network cards in the market.

FS provides a variety of Intel original network cards and cards developed based on Intel original chips that support iWARP. This includes the Intel E810 series, Intel X710 series, Intel XL710 series, Intel XXV710 series, Intel 82599ES series, and more. For specific options, you can visit the following link for purchase: FS Intel-Based Ethernet Adapters.

• RoCE (RDMA over Converged Ethernet)

RoCE (RDMA over Converged Ethernet) is a network protocol that allows RDMA to be executed over Ethernet. It was introduced by Mellanox as a low-cost alternative to meet market demands for InfiniBand networks. RoCE networks require lossless Ethernet to achieve low-latency operations, meaning the Ethernet switches integrated into the network must support Data Center Bridging (DCB) and Priority Flow Control (PFC) mechanisms to maintain lossless traffic. However, configuring a lossless network in modern enterprise environments is a complex process, and scalability can be significantly constrained.

FS can provide NVIDIA Ethernet network cards that support RoCE, allowing businesses to leverage RDMA capabilities over Ethernet networks.

Conclusion

In summary, RDMA NICs, with outstanding performance and low-latency features, offer significant advantages across various applications such as data centers, storage systems, and high-performance computing. Choosing the right RDMA NIC is a crucial step in ensuring the enhancement of your network performance and the specific selection process requires consideration of multiple factors to make an informed decision. If you have any uncertainties in your purchasing decision, this article may provide helpful guidance.