English

100G InfiniBand EDR Solution: Empowering High-Performance Computing

Posted on Dec 21, 2023 by
593

In the realm of complex workloads, where ultra-high-speed processing of high-resolution simulations, extensive datasets, and highly parallelized algorithms is paramount, the demand for cutting-edge solutions continues to escalate. In this context, InfiniBand emerges as a pivotal technology, providing significant advancements in performance for high-performance computing (HPC), artificial intelligence, and hyperscale cloud infrastructures. This article delves into the FS 100G EDR InfiniBand solution, specifically focusing on the deployment of 100G QSFP28 EDR transceivers and cables over InfiniBand protocol links to enhance network latency and enable higher data rates.

100G EDR solution

Introduction to InfiniBand HDR 100Gb/s Cables and Transceivers

InfiniBand EDR 100Gb/s Active AOC Cables

The NVIDIA InfiniBand MFA1A00-E001, a Class 1 FDA Laser based active optical cable, is tailored for deployment in InfiniBand 100Gb/s EDR systems. Available in various lengths ranging from 1m to 100m, these active fiber cables are favored in high-speed InfiniBand HPC environments due to their predictable latency, Max. power consumption 3.5W, and airflow enhancement. Featuring hot-swappability, the NVIDIA InfiniBand MFA1A00-E001 facilitates easy installation and replacement. Each end of the cable incorporates four multimode fiber (MMF) transceivers with a data transmission rate of up to 26 Gb/s, surpassing the capabilities of Direct Attach Copper (DAC) cables in terms of airflow, bend radius, and transmission distances.

100G EDR AOC

InfiniBand EDR 100Gb/s Passive Copper Cables

The NVIDIA InfiniBand MCP1600-E001E30 offers a cost-effective alternative for InfiniBand 100Gb/s EDR applications, with lengths available in 0.5m, 1m, 1.5m, 2m and 3m. This passive copper cable boasts four high-speed copper pairs, each supporting data rates of up to 25Gb/s. Featuring an EEPROM on each QSFP28 port for host system communication, this passive DAC solution ensures efficient connectivity for short-haul interconnections. It not only enables higher port bandwidth, density, and configurability, but also reduces power demand in the data center.

100G EDR DAC

InfiniBand EDR 100Gb/s Optical Modules

The 100Gb EDR optical modules, packaged in QSFP28 form factor with LC duplex or MTP/MPO-12 connectors, are hot-swappable and suitable for both EDR InfiniBand and 100G Ethernet. These modules employ four high-speed differential signal channels, each operating at 25 Gbps. They can be categorized into QSFP28 SR4, QSEP28 PSM4, QSFP28 CWDM4, and QSFP28 LR4 based on transmission distance requirements.

100G EDR transceiver

QSFP28 Model Interface Type Rate Fiber Type Wavelength Transmission Distance Laser Type DDM Protocol
QSFP28 SR4 MPO/MTP-12 EDR MMF 850nm 70/100m VCSEL Support InfiniBand EDR
QSFP28 PSM4 MPO/MTP-12 EDR MMF 1310nm 2km DFB Support InfiniBand EDR
QSFP28 CWDM4 LC Duplex EDR SMF 1271nm, 1291nm, 1311nm, 1331nm 2km DFB Support InfiniBand EDR
QSFP28 LR4 LC Duplex EDR SMF 1310nm 10km EML Support InfiniBand EDR

100Gb InfiniBand EDR System Scenario Applications

InfiniBand has gained widespread adoption in data centers, artificial intelligence, and other domains, primarily employing the spine-leaf architecture. In data centers, transceivers and cables play a pivotal role in two key scenarios:

  • Data Center to User: Involves end-user interactions such as web browsing, email exchange, and video streaming, connecting switches with servers or storage devices.

  • Data Center Interconnects: Facilitates data replication, software and system upgrades, linking switches together.

Switch to switch interconnects

Spine and Leaf configurations of HDR100 InfiniBand built at 100 Gb/s, also connect Core Switch and Spine Switch at 100G EDR rate.

100G EDR interconnects

Product Category Distance Application Area
QSFP-100G-PCxxx DAC 0.5m to 3m Spine switch to Leaf switch
QSFP-100G-AOxxx AOC 1m to 100m Spine switch to Leaf switch
QSFP-SR4-100G Optical Transceiver ≤100m Spine switch to Leaf switch
QSFP-PSM4-100G Optical Transceiver ≤500m Spine to Core switch
QSFP-CWDM4-100G Optical Transceiver ≤2km Spine to Core switch
QSFP-LR4-100G Optical Transceiver ≤10km Spine to Core switch

Switches to Servers or GPU Computing Systems

Direct Attach Copper (DAC) is widely adopted in data centers due to its lack of electronics and minimal latency. Conversely, AOC active optical cables offer an integrated solution for short-distance multi-channel data communication, catering to communication distances from 1m to 100m. Ideal for supercomputers with InfiniBand standards like SDR, DDR, and QDR, AOC cables provide superior airflow heat dissipation and a smaller bend radius compared to copper cables. Additionally, they offer extended transmission distances and improved Bit Error Rate (BER) performance. However, for longer transmission distances, consideration of optical modules becomes necessary.

switch to server or GPU

Product Category Distance Application Area
Q28-PC0xE DAC 0.5m to 3m Leaf to NIC/Storage
QSFP-100G-AOxxx AOC 1m to 100m Leaf to NIC/Storage
QSFP28-SR4-100G Optical Transceiver ≤100m Leaf to NIC/Storage

Summary

As the landscape of 100G InfiniBand EDR evolves, FS 100G EDR InfiniBand solution emerges as a mature and robust option. With its high bandwidth, low latency, and reduced power consumption, this solution facilitates higher port bandwidth, density, and configurability at a lower cost. Tailored for large-scale data centers, high-performance computing, artificial intelligence, and future network expansion, customers can select products based on application requirements, transmission distance, and network deployment. FS 100G EDR InfiniBand solution stands as an optimal choice, aligning with the ever-growing demands of modern computational workloads.

You might be interested in

Knowledge
Knowledge
Knowledge
See profile for Sheldon.
Sheldon
Decoding OLT, ONU, ONT, and ODN in PON Network
Mar 14, 2023
385.1k
Knowledge
See profile for Irving.
Irving
What's the Difference? Hub vs Switch vs Router
Dec 17, 2021
367.1k
Knowledge
See profile for Sheldon.
Sheldon
What Is SFP Port of Gigabit Switch?
Jan 6, 2023
334.6k
Knowledge
See profile for Migelle.
Migelle
PoE vs PoE+ vs PoE++ Switch: How to Choose?
Mar 16, 2023
419.9k
Knowledge
Knowledge
Knowledge
Knowledge
See profile for Moris.
Moris
How Much Do You Know About Power Cord Types?
Sep 29, 2021
293.7k