English

Introducing 200G HDR InfiniBand Solution

Posted on Jan 11, 2024 by
586

The Emergence of 200G HDR InfiniBand

As the demand for data analysis continues to grow, there is an increasing requirement for higher data throughput to facilitate detailed analysis. While applications like automotive construction or weather simulations previously relied on 100Gb/s bandwidth just a few years ago, today's high-performance machine learning, storage, and hyperscale technologies necessitate even faster networks. A bandwidth of 100Gb/s is no longer sufficient for many advanced data centers. Whether it's for brain mapping or homeland security, the most demanding supercomputers and data center applications strive to achieve exceptional accomplishments within remarkably short turnaround times.

Therefore, there is a race underway to develop 200Gb/s technologies capable of meeting the requirements of today's most advanced networks.

HDR demand

FS Introduces 200G HDR InfiniBand Solution

The fast data throughput, ultra-low latency, and easy deployment advantages of 200G HDR InfiniBand technology make it the optimal interconnect solution for compute-intensive HPC and AI workloads. Recently, FS introduced a 200G HDR InfiniBand solution that was deployed in the field of artificial intelligence. This article will explain the solution in detail.

How Does the FS 200G HDR InfiniBand Solution Work?

The 200G HDR InfiniBand Network Solution is designed to cater to the demanding data transfer and communication needs of modern HPC and AI workloads. The design and expansion of this AI network solution are based on the POD model. Generally, 32 servers are in a POD that form a cluster. Due to the 1:1 convergence ratio requirement in the IB network design, the network architecture ensures non-blocking forwarding and lossless traffic in both uplink and downlink directions. As business grows and network demands increase, network expansion becomes inevitable, and every network expansion costs a lot of money, which highlights the importance and economic impact of the AI network solution.

FS 200G HDR InfiniBand Solution Architecture

The 200G HDR InfiniBand Network Solution leverages robust architecture and a range of high-quality products to enable efficient and secure networking in data centers.

The solution utilizes a scalable spine-leaf architecture, where the spine layer uses 200G HDR InfiniBand switch MQM8700-HS2F to provide high bandwidth and low latency connectivity. The leaf layer also adopts InfiniBand switch MOM8700-HS2F and GPU servers to achieve highly compute-intensive task processing. Besides, the storage layer uses open networking switch N8550-48B8C pre-loaded with PicOS® and storage server to form an efficient storage network.

This architecture ensures optimal performance and flexibility for AI Networking. Here is the topology of this solution.

AI topology

What Are the Advantages of FS 200G HDR InfiniBand Solution?

The 200G HDR InfiniBand Network Solution offers several advantages for high-performance computing (HPC) environments and AI applications. This type of networking technology is designed to support the large data volumes and high-speed requirements of advanced computing systems. Here are some of the key benefits.

  • Low Latency: The HDR InfiniBand solution delivers low-latency communication, minimizing the delay in data transfers between nodes.

  • High Bandwidth: InfiniBand offers much higher bandwidth than traditional Ethernet and is suitable for extremely data-intensive applications. It is mainly reflected in the two aspects, one is that InfiniBand is a lightweight protocol. Compared with Ethernet, it generates less overhead in data transmission. This means that in practical applications, InfiniBand can use its bandwidth more efficiently. Another is that the InfiniBand design includes more efficient flow control and congestion management mechanisms. These mechanisms ensure the efficiency of data transmission, reduce possible network congestion and packet loss, and thus provide higher effective bandwidth in practical use.

  • High Scalability: HDR InfiniBand AI networks are designed with scalability in mind, allowing them to support a large number of nodes in a cluster. This is essential for growing AI applications and data centers, as it ensures that the network can accommodate future enhancements without significant overhauls or bottlenecks.

  • Enhanced Reliability: InfiniBand provides built-in mechanisms for error handling and recovery, ensuring high reliability and consistent performance. This is essential for mission-critical applications that cannot tolerate disruptions.

FS 200G HDR Infiniband Solution Product List

Networking Layer Product Type Model Description
Spine and Leaf Layer 200G IB Switches MQM8700-HS2F NVIDIA® 40-Port 200G InfiniBand Data Center Switch, 40 x HDR QSFP56, x86 Dual Core, NVIDIA Quantum™ Chip, P2C Airflow
200G IB Adapters MCX653105A-HDAT-SP NVIDIA Mellanox MCX653105A-HDAT ConnectX®-6 InfiniBand/VPI Adapter Card 200GbE/HDR, Single-Port QSFP56, PCIe 4.0 x 16, Tall&Short Bracket
200G IB Modules QSFP-SR4-200G NVIDIA InfiniBand MMA1T00-HS Compatible QSFP56 200G SR4 850nm 100m DOM MPO-12/UPC HDR MMF Optical Transceiver
MTP®/MPO Fiber Cables 12FMTPOM4 MTP®-12 (Female) to MTP®-12 (Female) OM4 Multimode Elite Trunk Cable, 12 Fibers, Type B, Plenum (OFNP), Magenta
GPU Server SYS-821GE-TNHR GPU SuperServer SYS-821GE-TNHR (Complete System Only)
Storage Layer 25G Ethernet Switches N8550-48B8C 48-Port Ethernet L3 Data Center Open Networking Switch, 48 x 25Gb SFP28, 2 x 10Gb SFP+, with 8 x 100Gb QSFP28 Uplinks, Broadcom Chip, PicOS® Installed, with Perpetual License and 1-year Service Bundle
25G Ethernet Modules SFP28-25GSR-85 Cisco SFP-25G-SR-S Compatible 25GBASE-SR SFP28 850nm 100m Duplex LC MMF DOM Optical Transceiver Module
Fiber Patch Cables OM4LCDX LC UPC to LC UPC Duplex OM4 Multimode PVC (OFNR) 2.0mm Fiber Optic Patch Cable
Storage Server SSG-640SP-DE1CR90 Storage SuperServer SSG-640SP-DE1CR90 (Complete System Only)

Summary

It's important to weigh these disadvantages against the potential benefits when considering upgrading to a 200G HDR InfiniBand network. For many organizations, especially those whose operations rely on the utmost computing and network performance, the advantages will outweigh the challenges. However, for others, it may be more cost-effective and practical to consider lower-speed InfiniBand networks or alternative technologies that better suit their needs.

You might be interested in

Knowledge
Knowledge
Knowledge
See profile for Sheldon.
Sheldon
Decoding OLT, ONU, ONT, and ODN in PON Network
Mar 14, 2023
386.2k
Knowledge
See profile for Irving.
Irving
What's the Difference? Hub vs Switch vs Router
Dec 17, 2021
367.6k
Knowledge
See profile for Sheldon.
Sheldon
What Is SFP Port of Gigabit Switch?
Jan 6, 2023
335.5k
Knowledge
See profile for Migelle.
Migelle
PoE vs PoE+ vs PoE++ Switch: How to Choose?
Mar 16, 2023
420.5k
Knowledge
Knowledge
Knowledge
Knowledge
See profile for Moris.
Moris
How Much Do You Know About Power Cord Types?
Sep 29, 2021
294.7k