English

Explore the InfiniBand and InfiniBand Switch

Posted on Mar 22, 2024 by
348

InfiniBand and InfiniBand switches are essential components of high-performance computing interconnect technology. This article will introduce the concept of InfiniBand and discuss the functions and uses of InfiniBand switches. Additionally, we will explore how to bridge InfiniBand with Ethernet.

What Is InfiniBand?

In 1999, as CPU performance rapidly advanced, existing I/O systems became a bottleneck, limiting server performance. The telecommunications industry urgently needed a powerful next-generation I/O standard and technology to cater to high-speed communication networks, leading to InfiniBand's emergence. InfiniBand Architecture (IBA) gained significant popularity in clustered supercomputers around 2005. Since then, it has rapidly expanded its presence in the telecommunications industry.

InfiniBand is a leading interconnect technology in high-performance computing (HPC), enterprise data centers, and cloud computing environments. Renowned for its infinite bandwidth, InfiniBand sets the standard for high-performance networking communication in the HPC domain. It boasts exceptional characteristics such as high throughput, low latency, and remarkable scalability. Leveraging this cutting-edge technology, InfiniBand facilitates seamless communication between servers, server-storage connections, and server-LAN/WAN/Internet interfaces. By employing InfiniBand architecture, multiple links are intelligently interconnected to enable non-blocking bandwidth, ensuring efficient data transfer between processors and I/O devices.

InfiniBand topology HPC cluster

What Is an InfiniBand Switch?

An InfiniBand switch (IB switch) is a device that adds InfiniBand capabilities to network switch hardware, similar to PoE switches, SDN switches, and NVGRE/VXLAN switches. Leading IB switch vendors in the market include Mellanox, Intel, and Oracle. The price of InfiniBand switches varies depending on vendors and switch configurations.

IB switch ports come in different numbers, connector types, and IB types. For example, Mellanox InfiniBand switches have QSFP56 /OSFP HDR/NDR port counts ranging from 8 to 64. HDR and NDR InfiniBand switches, with transmission rates of 200Gb/s and 400Gb/s respectively, incorporate state-of-the-art technologies such as Remote Direct Memory Access (RDMA), adaptive routing, and NVIDIA Scalable Hierarchical Aggregation and Reduction Protocol (SHARP)™.

InfiniBand Architecture

Functionality of InfiniBand Switches

InfiniBand switches are the linchpin of network connectivity, facilitating seamless communication among devices. They play a vital role in packet forwarding and routing, as well as redundancy.

  • Data Packet Forwarding and Routing: InfiniBand switches excel at efficiently transferring data packets within the network. Sophisticated routing algorithms and protocols ensure optimal data flow and minimal latency.

  • Multi-Path Redundancy and Load Balancing: With support for multiple paths, InfiniBand networks achieve redundancy and load balancing, enhancing fault tolerance and resource utilization.

  • Scalability and Flexibility: InfiniBand switches provide scalability to accommodate large-scale systems, ensuring seamless expansion. Their flexibility enables various network configurations to meet diverse requirements.

Application of InfiniBand Switches

InfiniBand switches find extensive applications in high-performance computing, data center interconnect, storage networking, and virtualization environments. Their high bandwidth and low-latency interconnectivity capabilities enable efficient data transfer and communication, meeting the demands of modern large-scale computing and data processing. FS can provide NVIDIA InfiniBand 200G and 400G switches.

  • High-Performance Computing (HPC): InfiniBand switches are extensively applied in the field of HPC. They provide low-latency, high-bandwidth interconnectivity, enabling efficient data exchange and communication within large-scale parallel computing clusters.

  • Data Center Interconnect: InfiniBand switches offer high bandwidth and low latency interconnectivity, meeting the need for high-speed communication among servers, storage devices, and networks within data centers. 

  • Storage Networking: In the realm of storage networking, InfiniBand switches provide high-speed, low-latency data transfer channels, catering to applications with high storage bandwidth and response time requirements. They are widely used in high-performance storage systems, flash arrays, and distributed file systems, delivering fast data access and transmission capabilities.

  • Virtualization Environments: In virtualization environments, InfiniBand switches provide high-performance interconnectivity for virtual machines, facilitating fast data transfer between virtual machines and physical devices.

How to Bridge InfiniBand to Ethernet?

To bridge InfiniBand with Ethernet, many modern InfiniBand switches feature built-in Ethernet ports and Ethernet gateways to enhance network adaptability. However, in cases where only InfiniBand ports are available on the InfiniBand switch, connecting a Layer 2 InfiniBand host to multiple Gigabit Ethernet switches presents a challenge. One approach is to use InfiniBand network cards/Ethernet Converged Network Adapters (CNAs) or similar network adapter devices to achieve InfiniBand-to-Ethernet bridging.

Another method involves purchasing Mellanox InfiniBand switches and ConnectX series network cards, which supporting Virtual Protocol Interconnect (VPI) between InfiniBand and Ethernet. This allows for explicit or automatic adaptation of link protocols, and a physical Mellanox InfiniBand switch can serve as a gateway, connecting InfiniBand devices to Ethernet networks.

Ethernet gateway Bridge-group bridges InfiniBand to Etherinet

Conclusion

InfiniBand and its switches have revolutionized high-speed networking, empowering various industries with their exceptional performance and versatility. As technology continues to advance, InfiniBand will shape the future of connectivity, enabling groundbreaking innovations across diverse sectors.

You might be interested in

Knowledge
Knowledge
Knowledge
See profile for Sheldon.
Sheldon
Decoding OLT, ONU, ONT, and ODN in PON Network
Mar 14, 2023
399.7k
Knowledge
See profile for Irving.
Irving
What's the Difference? Hub vs Switch vs Router
Dec 17, 2021
372.3k
Knowledge
See profile for Sheldon.
Sheldon
What Is SFP Port of Gigabit Switch?
Jan 6, 2023
346.6k
Knowledge
See profile for Migelle.
Migelle
PoE vs PoE+ vs PoE++ Switch: How to Choose?
May 30, 2024
429.2k
Knowledge
Knowledge
Knowledge
Knowledge
See profile for Moris.
Moris
How Much Do You Know About Power Cord Types?
Sep 29, 2021
305.8k