Explore the InfiniBand and InfiniBand Switch

InfiniBand and InfiniBand switches are essential components of high-performance computing interconnect technology. This article will introduce the concept of InfiniBand and discuss the functions and uses of InfiniBand switches. Additionally, we will explore how to bridge InfiniBand with Ethernet.

In 1999, as CPU performance rapidly advanced, existing I/O systems became a bottleneck, limiting server performance. The telecommunications industry urgently needed a powerful next-generation I/O standard and technology to cater to high-speed communication networks, leading to InfiniBand's emergence. InfiniBand Architecture (IBA) gained significant popularity in clustered supercomputers around 2005. Since then, it has rapidly expanded its presence in the telecommunications industry.

InfiniBand is a leading interconnect technology in high-performance computing (HPC), enterprise data centers, and cloud computing environments. Renowned for its infinite bandwidth, InfiniBand sets the standard for high-performance networking communication in the HPC domain. It boasts exceptional characteristics such as high throughput, low latency, and remarkable scalability. Leveraging this cutting-edge technology, InfiniBand facilitates seamless communication between servers, server-storage connections, and server-LAN/WAN/Internet interfaces. By employing InfiniBand architecture, multiple links are intelligently interconnected to enable non-blocking bandwidth, ensuring efficient data transfer between processors and I/O devices.

An InfiniBand switch (IB switch) is a device that adds InfiniBand capabilities to network switch hardware, similar to PoE switches, SDN switches, and NVGRE/VXLAN switches. Leading IB switch vendors in the market include Mellanox, Intel, and Oracle. The price of InfiniBand switches varies depending on vendors and switch configurations.

IB switch ports come in different numbers, connector types, and IB types. For example, Mellanox InfiniBand switches have QSFP56 /OSFP HDR/NDR port counts ranging from 8 to 64. HDR and NDR InfiniBand switches, with transmission rates of 200Gb/s and 400Gb/s respectively, incorporate state-of-the-art technologies such as Remote Direct Memory Access (RDMA), adaptive routing, and NVIDIA Scalable Hierarchical Aggregation and Reduction Protocol (SHARP)™.

InfiniBand switches are the linchpin of network connectivity, facilitating seamless communication among devices. They play a vital role in packet forwarding and routing, as well as redundancy.

InfiniBand switches find extensive applications in high-performance computing, data center interconnect, storage networking, and virtualization environments. Their high bandwidth and low-latency interconnectivity capabilities enable efficient data transfer and communication, meeting the demands of modern large-scale computing and data processing. FS can provide NVIDIA InfiniBand 200G and 400G switches.

• High-Performance Computing (HPC): InfiniBand switches are extensively applied in the field of HPC. They provide low-latency, high-bandwidth interconnectivity, enabling efficient data exchange and communication within large-scale parallel computing clusters.

• Data Center Interconnect: InfiniBand switches offer high bandwidth and low latency interconnectivity, meeting the need for high-speed communication among servers, storage devices, and networks within data centers. 

• Storage Networking: In the realm of storage networking, InfiniBand switches provide high-speed, low-latency data transfer channels, catering to applications with high storage bandwidth and response time requirements. They are widely used in high-performance storage systems, flash arrays, and distributed file systems, delivering fast data access and transmission capabilities.

• Virtualization Environments: In virtualization environments, InfiniBand switches provide high-performance interconnectivity for virtual machines, facilitating fast data transfer between virtual machines and physical devices.

How to Bridge InfiniBand to Ethernet?

To bridge InfiniBand with Ethernet, many modern InfiniBand switches feature built-in Ethernet ports and Ethernet gateways to enhance network adaptability. However, in cases where only InfiniBand ports are available on the InfiniBand switch, connecting a Layer 2 InfiniBand host to multiple Gigabit Ethernet switches presents a challenge. One approach is to use InfiniBand network cards/Ethernet Converged Network Adapters (CNAs) or similar network adapter devices to achieve InfiniBand-to-Ethernet bridging.

Another method involves purchasing Mellanox InfiniBand switches and ConnectX series network cards, which supporting Virtual Protocol Interconnect (VPI) between InfiniBand and Ethernet. This allows for explicit or automatic adaptation of link protocols, and a physical Mellanox InfiniBand switch can serve as a gateway, connecting InfiniBand devices to Ethernet networks.

InfiniBand and its switches have revolutionized high-speed networking, empowering various industries with their exceptional performance and versatility. As technology continues to advance, InfiniBand will shape the future of connectivity, enabling groundbreaking innovations across diverse sectors.