Hyper-Converged Data Center Network

What Is a Hyper-Converged Data Center Network?

A Data Center Network (DCN) links general computing, storage, and high-performance computing (HPC) resources within a data center. It serves as the conduit for all server data transactions. As IT architecture and storage technologies rapidly evolve, the DCN is transforming into a singular Ethernet network, departing from its multi-network origins. This shift is due to traditional Ethernet's inability to fulfill the demanding requirements of storage and HPC services. In response, a new design known as a hyper-converged DCN has emerged, built upon a seamless Ethernet network capable of accommodating general, storage, and HPC services. This advanced network supports comprehensive automation and sophisticated, network-wide intelligent operations and maintenance.

Why Choose Hyper-Converged Data Center Network?

AI Era's Challenge 1: Network Bottlenecks Amid Enhanced Storage and Computing

Enterprise digitalization spawns vast data, now a key asset. Extracting value through AI is vital in the AI era, with AI-driven machine learning and data-based real-time decision-making at the heart of operations. Unlike the cloud era, AI-era enterprise data centers prioritize efficient data processing over rapid service delivery.

To improve the efficiency of processing massive data through AI, revolutionary changes are taking place in the storage and computing fields:

• Storage media is moving from HDDs to SSDs for real-time data access, slashing storage latency by over 99%.

• For more efficient data computation, the industry has shifted to GPUs and specialized AI chips, boosting processing power by over 100-fold.

As storage media and computing power have significantly advanced, network latency has emerged as the main obstacle to enhancing performance in high-performance data center clusters. Network latency now constitutes over 60% of the total end-to-end latency, up from just 10%. Over half the delay in accessing storage or computing resources is due to network transmission.

Improvements in storage and computing technologies now face the challenge of low network efficiency which hampers overall performance gains. True enhancement in application performance relies on decreasing network latency to a level comparable with computing and storage speeds.

AI Era Challenge 2: Inevitable Shift from TCP/IP to RDMA Despite Demanding Network Solutions

As shown in the following figure, the TCP protocol stack incurs latency in the order of tens of microseconds during the reception, dispatch, and internal processing of packets on servers. Consequently, in systems requiring microsecond-level latency, such as AI data computation and SSD distributed storage systems, this delay introduced by the TCP stack emerges as the primary bottleneck. Furthermore, as network sizes grow and bandwidth increases, data transmission consumes progressively more CPU resources.

Remote Direct Memory Access (RDMA) facilitates the direct transfer of data between applications and network interface cards (NICs), curtailing latency within servers to approximately one microsecond. This technology also enables receivers to read data right off the senders' memory, leading to a significant reduction in CPU usage.

Service evaluation data indicates that RDMA boosts computing efficiency by a factor between 6 and 8. With server internal transmission latencies reduced to approximately one microsecond, it's feasible for the latencies within SSD distributed storage systems to drop from the millisecond spectrum to the microsecond domain. With the advent of the newest Non-Volatile Memory express (NVMe) interface standard, RDMA has ascended to the forefront as a predominant stack in network communications. Consequently, the shift from TCP/IP to RDMA is seemingly on track to becoming a standard progression.

Comparison between RDMA and TCP

There are two solutions for carrying RDMA on the interconnection network between servers: dedicated IB network and traditional IP Ethernet network. However, they both have disadvantages.

• InfiniBand (IB) Network: It uses closed architecture and proprietary protocols, making it difficult to interconnect with legacy large-scale IP networks. Complex O&M and dedicated O&M personnel result in high operating expense (OPEX).

• Traditional IP Ethernet Network: Within the context of RDMA, any packet loss exceeding 0.1% triggers a significant drop in the overall network performance, while a packet loss rate hitting 2% completely nullifies RDMA's throughput. Ensuring RDMA's performance isn't compromised entails maintaining packet loss rates below 0.001% or entirely eliminating packet loss. Traditional IP Ethernet setups may discard data packets during network congestion. To circumvent this, the traditional IP Ethernet frameworks employ Priority Flow Control (PFC) and Explicit Congestion Notification (ECN) strategies, which leverage backpressure signals to achieve zero packet loss by diminishing the pace of data transmission, although this does not enhance throughput.

Therefore, efficient RDMA must be carried over an open Ethernet network with zero packet loss and high throughput.

AI Era Challenge 3: The Rise of Distributed Architectures Intensifies Network Congestion, Catalyzing a Network Revolution

During the digital evolution of businesses like those in the finance and internet sectors, numerous application systems transition to distributed architectures. Amidst this shift, a significant quantity of personal computers supplant intermediate computers, offering advantages of reduced expense, enhanced scalability, and greater manageability. Nevertheless, this trend presents obstacles in terms of networking connectivity:

• The shift towards a distributed framework necessitates extensive server-to-server communication.

• In situations with Incast traffic, where multiple sources send data to a single point, there are surges at the receiving end that momentarily overwhelm the receiver's interface capacity. This leads to network congestion and subsequent packet loss.

• As the intricacy of applications within distributed systems escalates, there is a corresponding increase in the size of the data packets being transferred amongst servers. These larger packets exacerbate existing issues with network congestion.

Traffic model of distributed architecture

What Are the Core Indicators of a Hyper-Converged Data Center Network?

The primary performance metrics for the forthcoming generation of Data Center Networks (DCN) in the AI-driven era include the elimination of packet loss, minimal latency, and maximized throughput. These metrics are essential to facilitate the data handling demands of complex, distributed structures. Attaining the apex in all three aspects is challenging since they are interdependent, posing significant hurdles to achieving an ideal level of overall performance.

Three core indicators affecting each other

The fundamental technology integral to realizing no packet loss, reduced latency, and enhanced throughput lies in the refinement of congestion control algorithms. The widely implemented congestion management algorithm for lossless networking within data centers is Data Center Quantized Congestion Notification (DCQCN). This algorithm necessitates a coordinated effort between Network Interface Cards (NICs) and the network infrastructure. Configuring it requires setting myriad parameters—dozens for each node, culminating in several hundred thousand across the network. To streamline this process, standard configurations are employed. Consequently, by utilizing these standardized settings, it is possible to meet the targeted performance criteria across various traffic scenarios.

What Are the Differences Between a Hyper-Converged Data Center Network and the HCI?

Hyper-converged infrastructure (HCI) consists of modular units that bundle processing, networking, and storage capabilities, and are compatible with server virtualization technology. These units can be interconnected via a network to facilitate a smooth scaling-out process, creating a cohesive pool of resources.

HCI amalgamates virtualized compute and storage resources within a singular system infrastructure. At its core, the virtualization layer, facilitated by the Hypervisor, operates atop physical servers, while the distributed storage service, which is utilized by virtual machines (VMs), functions within the virtualization layer. This storage service can be deployed either as a virtual machine itself or as a component that is fused with the virtualization software. HCI not only merges computing and storage resources but also connects networks and assimilates additional platforms and services. It is broadly accepted within the industry that the combination of a software-defined storage layer, distributed in nature, along with virtualized computing capacities, form the essential foundational elements of HCI architecture.

Compared to HCI, a hyper-converged Data Center Network (DCN) is dedicated solely to the networking aspect, offering an innovative solution for the networking layer that links computing and storage resources. Within a hyper-converged DCN, there is no necessity for the reconfiguration or consolidation of computing and storage elements. Furthermore, this type of DCN permits rapid, cost-efficient expansion utilizing Ethernet technology.