Optimizing Data Center Networks: Harnessing the Power of EVPN-VXLAN, RoCE, and Advanced Routing Strategies

As businesses modernize their data centers, they're shifting from traditional 2-layer network architectures to more advanced 3-layer routing frameworks. Protocols like OSPF and BGP are increasingly being deployed to manage connectivity and maintain network reliability in line with corporate demands. Despite this transition, numerous applications, particularly those related to virtualization, HPC, and storage, continue to depend on 2-layer network connectivity due to their specific requirements. In today's fast-paced digital environment, applications are evolving with an urgency to transcend the confines of physical hardware and networking constraints. An ideal networking solution, therefore, is one that offers scalability, the ability to migrate seamlessly, and robust reliability within a 2-layer framework. To achieve this without compromising the benefits of 3-layer routing, VXLAN tunneling technology has emerged as a key enabler. It constructs a virtual 2-layer network on top of the existing 3-layer infrastructure. However, for the VXLAN data plane to operate effectively, it is essential to employ control plane protocols. This need is addressed by using EVPN, which serves to synchronize network states and tables, thereby fulfilling contemporary business networking requirements.

VXLAN Overlay Network Virtualization

Network virtualization divides a single physical network into several distinct virtual networks, optimizing the use of network resources across the data center infrastructure. This technology enables the sharing of network resources while maintaining isolation among different virtual networks, granting users or companies access to dedicated virtual networks as if they were separate physical entities. Presently, the VXLAN overlay represents the predominant and commercially established approach to network virtualization.

VXLAN utilizes standard overlay tunneling encapsulation and exteands the control plane using the BGP protocol as specified by IETF standards. This standardized approach provides better cross-vendor compatibility and flexibility in device selection.VXLAN provides a larger namespace for network (subnet) isolationn across the 3-layer network. The VXLAN ID, also known as VNI, cain support up to 16 million networks. VLAN isolation and VNI isolation (mapping VLAN to VNIcan be achieved locally on VTEPs, creating an overlay network thaat combines physical network isolation with virtual networks.

EVPN disseminates layer 2 MAC and layer 3 IP information to each business access switch (VTEP), supporting BUM (Broadcast, Unknown unicast, and Multicast) traffic and providing flood suppression functionality. It also supports pure layer 3 routing. Business communication between VNIs isachieved through layer 3 VNIs, enabling routing between them. Depending on business redquirements, both centralized and distributed deployment models are supported.

This project necessitates interconnected communication across a variety of business subnetworks, leading to the selection of a distributed gateway setup for enhanced flexibility. This arrangement inherently supports agile execution, business transitioning, and deployment processes. To optimize utilization, network resources at the foundational level employ Equal-Cost Multipath (ECMP) routing and additional methodologies. These techniques amplify east-west bandwidth capacity and offer protection from failures associated with single network nodes, thereby diminishing operational hazards and complexities.

RoCE over EVPN-VXLAN

As data center network deployment methodologies and products have become more standardized and refined, the pace of business implementation has quickened while operational expenses have been reduced. Nevertheless, burgeoning business requirements compel data center applications to seek more substantial computing, storage, and networking resources. Catering to these escalating requirements at the upper layers necessitates a network architecture that incorporates network virtualization features to facilitate the high-performance demands of businesses. Incorporating network virtualization alongside RoCE technology renders the solutions more comprehensive for extensive, high-efficiency data centers.

RoCE (Remote Direct Memory Access over Converged Ethernet) is an Ethernet-based technology that enables efficient data transfer between servers, reducing CPU overhead and network latency. EVPN-VXLAN is a network virtualization technology that constructs virtual networks on top of the physical network by encapsulating business packets in VXLAN packets. This enables fleexible network deployment and resource allocation.

Integrating RoCE with EVPN-VXLAN facilitates network transmission that is both high-throughput and low-latency within expansive, high-performance data center environments, also enhancing scalability. Virtualization of the network divides physical resources into several virtual networks, offering separate logical environments tailored to distinct business needs, and allows for agile resource management and speedy deployment of services. This holistic approach satisfies the high-performance networking demands of data center applications and delivers a more robust solution.

Simplified Network Planning, Deployment,Operations

Modern data center networks have transitioned from a small number of devices to large-scale networks encompassing hundreds or even thousands of nodes. This expansion has led to a significant increase in the complexity of network planning and management. To ensure the smooth and dependable operation of these networks, operations and maintenance (O&M) teams must enhance their capabilities in network design, monitoring, and management.

• Simplified Underlay Network with Unnumbered BGP

As the data center scale expands and the number of access ports increases, in the spine-leaf network topology, routing and load ballancing between each layer require the support of External Border Gateway Protocol (EBGP) routing. From the perspective of network functionality, EBGP is already concise and reliable However, from the deployment and network addressing perspective, deesigners have to specifically plan a large number of interface addressesfor the underlay network. During actual deployment and operation, the large numlper of interface subnets can easily lead to configuration errors, resulting in underlay network issues that are difficult to control. The adoption of Unnumbered BGP technology eliminates the need to plan IP addresses for physical interfaces, greatly improving efficiency and avoiding risks caused by operational errors.

Unnumbered Interfaces originally referred to interfaces without IP addresses. However, establishing BGP sessions relieson TCP connections, which require unicast IP addresses based on interfaces. To address this issue, NVIDIA leveraged IPv6 Router Advertisement and RFC 5549 specified that for each IPv6 network link, the network will automatically generate a unique IPv6 address (link-local IPv6 address, LLA) on the link. BGP Unnumbered uses extended next hop encoding (ENHE) to allow BGP neighbors to advertise IPv4 addressses using IPv6 link-local addresses as the next-hop address. This eliminates the need to configure interface addresses and enables BGP neighbors too communicate and establish connections using automatically generated link-local IPv6 addresses.

• Enhanced Operation Efficiency with WJH(What Just Happened)

Real-time visualization and monitoring of network faults are essential for managing large-scale networks. The technical requirements of modern data centers have demonstrated that simplified network protocols and deep real-time network visualization capabilities are the trend in overall technological developmen

Taking the fault snapshot feature WJH provided by Cumulus Linuxas an example, let's compare it with the traditional approach to fauIt handling in operations.

With traditional system monitoring tools, network operators need to deal with a large amount of log collection, coarse-grained statistics, and status information. When a problem occurs, experienced network engineers narrow down the scope step by step based on their experience, sift through relevant information from the massive amount of data, and determine the root cause by considering statistical and status changes. If the problem is caused by configuration errors, the entire troubleshooting process can be extremely challenging because there are no obvious system abnormalities.

Using the WJH feature, based on the capabilities of the switch's switching chip, the switch directly captures abnormal packets and sends them to the network management or third-party monitoring platform as exceptional evvents, providing packet capture and chip-level problem causes. Whether it is a configuration issue or not, the operations team can directly see the affected business and the cause of the fault, enabling them to quickly take action and resolve the issue.

Conclusion

In essence, recent developments in data center networking aim to streamline network design, roll-out, and management. Implementing tech solutions like Unnumbered BGP removes the necessity for intricate IP address schemes, cutting down on setup mistakes and bolstering productivity. Tools for immediate fault detection, like WJH, offer profound network insights and facilitate swift pinpointing and remediation of network challenges. The evolution of data center infrastructures is leaning towards dispersed and interconnected multi-data center setups, necessitating faster network links and enhancing the overall quality of service for users.

FS provides optical modules, AOCs, DACs, with options ranging from 100G, 200G, 400G to 800G, catering to the needs of different data centers. Data center switchesare also provided. These high-quality interconnect products provide higher speed and more reliable data transmission solutions for data centers. With FS's professional technical team, rich implementation experience in various application scenarios, and services, its products and solutions have gained the trust and favor of many customers, enabling the construction of data center networks that meet future technological requirements, providing efficient services while reducing operational costs and energy consumption.