iNOF

What Is iNOF?

Intelligent Lossless NVMe Over Fabrics (iNOF) technology enables efficient host management and leverages intelligent lossless network technology to integrate computing and storage networks within storage systems. This article delves into the necessity of iNOF, the functioning of the iNOF network, the relationship between BFD and iNOF, and a common use case for iNOF.

Why Do We Need iNOF?

In a conventional network architecture, the deployment of three separate networks (Ethernet, dedicated computing network, and dedicated storage network) leads to significant costs in terms of deployment and maintenance. Furthermore, the performance requirements of storage systems are not adequately met by the traditional TCP/IP-based Ethernet. These challenges can be effectively addressed through the implementation of intelligent lossless network technology.

Intelligent lossless network technology leverages the RDMA over Converged Ethernet version 2 (RoCEv2) protocol and employs an Intelligent Lossless (iLossless) algorithm that combines Priority-based Flow Control (PFC) and Artificial Intelligence Explicit Congestion Notification (AI ECN) technologies. This integration enables the transmission of traffic on Ethernet with lossless properties, low latency, and high throughput, thus meeting the specific demands of storage systems and facilitating the convergence of computing and storage networks. Given that storage systems commonly manage substantial amounts of data, a large number of hosts need to be efficiently handled, with an increasing number of new hosts continuously connecting to network devices within the system.

To enhance the capabilities of intelligent lossless network technology and better cater to the needs of storage systems, the introduction of iNOF technology is paramount. iNOF empowers network devices to effectively manage the connection and disconnection of hosts, promptly identify network faults, and ensure seamless operation with zero packet loss, minimal delay, and maximal throughput on the intelligent lossless network.

How Does the iNOF Network Work?

The following figure shows how iNOF is implemented.

Components of the iNOF network:

• 1. Host: A network server or disk device with iNOF capabilities.

• 2. iNOF reflector: Responsible for managing connected hosts. Each iNOF reflector establishes a backup connection with another iNOF reflector to ensure redundancy (optional in simpler network setups).

• 3. iNOF client: Manages connected hosts and establishes iNOF connections with each iNOF reflector. iNOF clients do not need to establish connections with other iNOF clients and only require direct connectivity to hosts.

• 4.iNOF zone: Hosts are organized and managed based on iNOF zones. The system includes a default zone, but users can create custom zones based on specific service requirements. When a host joins or leaves a zone, iNOF notifies other hosts within the same zone. Communication between hosts in different iNOF zones is not allowed.

Working process of the iNOF network:

When a new host connects to the network or an existing host disconnects, the devices on the network follow these steps:

• Step 1: The host sends an LLDPDU (Link Layer Discovery Protocol Data Unit) containing the connection or disconnection information to the connected iNOF client.

• Step 2: The client synchronizes the received information to other hosts in the same zone and the connected reflectors using iNOF packets.

• Step 3: The reflectors synchronize the received information to other connected clients using iNOF packets.

• Step 4: Other clients synchronize the received information to hosts in their respective zones and the connected reflectors using LLDPDUs.

Following these steps, all devices on the network have up-to-date host information. This information allows the network to adjust related configurations, ensuring low latency, zero packet loss, and high throughput as required.

Association Between BFD and iNOF

Bidirectional Forwarding Detection (BFD) is employed to swiftly identify communication faults between devices and communicate them to upper-layer applications. BFD for iNOF integrates BFD with the iNOF protocol, enabling BFD to notify the iNOF protocol about detected link faults. The diagram below illustrates the functioning of BFD for iNOF.

• 1. In an iNOF system, DeviceA, DeviceB, and DeviceC exchange address information via the iNOF protocol. host1 to host6 are connected to the network through these devices. Specifically, host1, host5, and host6 belong to iNOF zone zone1, enabling communication among them.

• 2. DeviceA, DeviceB, and DeviceC establish BFD sessions based on their peer addresses. Once the BFD sessions are established, BFD initiates the detection of link faults.

• 3. If the link between DeviceA and DeviceC experiences a failure, BFD rapidly identifies the fault, marks the status of BFD session 2 as "down," and notifies DeviceA and DeviceC about the fault.

• 4. DeviceA and DeviceC inform host1, host5, and host6 within the same iNOF zone about the fault and instruct them to perform a link switchover. This results in a revised path from the original optimal link (host1 -> DeviceA -> DeviceC -> host5/host6) to the new optimal link (host1 -> DeviceA -> DeviceB -> DeviceC -> host5/host6).

Typical Application of iNOF

In the centralized storage architecture with three-layer networking, a singular data center (DC) employs the compute leaf-spine-storage leaf design, as depicted in the accompanying illustration.

To improve the reliability of the product, a centralized storage cluster incorporates two network planes, namely plane A and plane B, which serve as backup for each other. Leaf1 operates on plane A, while Leaf2 operates on plane B. Within the iNOF system, leaf nodes act as clients, while spine nodes function as reflectors. The leaf nodes only require configuration to establish iNOF connections with the spine nodes, while other iNOF functionalities are configured on the spine nodes.