Data centers are undertaking profound changes due to the ongoing data center consolidation, virtualization and cloud technology, as well as the increasing demand on application workload extensity and agility. All these driving forces are changing every facet of data center. The topology is flattened from the traditional core-aggregation-access 3-tier design to the spine-leaf 2-tier architecture. And thus, the bandwidth capacity are boosted from 1 GE at access and 10 GE at aggregation to 10 GE at leaf and 40 GE at spine. Cisco Nexus 9500 and 9300 series switches aim to take data center networks through all these transitions and even beyond. With their unprecedented performance and port density, they are well suited in both the traditional 3-tier design and spine-leaf design providing forwarding capacity for fast increasing server-to-server traffic. This article will focus on the interconnection option for Cisco Nexus 9000 series switches.
Designed to meet the requirements of mission-critical data centers, the Cisco Nexus 9000 series switches deliver exceptional availability, and outstanding scalability. It encompasses larger Cisco Nexus 9500 series modular switches and smaller Cisco Nexus 9300 series fixed-port switches. The Cisco Nexus 9000 series switches are ideal for small-to-midsize data centers with five key benefits: price, performance, port-density, programmability, and power efficiency. It is the best platform for 1-to-10 GE migration, 10-to-40 GE migration, and can easily be integrated with existing networks.
The Cisco Nexus 9500 series switches include Cisco Nexus 9516, 9508, and 9504 switches. Each shares the same supervisor, line cards, system controller, and power supplies, and can be deployed in both traditional three-tier and leaf-spine fabric architectures. With an innovative midplane-free design, the Cisco Nexus 9500 provides front-to-back airflow, reduced power consumption, increased reliability, and outstanding scalability.
Including Cisco Nexus 9396 switch and Cisco Nexus 93128 switch, the Cisco Nexus 9300 series switches are fixed switches built to provide 1/10 Gbps and future FCoE server access connectivity. The Cisco Nexus 9396 is a 2RU fixed switch offering 48 ports of 10 Gbps (SFP+) and 12 ports of 40 Gbps (QSFP+) with a non-blocking switch architecture. The Cisco Nexus 93128 is a 3RU fixed switch with 96 ports of 10GBase-T and 8 ports of 40Gbps (QSFP+). Both Nexus 9396 and Nexus 93128 are well suited for data center top-of-rack (ToR), end-of-row (EoR), and collapsed aggregation and access.
Cisco Nexus 9396 and 93128 to Cisco Nexus 9508 series use QSFP+ to QSFP+ MPO connectivity for multimode connectivity.
For distances less than 400 meters, the use of MPO multimode fiber cabling is generally the preferred cabling method. The picture above shows a wiring option for connecting a Cisco Nexus 9396 to a Cisco Nexus 9508 and to a Cisco Nexus 93128 switch using MPO interconnect cable assemblies. With Cisco QSFP-40G-SR4 optics, this scenario is capable of distances of up to 100 meters on OM3 fiber or 150 meters with OM4 fiber. The QSFP-40G-CSR4 optics can be used for distances of up to 300 meters with OM3 fiber or 400 meters with OM4 fiber.
From the Cisco Nexus 9396 port, the QSFP-40G-SR4 or QSFP-40G-CSR4 optics connects to an MPO fiber interconnect cable to the front of the fiber optic adapter panel. The fiber optic adapter panel fits into the patch panel. The MPO fiber trunk cable plugs into the back of the fiber optic adapter panel using the port that corresponds to the port that the interconnect cable used in the front. On the remote end, the MPO fiber trunk cable plugs into the back of the fiber optic adapter panel. The MPO fiber interconnect cable plugs into the corresponding port on the front of the panel on one side and into the optics with the switch on the other end.
The MPO interconnect fiber assembly used in the scenario is a female MPO to female MPO cable, and the MPO trunk cable assembly is a male MPO to male MPO cable. The reasoning behind this cabling methodology is that the interconnect or intracabinet cables will always be female to female, because MPO optics in a switch or router are always male connectors per the IEEE standards and will always accept female MPO connectors. The trunk cabling will then always be male MPO to male MPO trunk cables. The use of this cabling methodology throughout the data center allows a simplified cabling scheme and quick identification of interconnect and trunk cabling.
In addition to using the female MPO for interconnect cables and the male MPO for trunk cables, you should use key up to key up (see the picture above) cabling for both the interconnect and trunk cables. This approach standardizes the type of cable used throughout the data center, eliminating the need for several different interconnect cable types.
The Cisco Nexus 9000 series switches were created to optimize network performance to allow virtualized environments to function without the restrictions that traditional network architecture can impose. For a Cisco Nexus 9000 series switching platform network to operate at the highest level, it requires a physical infrastructure built to help ensure superior performance. The interconnection option for the Cisco Nexus 9000 series switches in this article is for reference only. There will be more good options to be explored.