Tackling 800g Ethernet Challenge: Data Center Migration

For the past few years, the demand for more bandwidth, faster speeds, and lower latency performance has been driven by the accelerating adoption of cloud infrastructure and services. Changes in cabling and architecture are being compelled by improvements in switch and server technology. Consequently, the need for data center migration, especially for speed—including bandwidth, fiber density, and lane speeds—is increasing dramatically in 800G Ethernet.

Embrace 400G/800G Ethernet Change

In terms of application, Breakout applications are currently being promoted in switch ports; typically, 400G and 800G are broken out to 4×100G or 8×100G. As for capacity, currently supporting 64×400G ports and having a bandwidth of 25 Tb/s, Broadcom's Tomahawk 4 switching chip has the potential to support 32×800G ports in the future. On the physical layer, the 2017 approval of IEEE 802.3bs opened the door for 200G and 400G Ethernet. Therewith the adoption of 400G has ramped up rapidly. And it is conceivable that 800G Ethernet adoption is anticipated to extend in the near future, with an even higher rate of acceleration. Pursuing improving power and cost per bit, the industry is now collaborating to introduce 800G data center and start moving toward 1.6T and beyond.

Key Elements in 800G Ethernet Network Migration

1. Increased switch-port densities

As technology advances and market demands change, both SERDES(Serializer/Deserializer) and ASICs(Application Specific Integrated Circuits)' design and manufacturing techniques continue to make progress. With the function of converting parallel data into serial signals, SERDES technology achieves high-speed data transmission on the basis of relatively few physical pins and circuit resources. This enables switches to support more ports with limited hardware resources. Besides, as specially designed application-specific integrated circuits, ASICs can provide highly integrated hardware solutions. By using ASICs, switches can integrate multiple ports or interfaces on a single chip, reducing physical space footprint, and thereby increasing switch port density.

As a result, the total number of switches required for 400/800G networks is correspondingly reduced, which further raises the need for new optical modules and structured cabling.

2. Optical transceiver technologies

New-generation transceiver formats

The CFP (C form-factor pluggable) package was first released by the CFP MSA Association to be applied in early 100G optical modules. With the development of chip technology, MSA also introduced CFP2, CFP4 and CFP8 standards. The CFP8 format was introduced in 2017 to support earlier 400G optical transceivers, which can support 16 channels of 25G NRZ signals for 400G transmission. Later, it was gradually replaced by smaller sizes of QSFP-DD(dual-density four-channel) small pluggable format and OSFP format. The OSFP format supports eight sets of high-speed electrical transceiver channels, which can provide a connection interface up to 400Gbps(8x50G PAM-4). Now QSFP-DD and OSFP have become the preferred packaging technologies for most vendors.

Overall, with the upgrade of transceiver formats, the optical module power consumption is getting lower while the volume is getting smaller, which contributes a lot to 400G/800G data center migration. If you want to know more specifications about various transceiver formats, please check Differences Between QSFP-DD and QSFP+ / QSFP28 / QSFP56 / OSFP / CFP8 / COBO.

Higher-speed modulation schemes

For a long time, 1G, 10G and 25G optical modules have been using non-return-to-zero (NRZ) modulation technology. Unlike NRZ, PAM4 uses four signal levels, and each symbol period can represent two bits. At the same baud rate, the throughput of PAM4 is twice that of NRZ, which effectively reduces the loss of the transmission channel and improve bandwidth utilization.

In 400G/800G data centers, the NRZ modulation scheme requires vast optical fibers. Besides, the transceiver chip time margin, transmission link loss, and size can not meet the requirements of 400G/800G Ethernet. With the development of big data and cloud computing, PAM4, as the most efficient modulation technology at present, has become an inevitable trend in the development of 400G high-speed connection interfaces. At present, the 400G interface standard of 4*100G PAM4 and 8*50G PMA4 has been formally proposed by the IEEE Working Group and applied in 200G/400G/800G optical modules. In the near future, PAM4 will become the mainstream way of high-speed Ethernet signal modulation with its own advantages (such as high performance, etc.). In the wake of Ethernet migration, signal modulation technology will continue to develop and innovate in a more complex direction.

For more detailed information on modulation techniques, please view this blog: NRZ vs. PAM4 Modulation Techniques.

3. Options on connector

MPO and duplex connectors are widely used in 400G and 800G Transceivers. MPO connectors are often used for multimode fiber transmission - the connector has a high-density design that allows multiple fibers to be connected simultaneously. In MPO schemes, 400G Ethernet is usually transmitted through eight optical fibers, each with a rate of 50Gbps; 800G Ethernet over 16 optical fibers, each with a rate of 50Gbps.

Duplex connectors are generally used for single-mode fiber transmission, and each connector is connected to two optical fibers. For increasing channels and channel speeds, duplex connectors with a smaller footprint can provide more flexible split-line options for high-speed modules. 400G connection schemes use four duplex connectors; 800G eight duplex. By using duplex connectors, single-mode fiber can achieve high bandwidth and long-distance transmission. At the same time, it improves the transmission quality of the signal with great isolation capability.

In short, MPO connectors and duplex connectors both play an important role in 400G/800G Ethernet for high-speed and bandwidth transmission. Currently, the selection of these two types of connectors is not simply determined by rate, but by more factors such as the number of data channels it supports, the space occupied, and the transceiver and switch prices affected. In addition, in order to adapt to the needs of different scenarios, the selection of optical module connectors is showing a more and more diversified trend. In the future, further customized transceiver connector designs will also be produced for larger 800G data centers. New standards will also be proposed by organizations to meet the market demand brought by 800G Ethernet.

4. Cabling advances

To meet the challenges of 800G Ethernet change in migration, fiber cabling presents opportunities in the following aspects: OM5 fiber type; wavelength division multiplexing (WDM); ultra-low-loss (ULL) components; reduced number of server leaf switches(TOR); higher fiber-count cabling; right mix of SMF and MMF applications.

• OM5 fiber: The OM5 fiber has two major superiorities over its predecessors -- lower attenuation (3 dB/km) than OM3 and OM4 (3.5 dB/km); extended effective mode bandwidth (EMB) specification from 850 nm to 953 nm, which is more conducive to the application of SWDM technology.

• WDM: At present, two types of WDM technologies are mainly used in data centers -- coarse wavelength-division multiplexing (CWDM) and dense wavelength-division multiplexing (DWDM). They can provide more wavelengths and enlarge the capacity per fiber, which fits longer distance applications and saves costs.

• ULL: This technology includes high-quality components such as low-loss connectors, sockets, optical fibers, and splitters. With ULL components, the transceiver has lower insertion loss and return loss, which can provide more stable and high-quality signal transmission.

ultra-low-loss (ULL)

How to Implement 800G Ethernet Migration

1. Design higher-speed infrastructures

To redesign the data center for 800G Ethernet, we need to take into account the elements described in the previous section.

• Switch-port densities: increasing port density per switch or delayering switch framework to reduce the count of switch fabric tiers.

• Transceiver technologies: QSFP-DD and OSFP form factor(backward compatible with QSFP+ and QSFP28).

• Fiber cabling: new cabling designs—like 200-micron and rollable ribbon fiber to minimize footprints or a leaf-spine architecture that optimizes direct paths for server-to-server communication.

2. Arrange on existing infrastructure

• Fiber cores: using existing 8/12/24-fiber subunit trunks while the 16-fiber-based design is applied in new higher-speed infrastructures.

• Instrument testing and verification: using handheld test equipment to test channel performance(IL and RL are included).

800G Ethernet Challenge & Prospect in Network Migration

Move toward Ethernet 800G

With the advancement of science and technology, the modulation format and optical fiber transmission system have been upgraded time after time, thus the 800G data center migration is gradually put on the agenda. The IEEE (Institute of Electrical and Electronics Engineers) has developed standards to support 800G Ethernet, such as IEEE 802.3ck and IEEE P802.3cn. What's more, Multi-source agreement (MSA) successively Put forward 800G Pluggable MSA, 100G Lambda MSA, QSFP-DD800 MSA, and so on. These standards promote the implementation and application of 800G Ethernet.

Pursue high-speed Ethernet

Although 800G Ethernet migration is still at a relatively early stage, market demand has driven the exploration of 1.6T and 3.2T. At present, higher speed Ethernet still lacks the support of technology--such as link capacity and standard, hence facing many challenges. However, Co-packaged optics (CPO) and OSFP-XD MSA-proposing 16-lane count, provide opportunities for higher Ethernet.

Conclusion

With the advancement of technology and the drive of commercial demand, 800G Ethernet is expected to enter the commercial stage in the next few years. In the wake of continuous development and innovation of technology, high-speed Ethernet will continue to further improve its performance and application scope, creating more convenience and opportunities for people.