Cancel
https://media.fs.com/images/community/uploads/post/202109/15/post1-data-center-zhw1y4gmyt.jpg

Data Center Architecture Design: Top of Rack vs End of Row

August 31, 2021

Datacenter architecture design is largely dependent on data communication and the interconnection of switches. When there are a lot of servers to connect (like in a Data Center), networking needs to be flexible/scalable enough to handle the computing power required for large installations. Two popular network design schemes used in such scenarios are the ToR (Top of Rack) and EoR (End of Row) configurations. Here, we explain the ToR Vs. EoR debate in simple words.

Top of Rack (ToR) Architecture :

Top of rack (ToR) which is also known as In-Rack design. In this approach, the network access switch is placed on the top of the server rack hence; servers are directly connected to the network access switch. This means that 1 or 2 Ethernet switches are directly installed inside the rack, therefore copper cables stay inside the rack. It is cost-effective because it reduces the number of copper cables between racks. The rack is linked to the data center network by an Ethernet switch, often through a fiber cable. This fiber cable is a direct link from the common aggregation area to the rack.

Top of Rack (ToR) Architecture

In the ToR approach, every rack in the data center network is a separate entity that eases its management. Any change, upgrade, or malfunction in the rack usually affects that rack only. Fewer cables mean that one can opt for better quality and higher bandwidth cables in the same budget.

Advantages of ToR

ToR design requires fewer numbers of cables, which reduces the complexity in the cabling network; this is because all servers in the rack are connected to the switch/ switches mounted in the same rack. Contrary to EOR design, not every server has to be connected with a common aggregation switch, hence greatly reducing the number of cables going outside the rack.

Copper cables are used inside the rack and fiber optic cables are used to connect the rack with an aggregation switch, this arrangement allows an opportunity for upgrading a network with a bandwidth of 1 G/10G to a high-speed network of 10G/40G in the future with minimum cost and cable changes. Small rack size permits the use of 1 switch for more than one rack in ToR.

Since each rack in ToR design is autonomous, modular deployment is effortlessly possible with it, which means a rack with all essential requirements can be easily connected and deployed.

Disadvantages

TOR is a distributed network design, hence more switches are required.  An increase in the number of switches means an increase in the number of ports in aggregation switches. Maintenance cost is higher. Underutilization of switches is also possible which results in wastage of power and cooling systems.  If one switch fails, an entire rack goes offline. ToR configurations can oftentimes lead to land locking of equipment due to short cabling guidelines and data center policies that restrict cabling between two racks.

End of Row Architecture (EoR)

In EoR network design, there is a direct connection of each server in the rack with the end of row aggregation switch. This eliminates the need to connect servers directly with the in-rack switch.

Racks are normally arranged in such a way that they form a row, a cabinet or rack is positioned at the end of this row. This rack has the row aggregation switch, which provides network connectivity to servers mounted in individual racks. This switch, a modular chassis-based platform, sometimes supports hundreds of server connections. A large amount of cabling is required to support this architecture.

In ToR each rack is an independent unit whereas in EoR the whole row of servers acts as a  group within the data center. Any issue with the row aggregation switch impacts the complete row of servers.

End of Row Architecture (EoR)

Advantages of EoR

In EoR, the number of devices is far less compared to ToR because not every rack has a switch. Fewer devices result in less rack space, a decrease in maintenance effort, less power consumption and hence less requirement of cooling systems. It has improved port utilization compared to ToR and reduced latency and delay since data packets have to travel through fewer switches. Using EoR structured cabling, a single switch upgrade can enhance latency across multiple cabinets in a row. Low-latency communication between two servers in a row is achieved instead of servers residing in the same rack.

Disadvantages

Since the number of cables in EoR is on the higher side, therefore cabling complexity is increased compared to ToR.  The outcome is a complex, expensive, rigid, bulky cabling network. Maintenance and management of the cabling system are quite challenging. The quality of cables affects the overall bandwidth of the network. Upgrading the cabling system is difficult and costly.

Conclusion

ToR configuration requires one switch per rack which leads to increased power consumption and operational costs when compared with EoR setup. In this scenario, even the number of unused ports is often higher compared to the EoR arrangement. However, cabling requirements are much lesser in ToR compared to EoR and the fault tolerance of the whole data center is improved as faults largely remain isolated to a particular rack only. If an organization aims to save on operational costs then EoR configuration is preferred while ToR is the better choice if fault-tolerance is the ultimate goal.

This ToR VS EoR design debate can be briefly summarized in the form of the following table.

ToR Design
EoR Design
Minimum 1 Switch per rack
1 Switch per row of racks
Switch  Count is higher
Less number of switches
Less number of Cables
Higher number of Cables
Each rack is a separate
module Racks work as a group
Upgradation is easy
Upgradation is difficult
Underutilization of switch
Effective utilization of switches
Greater layer 2 data traffic
Lesser layer 2 data traffic
High power consumption
Les Power Consumption
Greater need for cooling
Lesser need of cooling
165

You might also be interested in