As designers push the limits with silicon, electronic switch-fabric components are a hot commodity for today's core and metropolitan applications.
PAUL LIESENBERG AND CHRIS BERGEN, ZettaCom
The pace at which specialized switch-fabric vendors have been acquired hints at the fact that this is a key technology for next-generation network systems, ranging from optical crossconnects to Internet Protocol (IP) backbone and metropolitan multiservice provisioning platforms (MSPPs). What seems to be more surprising is that at a time when all-optical technologies capture most attention, it is silicon-based switching-fabric components that are obviously regarded as the hot commodity in the core and metropolitan networking space.
The reason for that is quite simple. Electronic switching fabrics are necessary to deliver service-level granularity which, in turn, is indispensable for the different levels of categorization a network infrastructure needs to implement if it aspires to deliver the quality of service today's users demand. This categorization, based on the interpretation of packet-data unit headers, can only and exclusively be implemented in electronics.
Pure photonic switching technologies offer only spatial crossconnect capabilities-that is, physical port to physical port. While certainly serving a critical key application in the transport infrastructure, they do not suffice to implement the levels of logical abstraction required to deliver actual services. Therefore, the switching fabric has a crucial role to play when it comes to guaranteeing predictable levels of service in a network system. And since it is required, it has a key role to play in guaranteeing system aspects that have become crucial for a networking system, including scalability, flexibility, low power consumption, and a very high throughput/space coefficient.
To understand the switching-fabric marketplace, it is also important to understand why network system designers have become more and more receptive to the idea of outsourcing a component as critical as the system's switching fabric. The Internet revolution started an explosive growth of users, services, and traffic volume and traffic types. The advances in optical technology have been equally important for network infrastructures.
DWDM has multiplied the bandwidth available to services at the transport layer. Because of the explosion of both the service and transport layer, network systems that mediate between these network layers and are responsible for traffic classification, switching, and forwarding, threaten to become the bottleneck of the emerging infrastructure. That is not necessarily because these systems cannot deliver the capacity required-rather it is because they have traditionally lacked the utter architectural flexibility that current network trends demand.
They do not scale gracefully, making network growth a painful and expensive proposition, and they are not delivering on the promise of service convergence that by now several generations of network equipment have been promising. From narrowband Integrated Services Digital Network to ATM to multiservice platforms, subsequent generations of hardware were not able to significantly ease costly operational aspects of service-provider network infrastructures. Next-generation architectures are most definitely expected to change that.
The combination of increasingly complex service implementation aspects and service convergence requirements leads to network system companies realizing that their expertise has to shift to service control and thus to higher-layer software aspects. They must outsource more and increasingly critical hardware components to a new generation of component companies that now must develop products for the very high performance, reliability, and feature standards that network system companies require. Finally, the network communications integrated-circuit components market will not be successful based simply on the technical merits of its components. Rather, close partnerships and an excellent support structure will be just as important.
The requirements placed on today's networking systems' switching fabrics are very strict. We'll analyze the most important aspects, one by one, and try to give some suggestions on how to measure a particular switching fabric's performance in that particular area.
A requirement that comes up every time the Internet is positioned as the next-generation infrastructure is reliability. New-generation networking systems have to deliver reliability parameters that measure favorably against the trusted plain old telephone service. Given the switching fabric's central position in a networking system's architecture, its contribution to overall system reliability is high.
In addition, the switching fabric can implement mechanisms that mitigate the impact of other components' failure on overall system availability, too. For example, the switching fabric could provide immediate switchover between redundantly configured components. The switching fabric's design ought to ideally take into account the several aspects that greatly contribute to overall system reliability.
- The ability for immediate, loss-free rerouting around inner failures. Switching fabrics are modular architectures that consist of a greatly varying amount of components. It is imperative that there are no single points of failure in the architecture. The classic modular crosspoint matrix design typically implements different switching planes that work in parallel. Thus, the interface cards ought to be connected to each and every switching plane. Both 1:N and N:1 reliability schemes ought to be supported to provide the system designer with full flexibility when it comes to the implementation of a redundancy strategy.
- Immediate switchover between components. Since a switching matrix interconnects different system elements, it can be used to implement loss-free switchover functions quite easily. For example, if a high-availability line card is implemented with 1:1 redundancy, it is through switchover of the path to the switch fabric that standby redundancy is achieved. While the intelligence to do that resides on the line card, care must be taken to be sure the basic design of the switch fabric makes it possible to support such features.
- High integration level. Switching fabrics differ greatly in the number of discrete elements used to build a particular architecture. By their nature, architectures that use more components are more likely to fail, given the same overall meantime between failures for the different components. Thus, switch-fabric designers must take advantage of the latest advances in silicon fabrication to provide their customer with the best possible component reliability. High integration also typically results in low power consumption and lowered heat dissipation, a key requirement in today's crowded central-office environment.
- Architectural flexibility. It is a huge advantage for a system designer if a switching fabric allows for architectural flexibility when it comes to the introduction of reliability schemes. That allows the overall system reliability to grow with the success and maturity of the service the architecture is primarily supporting. System designers can start off with a basic, simple-and cost-effective-redundancy scheme and evolve their architecture toward more evolved and complete reliability schemes.
There is little doubt that exponentially increasing traffic levels call for a higher switching capacity in networking systems, indicated by increased industry interest in terabit switching architectures for core networks. When analyzing switching-capacity claims, however, it is very important to be aware that the marketplace has strayed away from the stricter definition of "nonblocking capacity." It has recently become very common to count ingress and egress traffic separately, thus doubling the switching capacity.
Additionally, internal "system overhead" often is included in capacity claims. For instance, if a switching fabric claims to support 1 Tbit/sec but physically can only connect 16 OC-192 (10-Gbit/sec) ports, for all practical purposes the maximum system throughput will not exceed 160 Gbits/sec. Unlike the practice of counting ingress and egress traffic separately, there is some merit to quoting the system fabric's capacity irrespective of other architectural components. For one, it illustrates the potential expandability of the system in the future, showing the architecture could, for example, grow to support higher-density interface cards.
More importantly, quoting the total capacity of the switching fabric-particularly the speed and number of individual "threads" (the interconnection links between fabric and line cards)-helps designers assess the amount of internal system "speedup" designed-in. Speedup is a crucial aspect of switching architectures, since true nonblocking switches ought to operate at a faster inner rate than the fastest supported interface speed. For example, if a switch supports an OC-192 line card, the switch-fabric threads ought to operate at an even faster internal rate. This internal speedup is not only to accommodate necessary intrasystem overhead, but also to deliver true nonblocking performance for variable-size data-unit traffic and support multicasting capabilities without jeopardizing system performance.
If the internal switch-fabric path also supports 10 Gbits/sec, or the same speed as the line card port for OC-192, a system bottleneck is preprogrammed. It means the egress buffers-responsible for quality of service (QoS)-cannot be used efficiently, since the switch fabric can only serve data units at exactly the same rate the egress line can service. It also means the fabric is far more likely to become lossy in the case of internal contention, which will occur regularly in the real world.
It is often forgotten that the term "nonblocking" in switching refers to the fabric's behavior when servicing statistical Bernoulli traffic, representing a flow of information units with no correlation whatsoever. Unfortunately, real-world traffic often shows correlation, be it concurrent data units pertaining to the same flow, cells pertaining to the same IP packet, or traffic concentration onto one destination that for some reason attracts a lot of traffic. A truly versatile switch fabric has to provide a switch designer with a foundation to optimally service such traffic characteristics. Being merely nonblocking based on the academic definition of the term will not do.
Another aspect to keep in mind when evaluating switching capacity is that while there's little doubt the exponential increase in data traffic has resulted in a requirement for higher switching capacities, it remains to be seen whether monolithic, huge, multishelf multiterabit switching architectures are going to see very widespread acceptance-particularly since floor-space and power consumption have become such critical considerations in the networking system environment.
A more distributed network architecture with more, nimbler, and higher "switch-capacity volume metric" network elements, would also increase overall backbone switching capacity. Architectural versatility is the key to a successful switching fabric, allowing system designers to implement whatever fundamental architecture they choose and quickly adapt the architecture to rapidly evolving market trends. Ideally, the design tradeoffs between a single-shelf, compact system or multishelf system should not be a strict either/or proposition. Rather, the switching-fabric components should be flexible enough to build either architecture, allowing for smooth migration from one to the other with maximum investment protection.
Often, scalability is somewhat myopically seen as the ability of a switching architecture to grow in capacity. Yet, in today's disruptively fast-changing networking environment, it should have both physical and logical meaning. Scalability today means the ability to adapt to architectural changes with maximum investment protection, support increasing interface speeds and densities, and have the flexibility to efficiently support new services, along with the ability to increase switching capacity. Also, the flexibility to implement continuously improved reliability schemes is a further aspect of scalability, as it was with reliability.
The more traditional meaning of scalability, however, remains very valid: the ability to increase capacity with increasing service demand without disruption and with investment protection. To do that, a switching fabric should be deployable in a compact single shelf or in very large multishelf architectures, allowing the former to evolve into the latter.
The Internet revolution has resulted in an explosion of traffic and services. Switching architectures need the fundamental ability to accommodate different service types efficiently and balance their requirements to provide fair resource utilization within the system architecture. While the switching fabric certainly isn't the only system component providing QoS aspects-the line cards' buffer and bandwidth management schemes are of crucial importance-its overall contribution is very significant.
Line cards balance buffer and bandwidth utilization within the ports they service, while the switching fabric actually balances the requirements of priority types across different line interfaces. If, for example, one line card has high- and low-priority traffic and another has only low-priority traffic, older-generation switching fabrics would merely accept data units from both line cards, leaving prioritization to the line card themselves. A QoS-aware switching fabric, on the other hand, would serve the two high-priority traffic data units from the same card and backpressure the low-priority traffic on a system-wide basis. Of course, this example is coarse and does not quite illustrate the fine granularity a new-generation switching fabric has to implement to deliver on ever-increasing QoS requirements.
The optical revolution in the transport network coupled with fast-growing user numbers and higher-bandwidth services results in a requirement for switching architectures to support constantly increasing interface speeds. While there is currently some debate regarding how quickly OC-768 (40 Gbits/sec) will penetrate networks, in the case of a switching architecture, it is immaterial whether the requirement is for OC-768 or for high denisity, (i.e., 4-port OC-192 [10-Gbit/sec]) line cards. The latter is quite likely to become a hot requirement in a market that sees OC-192 multiplying due to ever-increasing DWDM deployment.
In a market that is also characterized by collocation issues (efficient floor utilization and low power consumption), the architectural flexibility to support ever faster and increasingly dense line cards is critical. Switching fabrics need the ability to flexibly add more bandwidth and make it available to line cards without any service disruption and while maintaining full backward-compatibility and investment protection.
It is evident that monolithic architectures lack the flexibility to deliver on all the system requirements outlined earlier. The next generation of network system will more likely implement intelligent, distributed, self-discovering switching fabrics that possess total architectural flexibility. Companies designing these next-generation switching fabrics require a combination of extensive silicon design experience and a strong system background. Engineering experience in system design becomes a critically important asset in this emerging component marketplace.
Paul Liesenberg is director of strategic marketing at ZettaCom (San Jose, CA). He can be reached at [email protected]. Chris Bergen is chief technical officer at ZettaCom. His e-mail address is [email protected].