By Serge Melle
A desire to significantly scale network capacity (typically on the order of 50% to 100% per year), simplify network layers and operations, and in many cases create multiservice networks that can support legacy TDM, wholesale bandwidth, Ethernet, IP, and more has led to an interest in packet-optical convergence. Several network architectures, each with relative merits, have been offered to achieve this goal. We’ll explore the various options for converging network platforms, functionality, and layers as a means of lowering total cost of ownership.
View from the top
As a precursor to this exploration, let’s visit a typical service provider’s core network, looking down the protocol stack from the IP layer to the WDM layer.
The IP layer comprises two distinct sets of functionality: Provider Edge (PE) routing and the simpler Provider (P) routing. PE routing supports IP services from client routers, be they from the same provider or peered with external users, clients, or providers. PE routers support a wide range of routing protocols, including Border Gateway Protocol (BGP), Open Shortest Path First (OSPF), Intermediate System to Intermediate System (IS–IS), and a rich set of Multi-Protocol Label Switching (MPLS) functions supporting services such as Layer 3 virtual private networks (VPNs) and virtual private LAN services (VPLS).
At the same time, a service provider’s core IP network supports P routing, typically in the context of MPLS-enabled router networks. The P routers connect to one or more PE routers, and primarily act as an MPLS label-switch router (LSR), responsible for switching the labels used to route packets at intermediate pass-through router nodes toward their eventual destinations.
While not necessary, typically the PE and P router functions are implemented in the same system, generically termed a “core router.” This integration results in burdening the overall cost of router interfaces with large memory and processing functionality to perform highly granular packet processing on a wide range of flows, support thousands of queues per line card, and perform complex multilevel scheduling, lookups, and manipulation of packets.
In most IP networks, core routers are typically connected to two to four other core routers in a sparse network topology, using the highest-speed router interfaces, which ideally provide the lowest cost per bit. The industry has seen router interfaces progress through SONET/SDH speeds of 2.5, 10, and 40 Gbps and more recently to 10-Gigabit Ethernet (10GbE) and 100GbE to leverage the improved economics of Ethernet WAN protocols. A packet will typically traverse multiple intermediate routers (usually called “hops”) between its source and destination. Thus a large proportion of packets at any router may simply be transiting through that node, requiring only MPLS switching but using instead an expensive feature-rich PE card across many more router hops than ideally required.
Meanwhile, the WDM network underlying the IP network serves two roles. First, it acts as a server layer, providing interconnection between IP network core routers over long-haul distances. Second, the WDM layer offers multiservice transport, supporting a wide range of services, including SONET/SDH, wholesale wavelength, storage area network, Carrier Ethernet, and others that originate from a platform other than a router. In many cases, the transport layer includes a digital switching layer, historically SONET/SDH but evolving toward Optical Transport Network (OTN) switching, to provide multiservice service restoration, reconfiguration, and remote provisioning. A typical core network architecture is shown in Figure 1.
Given the typical core packet-optical-network architecture, opportunities for reducing total network cost of ownership can be narrowed to three main areas:
- Reducing interconnections between the IP and WDM layers and the associated interfaces in each layer.
- Reducing the number of intermediate router hops, especially if these hops occur using fully featured – and thus generally more expensive – core routers built with PE capabilities.
- Reducing the number of router ports used in general, whether in PE or P routers.
One simplistic architecture proposed for packet-optical convergence is IP over WDM (IPoWDM), which targets reducing the interconnections between the IP and WDM layers by moving the colored WDM interfaces directly into the router. Thus core routers connect over a wavelength-switched network with no intermediate OTN switching layer and no changes to the IP network connectivity or topology.
While the intuitive conclusion is to assume network cost savings from this convergence, studies have shown these savings are minimal. In practice, IPoWDM simply moves the cost of colored WDM interfaces from one network layer to another, with no reduction in the number of router ports. At the same time, it fundamentally ties the capacity of the WDM layer to router port speed. That may limit achievable capacity and density compared to WDM system technology, which today supports scaling to 500-Gbps superchannels. In addition, operation of an IPoWDM core network over a purely photonic transport layer can add significant operational complexity as the predictability, reconfigurability, performance monitoring, and reliability of a switched transport network is lost.1
Another approach to packet-optical convergence proposes somewhat of an oxymoron, namely splitting the core router into two discrete platforms. Here, PE routers supporting IP services connect via short reach gray optics to separate P routers for MPLS midpoint LSR switching. That enables router offload of expensive core routers built to support PE capabilities and uses lower-cost P routers at intermediate nodes, thereby reducing total network cost.2 In this case, the P router may also use integrated WDM optics, implementing MPLS over WDM (MPLSoWDM).
While both IPoWDM and MPLSoWDM reduce connections between layers, and MPLSoWDM further reduces the cost of MPLS switching, the use of integrated WDM interfaces in the PE or P routers is a drawback. Also, since router topology remains the same, neither reduces the overall number of generally costly router ports. Finally, neither approach effectively supports a multiservice network for the many services that originate most cost-effectively elsewhere than from a router.
Alternative approaches
But increasing the connectivity of routers in the IP layer, and thereby enabling router bypass, does reduce the number of router ports. In such an architecture, when packet traffic between two router nodes reaches a level comparable to router port speeds (i.e., 10, 40, or 100 Gbps), a dedicated router trunk is established between these routers. Packet traffic between these two nodes can thus “bypass” intermediate routers where they would otherwise undergo midpoint LSR switching and thus avoid the intermediate router and WDM interfaces (see Figure 2).
Applied network-wide, this architecture removes many intermediate router hops. Studies have shown this approach can enable total network cost savings of 20%–30% in the IP layer and 10%–15% in the WDM layer, well in excess of what is possible from using IPoWDM.3
While both IPoWDM and MPLSoWDM can enable router bypass, the fact that these architectures tie the WDM port directly to the router makes them less than optimal. Effective router bypass optimizes port speed to router-to-router traffic flows and may involve a mix of trunk speeds at 10, 40, and 100 Gbps. As mentioned, integrating the lower-speed colored WDM optics into the router reduces the WDM layer’s capacity potential and limits network scaling.
Conversely, when the highest-capacity wavelength speed is used throughout, integrating WDM optics into the router also limits the granularity of router bypass and associated savings. In addition, the turn-up and reconfiguration of router trunks is directly constrained by the end-to-end optical-link engineering of each wavelength. This constraint creates operational limitations when issues such as optical regeneration, optical path restoration, and wavelength blocking come into play.
The alternative approach that does not incur these compromises proposes to integrate, without compromise, WDM transport, OTN multiplexing and grooming, and MPLS switching into a single converged packet-OTN (P-OTN) platform for multiservice-network applications. A fully converged P-OTN platform can simultaneously enable router offload using MPLS switching, provide bandwidth efficiency and operation simplicity using OTN switching, and ensure bandwidth scaling with WDM.
Evolution to a P-OTN network can happen in a phased and seamless manner through a staged introduction of converged network functionality (see Figure 3). As a first step, the convergence of OTN switching and WDM transport into a single integrated platform enables operators to set up router-to-router bypass trunks between either PE/P or P routers using the most cost-effective and bandwidth-appropriate interface.
OTN switching enables rapid and remote setup and reconfiguration of the router trunks, independent of optical-link engineering constraints, thereby simplifying operations. Meanwhile, the integrated WDM functionality enables multiplexing of all router trunks into 100-Gbps wavelengths or 500-Gbps superchannels, thereby maximizing WDM capacity and network scalability. The OTN switching is also a packet-friendly technology that maps well into existing service-provider OSS/BSS processes and delivers the predictability, performance monitoring, and reliability that service providers expect.
A second evolution in P-OTN functionality convergence adds packet-aware OTN transport, which enables VLANs from PE/P router ports connected to the P-OTN system to be mapped to ODUflex containers. This ability extends the benefits of router bypass by enabling setup of virtualized router bypass links within the OTN layer with a variable bandwidth granularity of Nx1.25 Gbps. Routers thus can connect to the P-OTN system using the most cost-effective and scalable interfaces (e.g., 100GbE), while the P-OTN platform can manage bandwidth-optimized MPLS flows between routers to enable the most cost- and bandwidth-efficient implementation of router bypass.
Ultimately, P-OTN systems can evolve to subsume the MPLS midpoint LSR switching directly within the converged packet-optical layer, peering directly to PE routers deployed at the edges of the IP network and supporting a wide range of IP services. Such a hybrid P-OTN functionality offers the possibility of network cost savings by integrating the specific P router functionality required for a midpoint LSR router, such as MPLS, pseudowires to support very high speed Ethernet services, modestly sized IP forwarding tables, and a limited set of core routing protocols, including OSPF(-TE), IS–IS(-TE), LDP, and RSVP-TE.
The right approach
Critically, a P-OTN platform converges multiple network layers. The platform eliminates hundreds if not thousands of short reach optics typically used for network interconnection, makes full use of router bypass and offload to optimize router investment, and unbridles WDM to scale to 500-Gbps superchannels (and 1-Tbps superchannels in the future) as required to support rapidly growing networks.
By considering the relative merits of different approaches to packet-optical-network convergence, operators can optimize their network evolution to their specific needs for capacity scaling, service support, and phasing of network architecture and functionality. Ultimately, the availability of fully featured converged P-OTN platforms will enable operators to optimize their networks most efficiently across all these metrics.
References
- Melle, S., G. Bennett, C. Liou, C. Villamizar, and V. Vusirikala, “Alien Wavelength Transport: An Operational and Economic Analysis,” Optical Fiber Communications/National Fiber Optic Engineers Conference, Session NThF2, Anaheim, CA, March 22–27, 2009.
- Kompella, K., and P. Belotti, “Transport networks at a Crossroads: The roles of MPLS and OTN in multilayer networks,” Optical Fiber Communications/National Fiber Optic Engineers Conference, Session OTuG, Anaheim, CA, March 6, 2011.
- Melle, S., D. Perkins and C. Villamizar, “Network Cost Savings from Router Bypass in IP over WDM Core Networks,” Optical Fiber Communications/National Fiber Optic Engineers Conference, Session NTuD, Anaheim, CA, Feb. 24–28, 2008.
SERGE MELLE is vice president of solutions marketing at Infinera, responsible for market development, technical customer support, and network-architecture strategy. He has been with the company since 2002.