Adapting optical networks for Internet-based realities requires a unified control plane.
Internet service has traditionally been carried over a telecommunications infrastructure that was primarily designed to support voice and fixed circuits over a time-division multiplexing (TDM) transport. This approach has resulted in a complex multilayer network that is extremely difficult to manage, provision, and update.
In the 1990s, the Internet Protocol (IP) infrastructure was constructed using four layers of networking equipment (see Figure 1). Today, the development of WDM/DWDM technology has added yet another layer to the network equipment architecture-a fifth layer allowing service providers to rapidly grow their transmission capacity by allowing multiple wavelengths (or lambdas) to be transmitted over a single optical fiber.
Many of the largest carriers and service providers do not believe the existing multilayer architecture can scale to support the future requirements of the Internet. The complexity of the equipment layers is both costly to manage and difficult to update. In the cases of traditional telecommunications carriers, the majority of equipment was originally designed for voice traffic, with IP transmission capabilities supported as an afterthought. Furthermore, multilayer architectures typically suffer from the least-common-denominator effect, where the limitations of any one layer restrict the scalability of the network as a whole. As we enter the 21st century, the fundamental requirements of the new IP infrastructure can best be met by evolving the network to a two-layer architecture, with an IP-based Multiprotocol Label Switching (MPLS) service layer built atop an optical transmission layer.
The evolution toward a two-layer network architecture is currently underway as IP/MPLS and DWDM emerge as the two dominant technologies in recent network installations. IP routers are beginning to subsume a number of functions previously performed by other equipment layers.
The commercial availability of ASIC-based, wire-speed routers eliminates the need for ATM/frame relay switches to overcome the packet-forwarding limitations of software-based routers. OC-48c/STM-16 and OC-192c/STM-64 router interfaces eliminate the need for multiplexers to match the speed of router interfaces to the optical transmission infrastructure. MPLS replaces the need for ATM/frame relay equipment to perform traffic engineering functions. MPLS's fast reroute eliminates the need for restoration via SONET automatic protection switching. Finally, generalized MPLS (GMPLS) provides a scalable and cost-effective approach that allows carriers and service providers to rapidly provision optical bandwidth to facilitate the deployment of new services.
The two-layer network architecture will consist of IP routers, interconnected by optical crossconnects (OXCs) that use WDM/DWDM transmission over optical fiber. Those IP routers and OXCs are complementary, not competing, technologies. For this reason, the two layers cannot be converged into a single layer.
DWDM technology allows multiple wavelengths of light to be transmitted over a single optical fiber. That allows an enormous increase in the amount of data that can be transmitted over the existing fiber plant without requiring the expensive task of laying more fiber. DWDM's technological breakthrough is an essential component of growing the raw bandwidth essential for building large-scale IP networks, but DWDM technology alone is not sufficient to address the growing need for flexible, scalable, and affordable bandwidth.
OXCs augment the capabilities of DWDM by allowing the multiple wavelengths that emerge from a fiber to be routed and crossconnected separately. An OXC allows very large bandwidth connections, consisting of entire wavelengths, to be established between remote locations. OXCs are ideal for interconnecting streams of traffic from incoming lambda A on fiber B to outgoing lambda X on fiber Y. OXCs perform functions similar to those performed by SONET add/drop multiplexer and digital crossconnect combinations, but they do them at much higher data rates and with significantly greater ease of provisioning.
However, useful as OXCs are for interconnecting limited numbers of extremely large users, they are unable to provide the packet-processing functions required to create IP services, because they cannot recognize individual packet boundaries. Since OXCs are unable to buffer packets, they cannot provide the statistical multiplexing needed to efficiently provide direct connectivity between extremely large numbers of bursty users. Finally, OXCs support a moderately large, but still limited, number of very large pipes. For these reasons, optical-crossconnect technology alone is not sufficient to meet all the requirements of the new IP infrastructure. Wire-speed IP routers are required to provide the second part of the solution.
In their most rudimentary form, routers are packet switches that interconnect two networks. In addition to switching packets between networks, a router provides the packet-processing tools required for the creation of IP services. Packet processing requires that a router remove the data-link encapsulation from each packet, then inspect and potentially modify network-layer packet header fields. That function involves the traditional longest match route lookup, but might also include a number of specific packet classification functions.
Based on an examination of the input interface and packet header fields, a router can identify packets that need to receive specialized handling. Special packet handling can include rate limiting, prioritized delivery, statistics collection, packet filtering, and virtual-private-network classification. The fundamental packet-processing capabilities of routers allow service providers to create and deploy revenue-generating IP services. Because OXCs cannot recognize individual packet boundaries, they cannot perform the packet processing required to create IP services.
In addition to packet-processing functions, routers also support statistical multiplexing. The statistical multiplexing of traffic flows is essential to making extremely large IP networks operate efficiently. The basic idea behind statistical multiplexing is to combine numerous application flows on a given link but to consume network resources only for those flows that are actually transmitting data at any moment in time.
Because statistical multiplexing creates the likelihood that a particular link might occasionally be oversubscribed for a very short period of time, packet buffering must be available for each router interface. Since data applications are by their very nature bursty (they send a large volume of data for a short period of time, then send nothing for multiple seconds or minutes), a relatively small amount of packet buffering can deliver a significant gain in network operational efficiency. Technological advances in the foreseeable future will continue to support electronic but not photonic memory subsystems, so OXCs will not be able to provide the buffering capabilities required to perform the statistical multiplexing essential for efficient network operation.
As the IP infrastructure evolves to the new two-layer network architecture composed of IP/MPLS routers and DWDM/OXCs, two competing approaches are attempting to define how the remaining two equipment layers should interact-the overlay model and the peer model (see Figure 2).
Figure 2. The overlay model is supported by the Optical Domain Service Interconnect. Here, the Internet Protocol/Multiprotocol Label Switching (IP/MPLS) service and optical transmission layers are completely independent from a routing perspective. The interaction between the two layers occurs across a public user-to-network interface that is based on an entirely new signaling protocol that runs on top of the transmission control protocol. However, it has scaling limitations. In the peer model, the IP/MPLS layer functions as a peer of the optical transmission layer, such that a single routing protocol instance runs across both domains. The Internet Engineering Task Force supports this approach.
The overlay model is supported by the Optical Domain Service Interconnect (ODSI), an industry forum. In this approach, the IP/MPLS service and optical transmission layers are completely independent from a routing perspective. The interaction between the two layers occurs across a public user-to-network interface (UNI) that is based on an entirely new signaling protocol that runs on top of the transmission control protocol (TCP). This approach makes the optical transmission layer appear as a full set of n2 links to the routers in the IP/MPLS domain.
As providers learned from their experience with the deployment of IP-over-ATM networks in the 1990s, this approach creates a number of issues, including severely limiting the scalability of the IP-routed network. Additionally, the creation of two separate control planes makes it impossible for lambdas to make the most efficient use of the underlying physical topology of the optical transmission network.
The alternative approach is the peer model advocated by the Internet Engineering Task Force (IETF), an industry consortium. This approach is known as generalized Multiprotocol Label Switching (GMPLS) and based on extensions to traditional MPLS routing and signaling protocols. In the IETF approach, the IP/MPLS layer functions as a peer of the optical transmission layer such that a single routing protocol instance runs across both domains.
While the GMPLS peer model still requires a full n2 mesh of optical paths between edge routers to provide full connectivity, this mesh is used only to transmit user data. From a routing perspective, each GMPLS node exchanges routing information with a direct neighbor, and this feature allows the routing protocols to scale to support the construction of extremely large IP networks. The use of a fully integrated routing protocol also simplifies the provisioning of wavelengths across the optical domain, supports rapid automated recovery in the event of equipment failures, and makes more efficient use of underlying optical-network resources.
There is also a hybrid approach proposed by the Optical Internetworking Forum (OIF), an industry forum. In an effort to reduce time-to-market, the OIF is working to define a UNI based on MPLS signal standards: RSVP-TE and CR-LDP.
Last August, the OIF rejected a proposal to support the new signaling protocol developed by the ODSI as a third option for OIF UNI signaling. While the OIF is limiting its efforts to defining a UNI that operates using the overlay model, it offers carriers and service providers an extremely flexible approach for provisioning optical channels.
A provider can elect to use the OIF UNI for applications such as allowing subscribers to dynamically provision bandwidth across the core when the service provider does not want to expose the topology of its optical transmission network to the outside world. But when the service provider needs to dynamically provision trunk bandwidth across its own multivendor optical core, it can use GMPLS. Synergies exist between the OIF UNI and GMPLS, because they are based on the same set of extensions to traditional MPLS signaling and routing standards.
Traditional MPLS supports the forwarding of IP datagrams based on the value of an explicit label carried in a packet's "shim" header. This architecture requires that a label-switched router (LSR) has the ability to both recognize packet (or cell) boundaries and make a forwarding decision based on the contents of the packet (or cell) header.
GMPLS extends the traditional MPLS architecture to support a new class of LSRs whose forwarding plane does not recognize packet boundaries and cannot process packet headers. This new class of LSRs includes equipment that makes forwarding decisions based on an implicit label represented by a time slot (TDM), a wavelength (or lambda), or a physical port (fiber).
To support this new class of optical crossconnects, GMPLS extends traditional MPLS routing and signaling protocols, while adding new functionality. These changes influence the ways labels are requested and distributed, how bandwidth is allocated, the bidirectional nature of LSRs, and the mechanisms used to communicate network failures. GMPLS extends and updates traditional MPLS mechanisms in a number of areas:
- IGP Extensions. GMPLS uses interior gateway protocol (IGP) (intermediate system to intermediate system, or IS-IS, and open shortest path first, or OSPF) extensions to support the advertisement of various types of links into the link-state database-normal links, nonpacket links, and forwarding adjacencies. If nodes at both ends of a link can transmit and receive packets, the link is called a normal link. If neither node can multiplex or demultiplex individual packets, the link is referred to as a nonpacket link. If an LSR creates and maintains a label-switched path (LSP), it can announce the LSP into the IGP as a forwarding adjacency. Because the IGP floods information about forwarding adjacencies along with information about conventional links, an LSR can use forwarding adjacencies as well as regular links when it performs an LSP path calculation.
- LSP hierarchy. GMPLS defines an LSP hierarchy that allows the nesting of LSPs to support the establishment of traffic trunks, with the caveat that an LSP must always begin and end on similar equipment. At the bottom of this hierarchy are LSPs that begin and end at packet switching nodes, followed by LSPs that begin and end at TDM switching nodes, followed by LSPs that begin and end at lambda switching nodes. Finally, at the top are LSPs that begin and end at fiber switching nodes. Figure 3 illustrates how the GMPLS hierarchy supports the nesting of LSPs.
- Signaling extensions. After calculating the path for an LSP, the ingress LSR uses RSVP-TE or CR-LDP to establish label bindings along the path. The generalized label request object has been defined to allow the signaling protocols to support the establishment of LSPs based on packet, TDM, wavelength, or fiber labels. Additional signaling extensions allow a label to be suggested by an upstream node, allow the range of labels that might be selected by a downstream node to be restricted, and support bidirectional LSPs and rapid failure notification mechanisms.
- Link management protocol (LMP). LMP runs between adjacent nodes to simplify link management. It is responsible for establishing and maintaining control channel connectivity; verifying the physical connectivity of bearer channels; and rapidly identifying link, fiber, or channel failures within the optical domain.
- Link bundling. In some cases, a pair of LSRs might be connected by a number of parallel links or lambdas. To enhance traffic engineering scalability, GMPLS allows the set of parallel links to be advertised into the IGP as single link. All the component links in the "bundled link" must have the same link type, traffic engineering metric, and link multiplexing capabilities.
- Constraint-based routing. GMPLS distributes relevant transmission network topology information, including forwarding adjacency state, using IGP extensions. This information can be used by a constraint-based routing system to compute point-to-point paths across the optical transmission network. After the path is calculated, GMPLS relies on RSVP-TE or CR-LDP to establish label bindings along the specific path.
GMPLS provides an extremely powerful and flexible solution for signaling and routing in the new IP infrastructure:
- GMPLS's support for open standards allows carriers and service providers to select best-of-breed equipment as they continue to build out their networks.
- The peer model exposes the topology of the transmission network to IP routers, which allows IP routers to make more efficient use of optical-network resources when calculating the physical paths for LSPs.
- By removing the overlay model's requirement for an n2 mesh of optical channels to support the ex change of routing information, the peer approach enhances the scalability of IP routing.
- GMPLS leverages existing provider and vendor operational experience with MPLS traffic engineering.
- GMPLS eliminates the need to reinvent, test, and qualify an entirely new class of control protocols.
- Open standards promote the parallel evolution of UNI and network-to-network interface (NNI) standards so they can continue to support rapidly changing provider and subscriber requirements.
- GMPLS working in parallel with the OIF UNI gives providers the flexibility to deploy the proper tool for specific application needs.
- GMPLS makes possible the rapid development and deployment of a new class of optical crossconnects that facilitate provisioning on demand.
The exponential growth of the Internet, along with its commercial and social impact, is altering the fundamental relationship between Internet services and the underlying optical transmission network. The former attitude-that the Internet is simply one of many services that runs across the optical transmission network-needs to be changed. Today, the critical question that needs to be answered is, "What is the best way in terms of cost and operational efficiency for the optical transmission network to be designed to support IP router connectivity?"
The Internet changes the way that networks are built and operated. With the deployment of OC-48c/STM-16 and OC-192c/STM-64 interfaces, IP routers have become the largest consumers of bandwidth, so it is critical they have the ability to dynamically request and control the provisioning of capacity across the optical transmission domain. The deployment of overlay network architectures in the 1990s negatively affected the scalability of the IP network and led to the eventual development of MPLS as a traffic engineering tool.
It is essential that the lessons of the '90s not be forgotten and the GMPLS peer architecture be deployed so the Internet can continue to scale to meet the demands of its ever-expanding user community. Finally, the deployment of the GMPLS peer model creates an "open" NNI standard that supports interoperable and scalable parallel growth in the IP service and optical transmission domains.
Chuck Semeria is a technical consultant at Juniper Networks (Sunnyvale, CA).