Packet-aware access networks: Flexible MPLS branches out
Carriers are looking for ways to lower the costs and improve the efficiencies of access networks. As data services continue to grow, the current practice of mapping data traffic onto TDM circuits is becoming less practical. To improve the efficiency of their access networks, carriers would like to take advantage of statistical multiplexing to merge data traffic streams together. However, the current generation of SONET/SDH transport and switching equipment lacks the packet-awareness to perform the functions required.
The current mode of operation is costly for providers because of the growth in data traffic. The high amount of data traffic in a traditional SONET/SDH access network consumes large numbers of TDM circuits, leading to the premature deployment of more transport gear despite the fact that lines are lightly utilized. Traditional SONET/SDH equipment takes traffic coming from customer premises equipment and maps that traffic onto TDM circuits at various speeds (NxDS-0, T1/E1, DS-3/E3, OC-3/STM-1, etc.). When mapping traffic onto these circuits, the entire circuit bandwidth is always dedicated to the customer on the SONET ring, despite the customer often using a small portion or none of the bandwidth. The dedication of bandwidth to these circuits can also be thought of as the transport of idle traffic (idle cells and frames).
Adding to the high cost of this architecture is inefficient use of interfaces on edge data equipment such as IP routers and ATM switches. This equipment connects to the SONET/SDH gear via discrete or channelized interfaces to extract the data traffic and deliver the required service. Since the TDM circuits are underutilized, the edge-router/switch ports are underutilized and an excess number of ports are required. The net result is "stranded" bandwidth.
Carriers are turning to MPLS for a solution to this problem. MPLS has undergone quite an evolution from its initial role of providing traffic engineering, to delivering SONET-like recovery from network failures and now acting as a unifying technology for the delivery of Layer 2 and Layer 3 data services over provider backbones. MPLS is once again poised to prove its utility in a new capacity: packetizing the TDM access network.
Many carriers have deployed or have plans to deploy MPLS in the backbone to offer a variety of data services. That makes it a natural candidate to extend into the access to increase efficiency and lower cost. There is another trait of MPLS that makes it very suitable for packetizing the access network. Like SONET and SDH, it can carry virtually any type of traffic.
The role of MPLS as a unifying technology for the transport of various data types was born from standards work in the Pseudo-wire Emulation Edge to Edge (PWE3) working group of the Internet Engineering Task Force (IETF). PWE3 (also known in the industry as the "Martini draft") enables various traffic types to be mapped onto MPLS virtual circuits (VCs) or label-switched paths, including ATM, Frame Relay (FR), Ethernet, point-to-point protocol/high-level data link control (PPP/HDLC), and TDM circuits via circuit emulation.
Of course, MPLS is not the first technology to be considered for such a purpose. Other data-switching technologies exist to provide statistical multiplexing gains in the access network. However, they are not as well suited as MPLS for the following reasons:
- ATM is a popular Layer 2 aggregation technology because the properties of ATM cell switching make it well suited to deliver high-quality voice along with data over low-speed circuits. While ATM offers high levels of quality, it lacks efficiency and performance. Data traffic, especially IP, does not map very efficiently into ATM cells. But the most critical problem with ATM is the lack of high-speed interfaces on IP routers to tie into the access network. Although at least one vendor has introduced IP-routed ATM interfaces as high as OC-48 (2.5 Gbits/sec), interfaces typically max out at OC-12 (622 Mbits/sec). In contrast, packet over SONET and Ethernet interfaces (which can both carry MPLS) on routers support up to OC-192 (10-Gbit/sec) rates.
- FR was one of the first data link layer technologies to be used for aggregation. However, it does not have a widely supported standard for signaling and nearly all FR switching equipment has an upper limit of DS-3 (44.736-Mbit/sec) interface rates.
- Ethernet has the capacity and efficiency to packetize the access network, and in some cases, carriers are starting to deploy it for this purpose. However, Ethernet is limited in its ability to carry different types of traffic. It can handle IP but lacks the ability to transport legacy ATM and FR traffic. Furthermore, Ethernet still lacks the operation, administration, and maintenance capabilities that carriers require in their access networks.
Pushing MPLS into the access network to take advantage of statistical multiplexing will require next-generation access equipment. Most commonly, that will be next-generation SONET/
SDH gear with packet awareness. Services like private line and circuit-switched voice still exist and will continue to do so for some time, so they still need to be supported. These requirements make next-generation SONET/SDH equipment, which can provide statistical multiplexing for data traffic while still supporting circuit-switching, the logical choice. The required next-generation systems may come from an entirely new platform or in the form of new cards and software functionality on an existing add/drop multiplexer (ADM). This next-generation device is called a packet-aware multiplexer (PAM).
The topology of the access network will not change significantly. Because of existing fiber routes, a ring topology is most common with PAMs taking the place of traditional ADMs along the metro SONET ring. Aggregating the traffic from various PAMs across multiple rings will be the responsibility of a larger PAM device called the PAM concentrator. The PAM concentrator will aggregate MPLS and circuit-switched traffic from the various rings and direct private lines to an optical-switching device or long-haul transport equipment, while directing the data traffic to an edge switch/router or multiservice edge (MSE) device. The MSE device combines the functions of an edge router and switch into a single platform to enable convergence.
MPLS in the access network will extend from the MSE to PAM devices. The MSE devices may already be configured to run MPLS in the backbone network. For purposes of scalability and service uniformity, it is suggested that the backbone and access networks be treated as separate MPLS domains. The MSE will act as the gateway between these separate MPLS domains. This separation is logical since MPLS's purpose in each case is unique. In the access network, it is acting as a transport technology to add statistical multiplexing and efficiency. In the backbone network, it is acting as a service delivery vehicle for data services, in addition to performing traffic engineering. As the gateway between the MPLS domains, the MSE device becomes the common service delivery platform. In this topology, services will still originate on the MSE not the PAM device. That way, services can be offered in a consistent manner to customers regardless of how they attach to the network.
The MSE device is the only choice for the delivery of certain services such as IP-based virtual private networks (VPNs), which require not just packet awareness, but IP awareness as well, especially in cases where customers may still be connecting directly to the MSE device over legacy TDM circuits. If services are also offered on the PAM, then inconsistencies in the functionality or quality that customers experience is likely.
Scalability is the other key reason for keeping the MPLS domains separate. If one large MPLS domain were used, PAMs would require a complete view of the MPLS topology to provide end-to-end services. A single MPLS domain would be very large, given the hundreds of PAMs likely to exist in a large carrier network. But PAMs are not designed with the processing power and routing scalability of MSE devices since their purpose is to provide efficient transport at low cost.
The access MPLS domain will carry Martini VCs (PWE3 connections) between PAM and MSE devices. The PAM will take an individual customer interface such as a T1 or DS-3 and map it to a Martini VC. There will be a one-to-one relationship between a customer T1, for example, and the Martini VC. This connection originates at the PAM and terminates at the MSE device (see the Figure). The mapping of traffic into the Martini VC is done in accordance with the PWE3/Martini standards. There are specific instructions covered in those standards on how to encapsulate traffic into MPLS in a transparent manner, such that an ATM service, for example, behaves like ATM, not a compromised form of ATM. While performing the encapsulation of traffic onto Martini VCs, the PAM also removes all the idle cells and frames to preserve bandwidth. That is the essence of the improvements gained by running MPLS in the access. Instead of dedicating wasted bandwidth to a customer circuit, bandwidth is only allocated when needed in the access network.
The individual Martini VCs will be transparent to devices between the originating PAM and the MSE device, such as other PAMs and the PAM concentrator. Initial implementations of this technology will consist of statically configured Martini VCs between the PAM and MSE devices. As the technology evolves, dynamic signaling will be added to accelerate provisioning and provide network-layer resiliency in the access network. The signaling will be based on the label distribution protocol (LDP), which is required in the PWE3 standards. Optionally, the LDP may run over resource reservation protocol-traffic engineering (RSVP-TE), another MPLS signaling protocol that adds traffic engineering and fast reroute capabilities.
The MSE device takes all the Martini VCs and terminates them, just as it would have the individual channels on a channelized SONET interface. The MSE device treats the Martini VC like a directly attached customer interface. The interface acts as a user-network interface between the MSE device and customer premises equipment (CPE), since the access network transparently passes the underlying customer data. The MSE device will remove all the encapsulating headers of the Martini VC, including the MPLS label stack. What's left is the customer traffic that originated from the CPE device. At this point, the MSE device can deliver any data services required by that customer, just as if it were directly connected. The services might consist of a point-to-point switched data service, Internet access service, or IP VPNs.
Because dedicated bandwidth does not exist between the CPE and MSE device, there is the possibility for congestion in the access network. However, not all customers will be using their maximum amount of bandwidth at once. Some customers' traffic will be at low usage rates while others will be high, so carriers can still deliver all the aggregate bandwidth with a certain degree of probability. Carriers can adjust how much they choose to oversubscribe customer traffic in their network to find the right balance between efficiency and congestion avoidance. At some point congestion will occur, in which case it will be important to have the right quality of service (QoS) capabilities in place to ensure important traffic is preserved at the expense of best-effort services.
Another important aspect of packetizing the access network is the method by which MPLS is carried over SONET/SDH. A few different options exist, each with advantages and disadvantages; the appropriate data link layer and framing procedure is a debate all its own. One promising packet-based ring technology that can be used between the PAM and PAM concentrator is resilient packet rings (RPRs) as standardized in IEEE 802.17. RPR can transport MPLS traffic, provides a foundation for statistical multiplexing, and has the QoS capabilities to manage bandwidth during congestion. Between the PAM concentrator and the MSE device, Ethernet or some HDLC-based encapsulation may be used; MPLS can run over either of those. An emerging alternative to HDLC is generic framing procedure (GFP), which eliminates the byte-stuffing problem posed by HDLC and can serve as a framing protocol for a wider variety of data link layer encapsulations, including RPR. Often mentioned in the same breath as GFP is virtual concatenation, which together will serve to adapt the now efficiently packed data streams into SONET/SDH.
Carriers face a common dilemma: How do they handle the rapid growth in data traffic over a transport network designed to handle private line and circuit-switched voice? The time is ripe for a new architecture that adds packet intelligence to the access network and allows carriers to get the most from the bandwidth already deployed.
MPLS has proved to be flexible enough to serve in this new role. A new generation of SONET/SDH equipment is emerging that adds packet awareness and performs statistical multiplexing in the access network via MPLS in conjunction with other link layer technologies. This new architecture paves the way for efficiency and convergence in the access network and allows carriers to greatly reduce their costs. Equipment vendors are already in trials, demonstrating this technology to carriers. Look for early adopters of this technology to start deployments sometime next year.
Rafael Francis is director of product management at Laurel Networks (Pittsburgh).