Building the new carrier-grade optical Internet

Oct. 1, 2000

To migrate from overlaid networks to a single multiservice architecture, carriers will need a new edge routing platform.


As Internet Protocol (IP) traffic increases, the communications industry continues to invest heavily in high-capacity optical networks. This vast build-out and the availability of cost-effective IP switches to support an extremely efficient IP routing infrastructure are collapsing existing networks' multiple layers. It's no longer necessary to run IP over ATM over SONET over the optical layer. Instead IP over SONET, or directly over DWDM, eliminates this overhead. In the new network-also known as the optical Internet-IP/Multiprotocol Label Switching (MPLS) transport is emerging as a convergence layer that will improve performance and simplify the infrastructure.
Figure 1. A massive bandwidth supply is on the way. Carriers are making huge investments in their backbones in expectation of a great influx of data traffic.

Extensive deployment of DWDM and optical-switching technologies is providing the raw bandwidth that is making the optical Internet such a big opportunity. The bandwidth supply is expected to increase from 1.25 billion terabit-miles this year to 29.65 billion terabit-miles in 2003 (see Figure 1), according to Credit Suisse First Boston.

Service providers must focus on ways to utilize this abundant capacity and the flexibility of their IP/optical backbones as a common transport infrastructure for all network services. For services such as ATM, frame relay, and time-division multiplexing (TDM) private lines, the IP/optical backbone is being fortified with carrier-class quality-of-service (QoS) capabilities. Carrier-class IP networks must also attain the same SONET-grade levels of service availability that today's overlay networks exhibit. To this end, a new class of fault-tolerant edge routers is emerging to transform services for delivery on IP/MPLS networks.

By 2002, IP traffic will represent more than 90% of total public-network traffic, according to RHK Inc. (see Figure 2). While IP traffic is growing at an explosive rate, service-provider investments in optical technology and ultra-high-speed IP routers are growing faster than service revenue. Service providers will have invested more than $70 billion in core- routing and transmission equipment by 2003, reports RHK. Yet, revenues from IP services will only approach $25 billion in 2003, just 30% of the potential market.
Figure 2. To keep up with IP traffic growth, service providers must rapidly expand the capacity of their IP backbone networks.

This revenue shortfall needs to be rapidly supplemented. The most promising source of additional revenues will come from delivering traditional services such as ATM, frame relay, and TDM private-line services over these new IP networks. These traditional services are expected to earn $50 billion in revenues in 2003, reports RHK. By migrating these services to the IP infrastructure, service providers have an opportunity to address the full $75-billion market for ATM, frame relay, IP, and TDM services to bridge the revenue gap.

To take advantage of this revenue opportunity, service providers need to upgrade their IP/optical networks for the carriage of additional services by focusing on the following areas:

  • Fault tolerance to ensure services maintain a level of service availability equivalent to or better than what exists in overlay networks. In general, carriers need to achieve 99.999% (five-nines) or better availability.
  • Service transformation to provide the adaptation of non-IP services for IP/MPLS transport, while maintaining the integrity of service-specific characteristics such as alarms, timing, and transparency to the subscriber.
  • QoS to accommodate the specific requirements of each traffic type in terms of traffic prioritization, jitter, and propagation delay.
  • Traffic engineering to ease service provisioning for multiple types of services across the flexible, connectionless IP environment and support network-wide QoS.

The optical Internet is designed to overcome the scalability limitations of traditional overlay networks. To deliver ATM, frame relay, leased line, and IP services today, carriers typically have to operate four separate networks:

  • An IP network based on IP routers to deliver Layer 3 IP services.
  • A switched network to provide Layer 2 services such as ATM and frame relay.
  • A TDM network to transport and groom Layer 1 leased-line services.
  • An optical network to provide capacity and bandwidth management.
Figure 3. Today's overlay service networks add management complexity and limit scalability.

With traditional overlay networks, service providers will face tough challenges in managing complexity and scaling capacity to meet future demands (see Figure 3). Investment in high-capacity ATM switching technology is already dwindling as vendors focus on developing new IP/ MPLS switch/routers.

The optical Internet architecture brings carriers much-needed scalability, and it promises to significantly reduce operating costs. This next-generation network collapses the traditional stack of four overlay networks into a simplified, two-layer architecture based on an optical layer that delivers transport capa city and a converged IP/ MPLS services layer (see Figure 4).
Figure 4. The optical Internet employs an Internet Protocol/Multiprotocol Label Switching services layer and an optical transport layer, which provides high-bandwidth capacity.

The IP/MPLS core consists of ultra-high-capacity switching/ routing devices from companies such as Avici Systems, Cisco Systems, and Juniper Networks. The development of these core gigabit and terabit IP/MPLS switch/routers continues to focus on increasing the delivery rate of packets and enhancing the QoS capabilities needed for multiservice IP-based transport.

Aggregation service routers are a new class of edge device designed to terminate a large volume of multiservice traffic (ATM, frame relay, IP, and TDM) and transform it for routing over the IP/MPLS core. Located at the carrier's point-of-presence (POP) or super-POP, these next-generation devices replace traditional IP edge routers to give the optical Internet a scalable, high-performance multiservice capability.

Within the aggregation service router, ATM, frame relay, and TDM services are transformed into IP packets for transmission over the IP/MPLS backbone. To adapt these services, the aggregation service router performs wire-speed packet processing to classify and transform the traffic in real time, even on high-speed optical links. Using emulation, non-IP packets, circuits, and virtual circuits are encapsulated within IP streams that are tunneled across IP or IP/MPLS networks.

Carrier-class aggregation service routers must support a vast number of connections and subscribers, while measuring up to the five-nines of service availability evidenced in the public-switched telephone network.

A robust optical transport layer provides the required high-speed network capacity. At the edges, the optical access network consists primarily of SONET/ SDH add/drop multiplexers and an emerging class of multiservice provisioning platforms (MSPPs) that may also integrate DWDM technology. The function of an MSPP is to groom service feeds into optical payloads with the emphasis on payload efficiency; MSPPs have limited service intelligence. The optical edge platforms, in turn, connect to the metropolitan/interoffice infrastructure or the optical core.

The optical core consists of optical crossconnect systems and long-haul DWDM systems. Within the optical core, the emphasis is on low-cost transport, payload efficiency, and efficiency of bandwidth management.

Carriers have built fault tolerance into their traditional telephone networks using redundant facilities and fault-tolerant telephony switches. Dial tone is a prime example of maximum network reliability in voice networks. For the optical Internet to become the backbone for the carriage of all services, including mission-critical services such as private leased lines, every element of the architecture must be resilient. It must achieve the same level of fault tolerance that exists in today's voice network.

In the rush to expand their IP networks, carriers have deployed enterprise-grade routing equipment. These products are not particularly fault-tolerant. Traditional IP edge routers represent a single point-of-service failure. However, with IP services in such demand, service providers have accepted these limitations.

When an enterprise-class router loses its primary control circuitry and operation falls back to its redundant controller, the impact on end users is typically an outage of 5-15 minutes, the time it takes to re-converge at the IP layer. Restoring the service requires synchronizing the routing states and packet-forwarding tables to reflect the status of the service before the failure. Thus, the failure of a single router can devastate service availability. As IP-based services become mission-critical, several-minute outages are no longer acceptable.

If the optical Internet is to succeed, fault tolerance must be brought to the service edge. A new class of fault tolerant routers is needed to provide protection at the service layer and support redundancy of routing states and tables. This capability is essential to restore services with SONET-like speed of less than 50 msec to eliminate a point of failure (see Table 1).

As carriers migrate to the optical Internet architecture, the new aggregation service routers will address many of the limitations of existing IP routers. It is designed for multiservice transport and providing protection at the service layer (see Table 2).

Bringing resiliency to IP routers is not a trivial task. It requires a ground-up approach to router system design, including a resilient hardware architecture and fault-tolerant operating system that together provide a fast and reliable data infrastructure. Next-generation operating systems must allow routing information to be dynamically mirrored among multiple router control elements.

Alex Dobrushin is vice president of marketing and Danny McPherson is director of architecture at Amber Networks (Santa Clara, CA). For more information, visit the company's Website at www.ambernet