Gigabit Ethernet migrates from local to wide networks

Scaling from 10 through 100Mbit/s to Gigabit date rates and from copper to fibre has enabled Ethernet's extension from local-area to metro-area networks.
Fibre-only 10 Gigabit Ethernet promises both an extension to wide-area networks and an interface with the 10Gbit/s SONET/SDH backbone.

By Mark Telford

Since its introduction as 10Mbit/s 10Base-T via the IEEE 802.3 standard in 1980, Ethernet has been adopted as a simple, low-cost and data-optimised access technology for enterprise infrastructure, so that it now comprises 98% of local area network (LAN) connections.

Growth has been driven by increasingly powerful PCs, multimedia applications and networked client/server computers, which requires transfer of not only data but also video, voice and graphics in greater quantities and speeds. In particular, LANs can be scaled up for high-bandwidth applications via an all-Ethernet enterprise environment.

From shared media to packet switching
To increase the data rate to 100Mbit/s, the Fast Ethernet Alliance was formed in July 1993 to draft an extension of 10BaseT. Approved in June 1995, the IEEE's 802.3u 100BaseT specification includes:

• the same Media Access Control (MAC) client interface layer, based on the same Carrier Sense Multiple Access with Collision Detection (CSMA/CD) "listen-before-you-send" transmission protocol and error control functions as well as the same frame format and length and therefore cabling;
• a new Media Independent Interface (MII) sublayer between the MAC layer; and
• Physical Layers (100 Base-TX and 100BaseT4 for cable; 100Base-FX for fibre, used mainly to connect hubs and switches, between either wiring closets or buildings).

To support 100Mbit/s traffic, Ethernet shifted from shared media to switching, and borrowed physical layer (PHY) functions from ANSI's X3T9.5 Fibre Distributed Data Interface (FDDI), a Token Ring architecture which uses packet switching (though FDDI-II allows for circuit switching).

But the single-collision domain length of CSMA/CD limits copper cable runs between signal re-generation to 250m for 100Base-T, down from 2.5km for 10BaseT. This hinders Fast Ethernet's ability to form a backbone and is insufficient for medium-to-large workgroups. Networks can have Ethernet at each end, but connection is via a multi-layered complex of legacy MAN and WAN. However, 100Base-FX fibre allows 2km transmission in FDDI between wiring closets and campus buildings, while 10Base-FL fibre can be used for connecting to the wiring closet.

Gigabit & fibre-only 10 Gigabit Ethernet
In the 1990s Asynchronous Transfer Mode (ATM) was seen as the only technology able to span both the LAN and the WAN as an end-to-end, standards-based solution creating a single network delivering voice, video and data to desktops. But ATM was delayed in standardisation and product development. Compared to ATM, Ethernet allows software-controlled provisioning, involving no "truck rolls", muxing, or translations to ATM or any other legacy technology. Also, end-to-end Ethernet needs no expensive bandwidth-consuming conversion between Ethernet packets and ATM cells.

In July 1996 the IEEE 802.3z Task Force started developing a compatible standard for 1000Mbit/s Gigabit Ethernet (GbE). Approved in June 1998, this borrowed the PHY from the ANSI's X3T11 Fibre Channel physical medium dependent (PMD) specification.

This allows scaling from 10Mbit/s Ethernet switched at the desktop connected to 100Mbit/s Fast Ethernet up the riser to 1000Mbit/s Gigabit Ethernet in a data centre.

GbE also includes enhancements to Ethernet's media access control (MAC), the physical coding sublayer (PCS), optional interfaces (XGMII, XAUI, and XSBI), management and the optical specification for various PMD interfaces.

Intel Communication Group's marketing director Anthony Ambrose says there is a "strong move to GbE in 2002 for enterprise". Correspondingly, Intel's strategy is to reduce the cost of deployment by 80% in two years through component integration. In February 2002, Intel launched the first single-chip GbE products, including a dual-port controller (to combine two GbE channels) and a controller for desktops. Also, at March's CeBIT show Lumentis launched a one-slot dual GbE transponder.

Intel's LAN Access Division director Tim Dunn says that enterprise applications will be first. These include aggregation of multiple GbE links in a data centre, connecting large-capacity 1000Base-T workgroup switches (GbE over copper) on the LAN backbone, switch-to-switch connectivity across a large campus, and very-short-reach links between the electronic switches, routers and optical networking equipment inside points of presence (POPs).

But within the PHY are several layers for the PCS and PMD sublayer for fibre media. 10GbE includes not just a PMD for a reach of 100-300m on multimode fibre but also two serial PMD sublayers for reach of 10 and 40km on single-mode fibre at 1310 and 1550nm, respectively. So, 10GbE can also be a carrier technology for metro-area service providers by aggregating LAN Gigabit streams from user buildings and between inter-metro POPs via a backbone switch, putting 10Gbit/s onto long-haul fibre with a range similar to GbE.

Greenfield ventures can lease dark fibre and build their own metro-area networks, by-passing existing SONET and ATM infrastructures. For example, 10GbE enables the replacement of networks that use ATM switches and SONET multiplexers on a 2.5Gbit/s OC-48 SONET ring with simpler, low-cost, metro-sized, fibre-based Ethernet networks with Layer 3 and Layer 4 switches and 10GbE backbones (see Figure 1).

Also, 10GbE can be used for wide-area network connectivity. All-optical Ethernet can therefore integrate not just LAN and MAN but also WAN, using Ethernet for end-to-end Layer 2 transport.

A problem is that Ethernet alone does not have the built-in network management features needed for MAN and WAN such as fault-tolerance, security and Quality of Service (QoS).

However, combining Ethernet with the network management capabilities of IP layer routing protocols can offer QoS and traffic policing approaching those of ATM.

Interface with SONET/SDH
Nevertheless, due to the telecoms downturn and the shift from failing new operators back to cash-strapped incumbents who want to maximise use of expensive legacy optical network infrastructure, Ethernet still needs an interface which allows it to interconnect easily and at lower cost with SONET and SDH.

Existing networks requiring more bandwidth can be upgraded in a staged migration by:
1. encapsulating Ethernet within a SONET wrapper for end-to-end communication via dark wavelengths over a DWDM system;
2. running native 10GbE over all-optical networks, lighting a different fibre for each bitstream for voice, text and video.

Carrier networks can use hybrid platforms which integrate Ethernet functionality with SDH/SONET, DWDM and optical networking capability to deliver Gigabit Ethernet services within access and metro infrastructures. This reduces conversion between data transport protocols, simplify network management and expand capacity. Interoperability is promoted by the 10 Gigabit Ethernet Alliance, formed by 3Com, Cisco Systems, Extreme Networks, Intel, Nortel Networks, Sun Microsystems, and World Wide Packets in January 2000.

So, as well as a LAN PHY working at 10Gbit/s, 10GbE also has a WAN PHY which includes a SONET/SDH-type digital wrapper around the Ethernet frames with management bytes in the frame overhead. The compatible data rate therefore enables Ethernet to interface with the WAN infrastructure of 9.953281Gbit/s SONET OC-192/SDH VC-4-64c circuit-switched gear.

The small difference between 10GbE and SONET/SDH line rates eases implementation of a MAC address that can operate with both the LAN and WAN PHYs. The WAN PHY therefore allows 10GbE switches and routers to attach to SONET/SDH access equipment, enabling Ethernet to extend over the SONET backbone infrastructure for Layer 1 transport.

Outlook for Ethernet-based WAN
Due to Ethernet's flexibility and variety of optical interfaces, the operation and cost of 10GbE can be optimised to allow construction of MANs and WANs that connect geographically dispersed LANs between campuses or PoPs using either dark fibre, dark wavelengths or existing SONET/SDH networks.

The development of WAN-compatible 10GbE by marrying simple, ubiquitous Ethernet with fast and reliable optical networks also enables bandwidth growth, service differentiation and reliable "future proofed" networks.

Consequently, market research company Gartner forecasts that, by 2005, over 30% of high-speed WAN data services will be delivered over Ethernet, with the greatest growth in 10GbE (323%), first-mile (195%), storage-area networks (50%) and Media Switch (50%). Meanwhile, Synergy Research Group expects the 10GbE market to grow to USD103m in 2002 and USD420m by 2006.

Carriers are already experiencing demand for 10GbE for large enterprise customers. In response, many network and module suppliers have announced pre-standard 10GbE products.

Nortel Networks has hardened its Ethernet offerings for MAN and WAN by combining it with optical Ethernet (Ethernet over Fibre, Ethernet over Resilient Packet Rings (RPR), Ethernet over DWDM, and Ethernet over SDH). Ethernet over Fibre (EoF) can span up to 70km. It is mainly deployed in a point-to-point or mesh network topology, and delivers packet services over dark fibre, typically in the MAN.

Cisco is also developing 10GbE products. "Merchant semiconductor providers will offer SONET framers, WIS (WAN interface sub-layer) in silicon so we can build it into our systems," says product manager Bruce Tolley, who is also VP of the 10 Gigabit Ethernet Alliance and a Gigabit Ethernet Task Force member.

Cisco says that its Catalyst 6500 Series 10GBASE-LR and 10GBASE-EX4 Metro 10GbE modules extend Ethernet from the LAN to the MAN while protecting existing investments in fibre infrastructure. By supporting high-bandwidth connections with intelligent network services such as availability, security, and manageability, they enable new applications such as server-less buildings, data centre remote mirroring, disaster recovery, E-learning, imaging/3D modelling, as well as high-bandwidth inter-POP connections. Modules are compatible with the 7600 Series Internet Router and support its Architecture for Voice, Video and Integrated Data (AVVID).

Cisco has been shipping 10GbE modules since last year, deployed in late 2001 by Curtin University of Technology in Australia, Kyoto University in Japan and Arkansas State University for scaling Internet Protocol infrastructure and connecting its campuses over metro distances for storage-area networks.

Also offering 10GbE products for the enterprise environment is Enterasys Networks.

In addition, the XENPAK Multi-Source Agreement for interoperable 10GbE transceiver modules, instigated by Agilent Technologies and Agere Systems in March 2001 (www.xenpak.org), provides a hot-pluggable module with an SC fibre connector and 10 Gigabit attachment unit interface (XAUI).

In January, Agere started sampling the AE20LRCAA XENPAK-compliant 10GbE Serial Transceiver for MAN, optimised for 10km of single-mode fibre, for volume shipping in 2003. This will help speed the implementation of 10GbE applications into areas traditionally served by SONET/SDH-based OC-192 networks. "As Ethernet becomes faster and SONET becomes less costly and more versatile for data traffic, the two protocols are evolving as viable solutions for metro access-based applications," says Agere.

In March, Sweden's Optillion announced the first system-level interoperability demonstrations between a fully functional 10GbE product and a XENPAK transceiver (its 10GBase-LR module, pictured left, which includes the TOP 3010 for up to 10km and TOP 5010 for up to 40km).