Creating the borderless enterprise with metro Ethernet services
The possibility of Ethernet outside the enterprise caught the attention of many a few years ago when several startup carriers, dubbed ELECs (Ethernet local-exchange carriers), began to loudly market their services. Yipes Communications, Telseon, and Cogent Communications entered a number of metropolitan markets and offered to sell Ethernet over fiber connections ranging from 1 Mbit/sec to 1 Gbit/sec to business customers.
These Ethernet services were pitched as low-cost and more flexible than T1 (1.55-Mbit/sec) and T3 (45-Mbit/sec) lines; they eliminated the need for enterprise conversion from IP and SONET protocols and could offer incremental bandwidth upgrades as low as 1 Mbit/sec without requiring long lead times and additional hardware.
"It opened a lot of corporations' eyes," says Michael Howard, principal analyst and co-founder of Infonetics Research (San Jose, CA). A 2002 Infonetics study of 80 U.S. companies with optical connections to carriers indicated that 85% wanted Ethernet services.
In its native form, Ethernet is a proven (25-year-old) asynchronous frame-based data-networking technology that 90% of corporations run over their enterprise networks. In addition to the large enterprise installed base, the technology is viewed as a low-cost means of offering scalable, incremental bandwidth using 10/100/1,000-Mbit/sec standardized interfaces, which are widespread on both telecommunications and enterprise networking equipment. Last June, 10-Gigabit Ethernet (GbE) was ratified as a standard (IEEE 802.3) by the Institute of Electrical and Electronics Engineers.
Nortel Networks (Ottawa, Ontario), a dominant supplier of optical Ethernet equipment to service providers (WorldCom, AT&T, China Netcom, and Sprint), announced more than 20 new customers in the last year, including several enterprise customers, reports Jim Dondero, Nortel Optical Ethernet Solutions marketing director. "We are able to learn quite a bit about what is happening in the enterprise space," he says, "via the enterprises that are actually deploying optical Ethernet in a private build scenario. What some of their issues are in terms of the need to centralize servers and applications, extended LAN environments, deploying new higher-bandwidth applications, and the flexibility to reduce costs across the networks by using a technology that is much more flexible and easier to manage."
Pitched as a low-cost revelation in 2000, Ethernet services in some form were already being sold, albeit quietly, by most large incumbent carriers to satisfy individual customers' demands. BellSouth has provided dedicated point-to-point Ethernet connections, also known as private line Ethernet, since 1994. Many other carriers provided Ethernet services to customers, although Ethernet did not appear on the "official" service list.
Today, corporate customer retention is still the major driving force behind carriers' moves toward actively marketing and more aggressively rolling out Ethernet services. For the incumbents, the fear of cannibalizing their existing frame relay (FR) and T1 services remains, but losing large corporate accounts to competitors is even more risky. As data traffic increases within corporations and in public networks, most carriers agree that Ethernet outside the enterprise shows promise.
"There are some opportunities with Ethernet that you don't have with the standard T1 lines or E1 lines," explains Infonetics's Howard. If the technology for metering bandwidth is in place, Ethernet gives service providers the opportunity to sell different amounts of bandwidth based on fluctuating requirements.
Today, Ethernet over public networks is most often used by large enterprises as a dedicated private line to the carrier's point of presence for Internet access or as a dedicated point-to-point service connecting two enterprise locations such as SBC Communications' GigaMAN offering. These Ethernet connections typically replace TDM-based T1/E1, T3/E3, ATM, or FR links. Ethernet virtual private line services over shared bandwidth are also available. In North America, about 100,000 optical Ethernet connections from carriers' points of presence to buildings exist today, although a single building may support several connections, according to Infonetics.
An emerging service category is Ethernet virtual private networks (VPNs), also known as transparent LANs. Here private networks are extended across shared bandwidth on the metro network using virtual tunnels to connect multiple sites. Few carriers use Ethernet to offer these transparent LAN services today.
"One of the key issues that we think is important for driving metro Ethernet deployment is that the VPN services interwork with the existing frame relay and ATM VPN services," says Keith Allen, assistant vice president of strategy and solutions for Alcatel's Fixed Networks division (Paris). "We see the metro Ethernet services as a complement to the legacy services, allowing the service providers' revenues to increase by up-selling some of the existing services to a higher bandwidth capacity that is better served with an Ethernet access solution."
In April, Alcatel introduced a software upgrade to its 7670 routing switch platform that supports service internetworking between existing FR services and emerging Ethernet VPNs. For instance, a large corporation could upgrade its headquarters to high-speed Ethernet-based Internet access but still allow its smaller branch offices to stay at the lower-speed FR-based Internet access. That enables a simpler migration to Ethernet services and allows the service provider to get more revenue by selling more bandwidth at the head office. The Ethernet services could be priced at a lower cost per bit but higher monthly charge than FR services.
"As far as uptake, with a lot of the services that are deployed today, they don't provide a service-level assurance that is on parity with the existing frame relay services," notes Allen. "And that is a hindrance, especially in attracting Fortune 1000 companies to adopt the service or for any mission-critical application to be run over that service. As a vendor, we have to make any Ethernet solution truly carrier class to allow the service provider to offer a higher level of SLA [service-level agreement]."
There's no doubt that inherent limitations in native Ethernet, developed for enterprise-level data networks, present many technical challenges to manufacturers and service providers interested in rolling it out as a carrier class transport layer in service providers' networks. Many standard characteristics of carriers' networks today—five-nines reliability; less than 50-msec protection; end-to-end service management and quality of service; class of service; fault isolation; in-service performance monitoring; and basic operations, administration, and maintenance—are not supported by native Ethernet technology. Moreover, native Ethernet's utilization of network resources limits its scalability in larger networks.
The result is a variety of proposed technical implementations and workarounds that industry groups and standards bodies are actively pursuing with the help of carriers' input about their needs. The Metro Ethernet Forum (MEF), founded two years ago, is an industry organization comprising equipment manufacturers and service providers that is working to define carrier class Ethernet technical specifications as well as service definitions for Ethernet VPN and Ethernet private line services. The group expects to finalize its service definitions by August.
"If there are two connections from a carrier to a building, Ethernet itself does not have any capability to know that one is the primary and the other is the backup," explains Infonetics's Howard. "The MEF is helping to define a standard way using MPLS to have resilient connections."
The MEF is also making inroads with the carrier community. In November, MEF established a relationship with the International Telecommunication Union (ITU), an international standards organization. In December, Verizon Communications and Rockwell became forum members, boosting service-provider membership, which already included BellSouth, France Telecom, SBC, Sprint, and Utfors.
Today, Ethernet over existing infrastructures such as SONET/SDH can meet large corporations' needs, but it may not scale adequately or make financial sense for carriers over time to satisfy a broader audience of small- and medium-business customers that require lower-cost structures for services. Ethernet over WDM is also emerging. Many WDM equipment makers (CWDM and DWDM) are adding Ethernet interfaces to their systems. For service providers that already have WDM gear in place, it is relatively easy to add an Ethernet interface and use Ethernet over a wavelength for applications such as storage-area networking.
Last year, Ethernet over SONET/SDH (standard and resilient packet ring) accounted for 21% of the total $2.5 billion in worldwide metro Ethernet equipment revenue, according to Infonetics. Nortel and Cisco Systems were the market leaders for Ethernet interfaces on SONET/SDH equipment, even though Cisco just started to sell SDH systems in 2002.
The $2.5 billion in worldwide metro Ethernet equipment revenues includes customer and building premises equipment (44%) and metro core and edge equipment (56%). Ethernet switches and routers with Ethernet interfaces made up 66% of total metro Ethernet equipment revenue worldwide in 2002 (see Figure 1). Although this equipment market is still emerging, strong growth is expected. By 2006, the total worldwide metro Ethernet equipment market is forecast to increase 131% to $5.7 billion, a compound annual growth rate of 23%.
"One of the factors for our forecast for metro Ethernet equipment is that carrier class Ethernet switching gear is just coming to market," says Infonetics's Howard. Although some of these switches were available as early as 2000, there is a surge of activity right now. Ethernet switches from companies such as Cisco, Extreme Networks, Foundry Networks, Riverstone Networks, Atrica, and Luminous are built expressly for the carrier environment. As such, the switches offer NEBS Level 3 compliance, multiple power supplies, and other characteristics important to resiliency and service redundancy in carriers' networks. Some companies like Cisco are also integrating Ethernet switching capabilities into their SONET/SDH add/drop multiplexers.
In April, Foundry Networks introduced new silicon and a new terabit architecture described by the company as the first 40-GbE-ready switching platform. In a 7-ft rack (three individual systems), Foundry can offer 4 Tbits/sec of switching capacity and 1.5-billion-packet/sec throughput. The new platform uses Terathon, Foundry's fourth-generation ASIC, which has 80 million transistors—twice that of current Intel Pentium IV chips. The first two products based on the new platform, the BigIron MG8 and NetIron 40G, are aimed at enterprises and service providers. Improvements in silicon and optics pricing have allowed Foundry to offer a low-cost per 10-GbE port for both of the new platforms, according to company spokeswoman Jill Shanks.
How are carriers going to use this emerging carrier class Ethernet equipment and where is it going to fit in their evolving networks? That is an area of ongoing debate. Manufacturers say they will work with carriers to assess their existing infrastructures, Ethernet service demand, and service reach requirements to determine the optimum technology solutions. Carriers, meanwhile, are meeting their immediate Ethernet service needs over their existing infrastructures and questioning what the future holds.
"We talk to a lot of large carriers and our take is that many of them have the strategy today of satisfying their large business customers with Ethernet running on their SONET networks or in Europe on their SDH networks," says Howard. "When those carriers are looking out in the future at providing Ethernet services on a wider basis to medium-sized companies and even farther out to small companies, the business model for Ethernet over SONET/SDH does not work—it is not profitable. So many carriers are looking at an Ethernet overlay network or a separate Ethernet network in the metro area to handle those kinds of customers. These metro Ethernet networks would connect into their SONET-based or DWDM-based wide-area networks."
Part of their reasoning is that medium- and small-business customers typically cannot absorb the same network-service costs as larger corporations. Therefore, the network that the carrier operates has to have a lower-cost basis. "That's a quandary for the carrier," says Howard, "because it is another network, even if it is cheaper than SONET/SDH."
"Five years from now, we see a large number of service providers with nationwide Ethernet services and some international," says Nortel's Dondero. "Ethernet will start to eat into legacy-type services. Eventually, we expect Ethernet private line service to replace traditional private line services."
Meanwhile, optical Ethernet is gaining momentum worldwide. Today, almost every major service provider is trialing Ethernet services in some form. As shown in Figure 2, Asia-Pacific leads regional spending with 43% in 2002, followed by North America with 37%; Europe, Middle East, and Africa with 15%; and Central America and Latin America with 5%.