Availability of virtual backbone routers will give buyers of Internet capacity many benefits not possible with the current options of best-effort backbone access or raw bandwidth.
CARL BLUME, IronBridge Networks Inc.
The explosion in demand for Internet Protocol (IP) services and an increasingly competitive service-provider landscape have led to a dynamic wholesale market for backbone network bandwidth. Industry analysts at IDC estimate that the wholesale market in the United States alone was worth in excess of $3 billion in 1999 and will exceed $13 billion in 2004, an average annual growth rate of more than 35% (see Figure 1).Today, buyers of wholesale backbone services have only two options: They can buy access to an existing IP backbone, or they can buy "raw" bandwidth in the form of dedicated circuits. While there are some buyers whose needs are fully met by one of these options, there are many buyers that will find neither option satisfactory.
But now, thanks to a new enabling technology called "virtual backbone routing," we are on the threshold of a completely new form of wholesale network service. With virtual backbone routing, service providers will for the first time be able to offer a virtual IP network service. This new technology will bring substantial benefits to both buyers and sellers of Internet backbone capacity and expand the wholesale market to include new classes of participants.
Today, the most common form of Internet wholesale is the leasing of high-speed access connections to an IP backbone operated by another service provider. This approach allows an Internet service provider (ISP) to take advantage of an existing backbone infrastructure with little or no cost in operating its own facilities. But it also forces the ISP to accept the quality-of-service-usually "best effort"-that the backbone network provides. And it gives the ISP no opportunity to add value by engineering the network to support the differentiated services that business Internet users are increasingly seeking.
Alternatively, ISPs that want to retain full control over their backbone network may choose to lease discrete optical circuits from a wholesaler. The options are to lease "dark fiber"-an otherwise unused fiber for exclusive use-or to lease managed fiber. In the case of managed fiber, the wholesaler employs a SONET/SDH multiplexer or ATM switch to subdivide the circuit into channels of a specific bandwidth. With the advent of WDM, a variation on this latter option will be to lease one or more lambdas (i.e., wavelength subdivisions).
If an ISP chooses to lease bandwidth rather than high-speed access to an existing backbone, it needs to implement and operate its own network facilities-routers, access devices, and supporting systems-to deliver end-to-end Internet services to its customers. Looking at the market as a whole, service providers can be divided into three categories based on their position along the Internet wholesaling supply chain (see Figure 2). The sellers in the wholesale market are known as Tier 1 network service providers (NSPs). In the United States, Tier 1 NSPs include AT&T, Level3, Cable & Wireless, Enron, Genuity, PSINet, Qwest, and UUnet. Most of these NSPs also operate outside the United States, where they are joined by former monopoly telecommunications companies such as BT and Deutsche Telekom.
Tier 1 operators own most of their own fiber but may source some additional circuits on the wholesale market to extend their geographic reach. Most Tier 1 NSPs are also ISPs, selling Internet services directly to end users. Some have become specialized "carriers' carriers," focusing exclusively on the wholesale market. The buyers in the wholesale market are:
- Tier 2 ISPs, such as Mindspring and Earthlink, which sell Internet services directly to end users, with differentiation based on market segmentation. For their long-haul transport functions, these ISPs depend mostly on the wholesale market but build and operate their own points-of-presence (PoPs) and backbone nodes.
- Tier 3 ISPs, which are the large numbers of small ISPs that sell mostly to medium-sized and small businesses within a limited geographic area, often using customer support and supplementary services like Web design and hosting as their differentiators. Tier 3 ISPs do not have their own backbone network and sometimes lease PoP facilities as well as backbone services.
- Cable operators and multisystem operators, which act as ISPs to provide consumer and small-business customers with Internet services on their high-speed access networks that also carry other services such as television and telephony. Their focus is on their high-speed access networks, and they typically rely on backbone operators to provide their backbone capacity.
- Internet businesses, which include Web hosting companies and data center operators (e.g., Exodus) and integrated content providers (e.g., AOL). These companies also rely on backbone operators to provide their backbone capacity.
In addition to these buyers, there are some emerging customer categories that will become important consumers of wholesale services in the future. These include application service providers (ASPs), storage area networks, wireless Internet providers, and IP telephony operators.
An Internet backbone, in its simplest form, is built from routers and the long-haul circuits that interconnect them. With virtual backbone routers, each physical router is logically subdivided into multiple virtual routers. Each virtual router acts like an independent router but does so much more efficiently because it shares common resources. An ISP using a virtual backbone router service takes control over one virtual router in a backbone node. The virtual routers are in turn interconnected either with dedicated circuits or with Multiprotocol Label Switching (MPLS) label switched paths (LSPs).
In some implementations, the virtual router runs a separate instance of the routing protocols and has dedicated I/O ports, buffer memory, address space, route table, and network management (see Figure 3). Using this technology, a backbone service provider can create a virtual IP network service. This service, targeted at wholesale customers, allows regional IP networks to be extended across the virtual backbone, while maintaining full customer control over quality of service, security, and performance. Indeed, the virtual-router-based backbone gives the wholesale customer the same management control and network performance as its own physical network.
It is important to stress that virtual routers comply with all applicable standards, operate using standard protocols, and are managed by standard protocols. They peer with other routers just like physical routers and behave in all other respects just like physical routers. The characteristics of a wholesale service that is based on the power and flexibility of virtual backbone routers can be illustrated by comparing it with the other wholesale options we introduced here: leasing best-effort access to an existing backbone and leasing raw circuits.
Like buyers of best-effort backbone access, ISPs choosing virtual routers do not have to implement their own facilities. But unlike buyers of best-effort backbone access, they are able to secure, engineer, customize, and manage the network to serve their unique business models and customer base. They get all the benefits of using their own routers and associated systems but without building their own nodes and PoPs.
The customer's network is secured from other networks. Each virtual router has completely separate routing tables and associated address space and is served by a dedicated set of MPLS paths or I/O ports. Traffic cannot pass between virtual routers. For example, a hacker gaining access to one virtual router would have no way of gaining access to another virtual router, so the buyer is completely protected from "denial of service" attacks launched from another ISP's network. Virtual-router management uses simple network-management protocol (SNMP) version 3 to ensure that customer access to management data is also completely secure.
The customer has complete control over network performance because its virtual router is isolated from other virtual routers. This isolation is essential if the buyer is to offer differentiated services (DiffServ). DiffServ and policing can be implemented independently on each virtual router. Isolation from other network activity is ensured by completely independent queues and dedicated buffer resources for each virtual-router port. The virtual router also ensures isolation from a performance standpoint by providing wire-speed packet forwarding and completely consistent performance independent of the load on other virtual routers.
The virtual-router buyer has complete control over administration and management. Each virtual router runs completely separate instances of the routing protocols and is independently monitored and managed using SNMP. Each virtual router can use separate IP address domains without the risk of conflict. All network-management functions and information that pertains to a particular customer's service infrastructure can be made available to that customer and only to that customer.
Virtual routing has the same components as an Internet service built from raw bandwidth, but each routing component is provisioned in virtual form. The buyer gets all the benefits of being able to engineer the network to suit its own needs, as if it leased raw bandwidth but at a much lower cost.
Virtual backbone routers eliminate the expense of purchasing and maintaining physical backbone routers. In addition, the buyer can leverage the superior economics of a terabit-scale router. A terabit router shared by multiple users can be significantly less expensive than a dedicated gigabit router on a per-port basis.
ISPs currently buying best-effort backbone access that migrate to virtual backbone routers will gain the ability to engineer and deliver their own backbone services using dedicated virtual resources. That will allow them to offer value-added services such as virtual private networks (VPNs), expand their competitive differentiation, and extend their reach.
For example, enterprises are increasingly demanding differentiated and VPN services offering enhanced levels of performance and security. ISPs dependent on best-effort backbone access cannot offer such services because they are at the mercy of the performance of their backbone provider and cannot isolate their customers from other backbone users.
But by using a wholesale service based on virtual routers, a Tier 2 ISP can offer these services. For example, a virtual router may offer separate queues that can be configured per virtual-router interface, and the Tier 2 ISP can select among several prioritization schemes to enforce class of service on each queue. The virtual router will also support emerging VPN standards such as the proposed BGP/MPLS standard, RFC2547.
Meanwhile, virtual routers enable competitive differentiation. Because ISPs dependent on best-effort backbone access cannot control or engineer their own networks, they have no opportunity to provide competitive differentiation through service quality. Using a wholesale service based on virtual routers, they can.
Virtual routers also allow a Tier 2 ISP to build virtual PoPs beyond its current area of operation, while ensuring class of service is maintained. The concept of virtual PoPs is illustrated in Figure 4. Geographical reach is particularly important in connection with DiffServ, since in most cases these services require that the end-to-end transmission be carried by the same ISP. Virtual PoPs support such transmissions.
Additionally, virtual routers make it possible for wholesalers to offer a virtual IP network service for limited periods of time or as customers demand. Because the service is completely software-controlled and leverages the flexibility of MPLS LSPs for its links, the wholesaler can provision services in minutes. This responsiveness allows wholesalers to offer customers tailored virtual IP network services for special events, like Webcasts. Any application that can benefit from the isolation and control of virtual backbone routers and occurs dynamically is appropriate for a virtual IP network service.
Buyers of raw bandwidth also have much to gain from the option of buying a wholesale service based on virtual backbone routers. Following the paradigm already established for other virtual networking services, sharing facilities means lower costs because of economies of scale and the advantages of statistical gain. The higher the capacity of network facilities, the higher density of use they can support at a given performance level.
With virtual backbone routers, the cost benefits of sharing go well beyond the router itself to encompass all the ancillary equipment and infrastructure costs that come with router deployment, including local networking, cabling, test and management equipment, power supplies and cooling, and the floorspace within which it is installed. And not only are overall costs reduced, but also the costs occur as the facilities are used rather than as upfront capital costs.
For the buyer, less equipment means fewer operational staff. By outsourcing their infrastructure, a proportion of the administrative workload of running the network is removed. The benefits of reducing staff go well beyond reducing the salary burden. Recruiting, training, and retaining technical staff is now among the biggest challenges faced by businesses.
If there is one staffing challenge greater than finding staff to work locally it is finding staff to work in another location. And once found, they need to be managed. Wholesale service based on virtual routers reduces staff costs even further because all network administrative staff can now be located at central facilities and management functions are carried out using SNMP.
With the Internet expanding rapidly, all ISPs need to be able to meet growing demand. Virtual backbone routers ease capacity planning because performance and utilization can be monitored directly, making it easier to anticipate and schedule capacity upgrades. As with VPNs and DiffServ in the corporate sector, enterprises will typically look to a single ISP for their entire network. To compete for this business, an ISP must have coverage wherever its customers need it. And both enterprise and ISP must be ready to respond to the challenges of mergers, acquisitions, and divestitures. Again, virtual routers bring the prospect of greater flexibility in geographic coverage. Tier 2 ISPs can acquire high- or low-volume connections to additional locations on an economical basis.
Sellers of wholesale capacity also have much to gain from selling virtual-router-based services rather than best-effort access or raw bandwidth.
The first benefit is added value. Selling a wholesale service based on virtual backbone routers will yield a greater return from each unit of bandwidth. But virtual routers don't require a huge increase in operational costs. While sufficient routing capacity is required to run the customers' virtual routers, this need typically can be accommodated without increasing the number of backbone routers (given adequate scalability in the type chosen) and therefore would not add a significant administrative burden. With virtual routers, the customer is responsible for engineering, provisioning, and supporting the additional capacity, so there is no commensurate increase in support costs in these areas.
In addition to boosting revenues, adding value increases customer loyalty (and consequently reduces sales and marketing costs). While it will be technically possible to combine virtual infrastructure from different suppliers, most buyers will opt for a single source to simplify their own administration. As customers become familiar with operating the virtual routers offered by a particular wholesaler, their loyalty to that provider will increase. This factor will also translate into a strong early-mover benefit for wholesalers that introduce virtual-router-based services first.
Compared to wholesaling raw bandwidth, a major benefit of selling virtual-router-based service is that the seller retains control of the fiber. As new technologies increase the bandwidth derived from the fiber, the Tier 1 NSP will be able to leverage them directly into additional business.
Sellers that also offer their own ISP services will further benefit because with virtual routers wholesale customer traffic is isolated from that of the end-user customers, eliminating possible interference from potentially large and uncontrollable traffic sources.
Tier 1 providers need more than virtual routing alone to capitalize on the virtual IP network-services opportunity. The underlying backbone network needs to measure up to new levels of scalability, reliability, and consistency in performance. It must support multiple public networks off a common shared infrastructure.
The need for scalability arises simply because the Tier 1 network-or more specifically, the backbone routers on the Tier 1 network-must support more users, more traffic, more bandwidth, and more connections. The routers also will need to grow in all dimensions as the underlying network grows. Support for trunks at up to OC-192c speeds is a fundamental requirement but is of reduced value unless the router supports line-speed forwarding and has adequate buffer memory to support these very high data rates.
Reliability is vitally important for any backbone router. As the scale of the network grows, the cost of downtime becomes ever greater, and with multiple public networks running over a common infrastructure, the option of scheduling downtime for upgrades moves from being very undesirable to completely unacceptable. The backbone routers must offer nonstop operation under all circumstances, including network upgrades.
Because the opportunity for Tier 2 providers will increasingly depend on guaranteeing specific service levels to their customers, it is also very important to the wholesale provider that the performance they offer is absolutely consistent. It also must remain consistent as the network scales, whether by simple addition of users or by addition of new networks through additional virtual routers.
With intense, global-scale competition among large backbone operators, all market participants must be alert to new opportunities that can increase market share or grow revenues. The new virtual backbone router technology provides just such an opportunity. By offering wholesale services based on virtual routing, a backbone owner can win a larger share of its available market, boost returns on installed fiber, and strengthen the loyalty of its wholesale customers. Because virtual backbone routers offer users the power and flexibility to customize their service, they raise customer loyalty and increase retention.
Early movers into this new technology will gain a sustainable advantage, and those Tier 1 operators that move quickly to the starting line will be well-placed to reap the greatest rewards.
Carl Blume is director, product marketing, at IronBridge Networks Inc. (Lexington, MA).