by Leo Goyette
To paraphrase Mark Twain, "Reports of SONET's death have been greatly exaggerated." While Ethernet networks are undeniably gaining ground and likely represent the future, SONET will be with us for some time to come due to the large infrastructure already in place.
Blair Beck, planning engineer of Iowa Network Services, is one service provider with an eye on Ethernet who faces this reality. "We have a lot of embedded SONET equipment overlaid on DWDM that we have been leveraging to provide Ethernet transport to our customers. The incremental costs have been manageable but, due to increased demand, we're reaching a point where Ethernet over DWDM is looking to be the more cost-effective solution," he says.
As Iowa Network Services illustrates, multiplexing has long been used to reduce initial capital outlay and ongoing operational expense. Multiplexing lower rate and multiprotocol data onto a higher rate SONET payload can be accomplished using concatenation, Generic Framing Procedure (GFP), and other mechanisms. However, multiplexing multirate and multiprotocol signals into an Ethernet network creates some challenges due to Ethernet's packet-based and "best-effort" nature.
Ethernet, and particularly 10-Gigabit Ethernet (10GbE), is a good choice for data transmission because, besides being a widely used standardized protocol, it provides enough bandwidth to carry several lower-rate multiplexed signals. Ethernet packets have an advantage in that they can be broadcast for wide distribution as well as unicast for point-to-point data transmission with a simple switch, avoiding the expense and complexity of a SONET add/drop multiplexer (ADM).
By adapting the Internet Protocol (IP) in ways its creators barely imagined to support a multitude of new and exciting applications, Ethernet proponents have expanded its use from a LAN protocol to a ubiquitous underpinning for everything Internet.
IP is a connectionless, packet-based protocol, designed to deliver data on a "best effort" basis. Because of this, other protocols are generally layered on top of IP to create connections between endpoints. Packet-based networks are problematic for streaming data, but streaming data can be and is successfully carried over Ethernet networks. For example, everyone has at least heard of voice over IP (VoIP)—a clever, yet somewhat complex way to carry voice over Ethernet. VoIP relies on layering another protocol on top of IP along with data compression. VoIP applications have an advantage over other streaming data protocols in that there are usually pauses in the conversation. These pauses allow the circuit to "catch up" if necessary so that, although a conversation appears to be continuous, there are small gaps filled in by what is referred to as "comfort noise."
But what about the use of higher-rate streaming protocols such as OC-x, Fibre Channel, or proprietary protocols in applications (such as video) in which the data doesn't contain gaps? Many organizations—including the MEF, ITU, MPLS Forum, and the IETF—have Circuit Emulation Service over Ethernet (CESoE) initiatives, most of which primarily focus on OC-x protocols (and with good reason).
The object of unstructured circuit emulation is to faithfully transport traffic originating in non-packet-based protocols across Ethernet networks without regard to the traffic type. In other words, the data stream could be voice, video, packet data, etc., and it could use any type of protocol, including proprietary ones. Thus, circuit emulation can be used with a wide variety of data rates and protocols, and it enables a carrier to take advantage of an Ethernet network's cost efficiency and features while providing customers with the service(s) they desire. Part of achieving this goal is the ability to multiplex multiple lower data rate services into a single higher rate service, which reduces both capex and opex.
How to multiplex lower-rate emulated traffic streams based on the same protocol is fairly well-understood. But what happens when, as is often the case in real-world applications, multiple protocols need to be multiplexed? Adding multiple fibers or wavelengths, each carrying a single low-rate signal, is not a capex- or opex-friendly option, even when using Ethernet. A more cost-effective solution to the problem is to use a multiprotocol multiplexer.
Figure 1 illustrates a relatively simple multiprotocol multiplexer application. The dashed lines between the multiplexers indicate signals transported as Ethernet packets over a single fiber pair. For example, OC-48 data from the ADM connected to multiplexer "A" is packetized and sent through the 10GbE switch to multiplexer "C." The data is then converted back to a data stream and sent to the attached ADM.
Such Ethernet networks are dynamic. Users constantly add and drop services, which means that any sizable network is changing daily—and in all likelihood, is growing as well. Because of this constant change, a multiprotocol Ethernet multiplexer must be able to adapt to the changing needs of the network without forklift intervention. Ideally these changes can even be made remotely to eliminate or minimize truck rolls.
But a more fundamental requirement arises from the fact that when streaming data is converted to Ethernet packets, the process adds latency into the "circuit." Concatenating data into packets also adds a small amount of jitter into the system. This jitter needs to be mitigated at the receive side.
Latency will always be present when switching or routing Ethernet packets through a network. As long as the latency is a constant, or near constant, it is not a big issue. When it is not constant, it creates more network jitter, which creates a bigger problem at the receiver.
But the receiver's problems don't end there. An Ethernet network also adds the possibility of lost or non-sequential packets. Packets transported across an Ethernet network can become corrupted, dropped, or delivered out-of-sequence. In a streaming data circuit, there is always a signal transmitted. If a packet is lost, a "filler" packet can be transmitted to bridge the gap, but obviously the receiving device sees an error. Out-of-sequence packets can be accounted for if the packets can be re-sequenced prior to transmission.
For example, Fig. 2 illustrates the transmit FIFO inside of Multiplexer "C," which is connected by an OC-48 link to an ADM. For some reason, Packet 4 is delivered out of sequence. To maintain a steady error-free flow of data to the ADM, Multiplexer "C" must re-position Packet 4 between Packets 3 and 5 before they are transmitted to the ADM, or data corruption will result. Even if the re-sequencing is successful, allowing time for the process results in still more latency.
Finally, clock recovery is another issue when transporting packetized data. The received data has little or no clocking information from the transmitting side, and so the clock has to be recovered. Since, as all SONET proponents know, no two clocks are alike, there is a need to accommodate clocking differences between the transmitted and received sides. The implementation of this can be difficult, particularly in a multirate, multiprotocol multiplexer where each port can be running at a different rate.
All of these factors make network operations and management critical. Without information indicating the state of the network and the devices connected to it, network operators are operating blind. Back when Ethernet was mainly a LAN protocol, Ethernet devices did not offer the type of information required by a network operations center. This shortcoming is no longer acceptable.
"Ethernet OAM is critical for service providers wanting to provide services beyond best-effort. It allows them to monitor and manage customer service level agreements and provides demarcation functions for fault detection and isolation across multivendor and/or multiprovider environments," according to Dave Lee, vice president of U.S. business development of Telco Systems.
Today, a multiprotocol multiplexer providing CESoE, as with any device connected to a metropolitan or long-haul network, has to provide remote configuration and management as well as alarm notification. The information from a network node providing circuit emulation functionality is in many instances similar to information provided by other network nodes. Where it differs derives from the unique challenges discussed above.
One example would be the need to monitor and potentially send an alarm based on the amount of network jitter. If the gap between packets becomes too large, the data stream can under-run. Thus, if network jitter exceeds a specified amount for a period of time, an alarm could be sent indicating a possible service interruption.
There are several indicators pointing to the continued use of circuit emulation over Ethernet. First, interest in SONET networking may be waning, but unless people stop talking or watching videos over the Internet (a highly unlikely occurrence in the foreseeable future), streaming data is not going away and, in fact, is likely to increase. Second, there are challenges inherent in circuit emulation over an Ethernet network, but there are also solutions. Finally, although there are alternatives to Ethernet packet and SONET networks such as ATM, Ethernet networks are inexpensive and ubiquitous.
Given the above, it is highly likely carriers will continue to use circuit emulation over Ethernet networks. Moreover, since capital and operational budgets are tighter than ever, it is likely the need for multiprotocol multiplexers will only increase.
That said, today's Ethernet networks will one day give way to newer technology. When that occurs there may or may not be a need for circuit emulation or multiplexers. For now, packet-based multiplexers provide a way to overcome some of the problems associated with transporting streaming data using Ethernet's packet-based approach while allowing service providers to keep the cost of providing these services down.
Leo Goyette is CEO of Avvio Networks (www.avvionetworks.com).