By DAN DOVE
With 1-Gbps server interconnections as the mainstay for years, data centers are moving to 10 Gbps and appear poised to make relatively rapid leaps to 40- and 100-Gbps Ethernet for switch interconnects and 400-Gbps Ethernet between data centers. That's how quickly data-center bandwidth demands are spiraling upward.
Innovation of the IEEE 802.3™ family of Ethernet standards is accelerating too, and now is the time for end users to influence the ongoing advancement of the IEEE 802.3 standard. It is the marketplace itself that has consistently driven the refinement and expansion of IEEE 802.3. There are multiple ways that participants in the global Ethernet ecosystem can get involved today to help shape the direction of the technology.
Market-driven innovation
The IEEE 802.3 "Standard for Ethernet" defines wired connectivity for Ethernet local-area, access, and metropolitan-area networks globally. Entirely new business models, industries, and ways of life have grown out of Ethernet technology, which is celebrating its 40th anniversary this year. Innovation has been continuous and multidimensional throughout the standard's history, as the industry has sought to adopt additional media types, address bandwidth needs, and develop new protocols. That's the underlying story of the success of IEEE 802.3. In fact, the standard's development has always been and continues to be market-driven and open to everyone.
Consider, for example, the consensus-building activities that eventually yielded the IEEE 802.3ba™-2010 amendment. By the end of the first decade of the 2000s, Ethernet had successfully evolved from the LAN technology of choice for enterprises to a widely adopted carrier-grade option. It was evident, however, that the bandwidth needs associated with data centers, Internet exchanges, high-performance computing (HPC), and video-on-demand delivery had outpaced existing capabilities. So the IEEE P802.3ba project was launched in 2006, and the Ethernet ecosystem – stakeholders included users and producers of systems and components for servers, network storage, networking systems, HPC, telecommunications carriers, and multiple systems operators – moved swiftly to address the market need with a broader set of specifications than previously included in the existing IEEE 802.3 standard.
A closer look into the market revealed that the bandwidth requirements of computing and core networking applications were growing at distinctly different rates. On one hand, servers, HPC clusters, blade servers, storage area networks, and network-attached storage – typically making use of 1-Gbps and 10-Gbps Ethernet connections – showed a significant market potential for migration to a 40-Gbps Ethernet interface. On the other hand, core networking applications – switching, routing, and aggregation in data centers, Internet exchanges, HPC environments, and service-provider peering points – demanded a larger leap forward to 100 Gbps. This schism sparked the "Great Rate Debate," in which participants challenged each other to define optimal rate offerings that would deliver the best balance of performance and cost for a growing range of applications.
So with mass-market access to high-bandwidth applications accelerating and more powerful server architectures constrained by bandwidth bottlenecks for data centers, network providers, and end users alike, a new Ethernet generation was spawned. IEEE 802.3ba was rolled out in 2010 to extend the IEEE 802.3 protocol to operating speeds of 40 Gbps and 100 Gbps. It was the first standard ever to simultaneously define two new Ethernet speeds.
Remarkably, the new standard helped simplify complex link aggregation schemes that were being commonly used in network architectures, while maintaining maximum compatibility with the installed base of interfaces. The IEEE 802.3ba amendment successfully paved the way for a new generation of high-rate server connectivity and core switching in a single spec.
Reassessing market needs
In part because of the long hard work that went into determining the speeds to be targeted in the IEEE 802.3ba amendment, the IEEE 802.3 Industry Connections Ethernet Bandwidth Assessment ad hoc committee was formed in early 2011 to create a strategic view of the bandwidth trends affecting Ethernet wireline applications. The idea was to jumpstart future potential standards-development activities.
The ad hoc group gathered input from a range of application spaces (servers, data-center networks, HPC, financial markets, carrier and cable operators, Internet exchanges, the scientific community, etc.) and from users in various markets around the world. The result was the 2012 IEEE 802.3 Ethernet Bandwidth Assessment report, which delivered an important and broad examination of the market needs. (The report is free to download from www.ieee802.org/3/ad_hoc/bwa/.)
The report shows that bandwidth growth was being widely driven across application spaces and markets by parallel surges in users, access methodologies, access rates, and services (see Figure 1). Bandwidth requirements of network aggregation nodes were expanding more quickly than for end-station applications.
The report forecasts that networks will need to support an average 58% compound annual growth rate (CAGR), with the most aggressive CAGRs recorded in the financial sector and data-intensive science. More precisely, global networks would need to support 1 Tbps by 2015 and 10 Tbps by 2020.
No assumptions were made in the 2012 IEEE 802.3 Ethernet Bandwidth Assessment with regard to future interface speed needs. The decision about whether the exploding bandwidth requirements would be best serviced by higher interface speeds or parallel configurations using lower speeds would be left to future potential standards-development activities.
Evolving IEEE 802.3 again
Meanwhile, refinement of IEEE 802.3 continued as well. Today, IEEE P802.3bm is amending IEEE 802.3 specifications for lower-cost, lower-power, and higher-density 100-Gbps physical layer (PHY) and management parameters as well as 40-Gbps operation over extended (40-km) reach. The task force working on the new amendment is defining the following:
- 40G-BASE-ER4 (40-km) interface, which would allow low-cost 40-Gbps adoption in metropolitan applications, simplifying networks and entailing a low risk of change.
- 100G-BASE-SR4 and 100G-BASE-nR4 4×25-Gbps optical interfaces for multimode and singlemode fiber, respectively, which would reduce the cost and power of transceivers as well as the number of fibers per link in data-center applications.
- 4×25G electrical interface, which would eliminate the need for 10:4 multiplexing/demultiplexing in optical modules by creating four lanes at 25-Gbps data rate and fundamentally reducing width, cost, and power as well as increasing the density of 100-Gbps optics.
- Energy Efficient Ethernet, a technology that lowers power consumption for systems based on use, which is being extended to 40- and 100-Gbps data rates.
Again, IEEE P802.3bm is a market-driven activity. Today, even more throughput is needed inside the data center with the ongoing proliferation of smartphones and applications like video-on-demand, convergence, and cloud computing, which take advantage of dense cloud servers. Higher performance and capacity are required closer to the servers than ever before, and those requirements will demand higher-density, lower-cost, and lower-power 100-Gbps Ethernet technology (see Figure 2).
Working-group balloting on IEEE P802.3bm is scheduled to begin late this year. Technical revisions will continue next year, and the amendment is scheduled for completion in 2015. In the meantime, technical proposals for IEEE P802.3bm continue to be accepted. Indeed, a number of different alternatives are under consideration. For example, there are a variety of areas for consideration in development of the 100-Gbps approach for multimode fiber, such as equalization to compensate for bandwidth limitations of optics and traces, forward error correction, and its potential for increasing latency to achieve improvements in signal-to-noise ratio.
The task force has taken on the goal of supporting at least a 100-m reach on multimode fiber. In addition, a 20-m reach objective may be included in the specification, or it may lead to a separate PHY. If a single 100-m specification can be created that is so cost-effective there is no need for a 20-m option, then that would be the path the task force pursues. If 20 m would be substantially less expensive, then an additional shorter-reach alternative would be developed.
As many engineers know, standards development is a process of constantly trading market requirements against parameters such as power, cost, and performance. It's a multi-factored equation to be considered if the most effective possible standard is to be crafted. And that's why broad participation from across the Ethernet ecosystem is so important in the ongoing refinement of IEEE 802.3 standards. What engineers think is the best technical approach may not align precisely with what end users need, so it's critical that perspectives from across the landscape are taken into account to ensure that broad market demands are effectively addressed.
Getting involved
Standards are living documents. Expertise earned from the challenges and difficulties encountered in implementing standards in the field is invaluable in improving those standards.
Anyone can participate and direct the movement of an IEEE 802.3 project and guide development of Ethernet technology within the industry. End users can engage in many different ways, given their particular interests and environments, including:
- Through the Ethernet Alliance, there are opportunities to work within the 40-Gbps and 100-Gbps subcommittees, which are focused on driving consensus within industry toward approaches potentially adopted within IEEE. A systems architect who represents an end user's business, for example, could provide the kind of input that's especially needed in these activities.
- Participate as an individual within IEEE by becoming involved in the development of baseline proposals. In this way, end users can directly influence the development of the IEEE 802.3 standards family.
Furthermore, the application space is constantly changing. Consensus is being forged into the next areas of Ethernet innovation for wireline applications. There will undoubtedly be opportunities to guide future standards-development activities as the Ethernet ecosystem embarks on its next frontier of growth.
The IEEE 802.3 standard is the work of thousands of people over a period of 40 years. The development process is open to anyone, and all stakeholders directly participate in influencing the standard's ongoing innovation. Now is the time to get involved to ensure your voice is heard.
Note: IEEE 802.3, IEEE P802.3bm, and IEEE 802.3ba are trademarks of the IEEE.
DAN DOVE is IEEE P802.3bm™ Task Force chair and a member of the Ethernet Alliance. He also is a senior director of technology for Applied Micro Circuits Corp.
Past Lightwave Articles