The journey to 400 Gigabit Ethernet

May 26, 2015
For the past year, the development of 400 Gigabit Ethernet (400GbE) has been underway in the IEEE 802.3 Ethernet Working Group. Formally known as IEEE P802.3bs 400GbE, the project targets the development of 400GbE for a number of different of networks that must support an ever increasing amount of bandwidth, such as service provider and operator networks, hyper-scale data centers, and Internet exchanges. With these applications in mind, the project will define 400GbE operation over 100 m of multimode fiber (MMF), 500 m of single-mode fiber (SMF), 2 km of SMF, and 10 km of SMF.

For the past year, the development of 400 Gigabit Ethernet (400GbE) has been underway in the IEEE 802.3 Ethernet Working Group. Formally known as IEEE P802.3bs 400GbE, the project targets the development of 400GbE for a number of different of networks that must support an ever increasing amount of bandwidth, such as service provider and operator networks, hyper-scale data centers, and Internet exchanges. With these applications in mind, the project will define 400GbE operation over 100 m of multimode fiber (MMF), 500 m of single-mode fiber (SMF), 2 km of SMF, and 10 km of SMF.

At the point of writing this article, the IEEE P802.3bs Task Force has only adopted one baseline proposal, for the 100 m over MMF objective based on 25-Gbps non-return to zero (NRZ) signaling across 16 fibers in each direction. Figure 1 summarizes the current proposals being considered within the Task Force and highlights the challenges facing the Task Force members in choosing baseline proposals.

Figure 1. Summary of optical proposals for 400GbE.


This figure highlights how different proposals are leveraging different lane rates of operation, numbers of different optical lambdas (λ) or optical fibers, as well as different modulation schemes – NRZ, pulse amplitude modulation 4 (PAM-4), or discrete multi-tone (DMT) – for the different reach objectives. Within the IEEE 802.3 Ethernet Working Group, technical decisions, such as adopting a baseline proposal, require a 75% approval. Thus, given the number of different proposals and underlying baseline technologies, the challenge in achieving a 75% consensus should be evident.

It don't come easy

The recent proliferation of optical multi-sourced agreements (MSA) and proprietary approaches targeting 40GbE or 100GbE has some voicing concerns regarding the delay in the Task Force choosing baseline proposals for the different SMF objectives. While the debate over standardization will continue to rage on regardless of whatever is stated here, it must be noted that standardization does not occur in a vacuum separate from the industry. In essence, standardization is a process by which members of the industry participate and ultimately make the decisions. Thus, the results observed during the standardization process can be seen as mirroring the mindset of the industry itself.

Figure 2 illustrates the different axes of freedom in developing a given optical proposal. It is the product of these four axes that, when ultimately combined, provide the optimal solution. Simple math shows that as the target rate of operation increases, the number of potential solutions, based on these four axes, will increase. Different factors for each axis will then come into play.

For example, for 100-m and 500-m applications, parallel fiber solutions have been found to be acceptable, but not for longer lengths due to the associated cost. The number of optical lambdas can also be varied, but cost and reliability become a concern at some number of lambdas. For modulation, one must consider the complexity, associated power, and latency.

Figure 2. Axes of freedom for optical solutions.


While Figure 2 describes the technical options that may be leveraged during the development of an optical proposal, it can also be viewed as representing different viewpoints, based on preferences or company product strengths or areas of investment. For example, with the "modulation" approach companies with signaling expertise are looking beyond the traditional NRZ signaling that dominates Ethernet today. The "# of fibers" approach can be viewed as the "parallel single-mode" (PSM) 40GbE approach that is being deployed today. The "# λs" approach speaks to those companies favoring a photonic integration approach.

This multitude of approaches is not a new thing. As noted before, there has been a proliferation of optical MSAs supporting 40GbE and 100GbE. Figure 3 highlights data from LightCounting's "Ethernet Transceivers Forecast," and looks at the projection of non-IEEE-standardized Ethernet optical modules. It is clear to see that the forecast for non-standardized optics grows with rate of operation.

Figure 3. Non-IEEE-standardized optics - Ethernet transceivers forecast
(10GBASE-USR estimated to be no more than 10%. Counted as standardized 10G. 40GbE
BiDi not included as "non-standardized.")

It should be noted, however, that these "Ethernet" modules all use chip-to-module interfaces that were standardized by different IEEE 802.3 40GbE and 100GbE projects. Thus, the IEEE 802.3 Ethernet Working Group has played a key role in standardizing these "non-standard" Ethernet ports and modules.

Looking ahead

At this time the fate of 400GbE SMF optics in the IEEE P802.3bs 400GbE is unclear. The Task Force has made considerable progress in the development of the 400GbE architecture. The Task Force has also recognized the inherent dynamics in the market that have been mentioned within this article. It has also recognized that any proposals presented today will be based on today's technology. Careful consideration has been given to enable an architecture that will be able to deal with future developments in electrical signaling, optical signaling, or the underlying coding. Thus, 400GbE ports will emerge, and they will be able to support either IEEE 802.3 standardized or non-standardized optical solutions.

Figure 4. 2015 Ethernet Rate Roadmap.

Figure 4 shows the Ethernet Alliance's 2015 Ethernet Rate Roadmap. This roadmap lays out a plan for future Ethernet speeds based on different approaches, such as serial rate approaches, Quad Lane approaches, and even highly parallel approaches. Some of this touches on the different axes shown in Figure 2. It is clear that as the industry debates the optics for 400GbE, it is also truly in the formative stages of determining the roadmaps of these axes, which will enable the drive the industry to terabit speeds.

John D'Ambrosia is chairman of the Ethernet Alliance and chief Ethernet evangelist at Dell.