Ultra-Low Latency OTN Technologies Boosting Brokerage Competitiveness

In the 2018 motion picture The Hummingbird Project, a ragtag team of Wall Street day traders join forces to build a secretive, multi-million-dollar fiber-optics line connecting the Kansas and New Jersey stock exchanges. The objective: to shave off just 1 ms of roundtrip trading latency between the two exchanges—an ambition which, if achieved, would lavish them with riches beyond their wildest dreams.

Pure Hollywood fiction, or another instance of art imitating life? Well, judge for yourself: In 2010, Spread Networks built a secretive $300 million direct cable line connecting the Chicago and New York stock exchanges, reducing latency by 3 ms. Spread went on to lease their one-inch-thick cable to 200 customers at a value of $2 billion!

So, what is all the fuss about?

Ultra-low “tick-to-trade” latency is the backbone of high frequency trading (HFT), a method of trading that accounted for 50% of all equity trade volumes in the U.S. in 2020, as well as a large share of daily transactions in futures, foreign exchange, credit bonds, and commodities markets.¹ One estimate suggests that a 1-ms advantage in trading applications can be worth upwards of $100 million to a major brokerage.² The flip side is that these firms are disproportionately affected by network disruptions, with one study finding they suffer $6.48 million in losses for every hour of downtime.³

These estimates highlight two factors that are critical for an effective HFT network:

While ultra-low latency is essential, it is insufficient—what truly matters is the relative latency advantage of one brokerage over another.
The network needs to be highly reliable and secure, minimizing downtime and security breaches.

In the WAN segment of the network, these criteria are most optimally met by Optical Transport Network (OTN) technologies. Traditionally, the strictest WAN requirements were served by MSTP/SDH. But with bandwidth requirements outgrowing the capabilities of those technologies, and the increased obsolescence of facilities that manufacture them, private firms are beginning to migrate to other technology.

Figure 1. One capability enabled by private lines over OTN is a high degree of control over latency, as seen here with an App developed for the Bombay Stock Exchange and running on Bharti Airtel’s network. SDN enables financial services companies to select the lowest latency path within the network to help ensure optimal performance. A similar implementation has been deployed around the Shanghai Stock Exchange.

Many are turning to IP/MPLS. While the general market may be well-served by IP/MPLS, this technology is not the most optimized for latency, availability, and security—the very criteria demanded by the finance sector. That distinction goes to OTN, which along with the lowest latency (see Figure 1), enables the highest network availability (99.999%), and highest security (hard slicing and Layer 1 encryption).

Advantage: OTN

Highlighting OTN’s latency advantage, one study compared the respective impact of traffic congestion on two networks, one based on IP/MPLS and the other an OTN, both of which followed a path connecting the Shanghai and Suzhou financial districts.⁴ While OTN had the latency edge when network traffic was light (0.77 ms vs. 0.9 ms), the study showed that it particularly outperformed IP/MPLS when traffic was at its peak (0.77 ms vs. 8 ms; see Figure 2). It is in these millisecond differences where financial service firms acquire a substantial trading advantage over their rivals.

Figure 2. 24-hour latency (in msec) between networks in Shanghai and Suzhou. OTN ensures lower latency regardless of whether the load is light or heavy.

OTN’s ultra-low, and stable, latency profile derives from it being a Layer 1, time-division multiplexing (TDM) technology, where congestion, jitter, and latency are minimal by design. In contrast, packet-switched networks like IP/MPLS, which are based on statistical multiplexing, incur considerable latency penalties and packet delay variation as network traffic fluctuates.

While it is true that when it comes to latency, microwave networks tend to outperform fiber-optic networks, microwave technologies also suffer from some significant physical limitations, such as the “rain fade” phenomenon, where signal integrity degrades due to airborne particulates such as rain, clouds, or pollution. Consequently, most firms that have deployed microwave technologies have tended to supplement their networks with fiber-optic lines, due to the latter’s higher reliability and higher bandwidth capacity. In other network use cases—for instance, over ultra-long-haul distances—microwaves are not considered feasible, in which case fiber lines are the best option. The argument of this article is not that OTN-based fiber networks are superior to microwave or vice-versa—if anything they should be seen as serving complementary purposes. Rather, when it comes to the decision to deploy fiber networks, OTN provides the more robust, lowest-latency approach.

Though OTN technologies have been widely adopted by brokerages in Asia, including China and India, they are only now being gradually incorporated into the networks of their counterparts in Europe and North America. This presents a first-mover opportunity to financial services firms that are keen to adopt the latest technology innovations ahead of competitors who may be running their operations on IP/MPLS networks. Switching to an OTN network presents a unique opportunity for brokerages to reduce latency with minimal upfront investment.

This is also an opportunity for telecom service providers to monetize the best-in-class characteristics of OTN by offering strict service-level agreements (SLAs) targeting high-end markets, including the financial sector. The opportunity comes at a time when the two key criteria for optimal HFT trading—high market volatility and high trading volume—are met. These conditions will likely persist into the foreseeable future as well.

Two network use cases for OTN will be explored in this article, showing the value of this technology for both the financial sector and telecom service providers:

Connecting brokerages to a stock exchange within the same urban region
Interconnecting branches of a brokerage across regions and continents.

Shaving latency from brokerages to the stock exchange

Latency-based SLAs enabled by OTN technologies are already being monetized in the financial districts of Shanghai and Mumbai. China Telecom and Airtel have built concentric latency rings around the Shanghai Stock Exchange and the Bombay Stock Exchange, respectively, offering brokerages guaranteed latency limits based on their proximity to the stock exchange. These SLAs are differentiated on the scale of milliseconds (Figure 3).

Figure 3. OTN-based services offered by China Telecom Shanghai5 (left) and Airtel Mumbai6 (right), showing ultra-low-latency SLA rings targeting high-value customers in the financial sector.

These networks are designed with OTN customer-premises equipment (CPE) serving as the WAN-facing devices, co-located respectively in server rooms at the brokerage and the exchange. Ultra-low-latency hub OTN and/or OTN switching equipment direct traffic between the two sites. Buoyed by this network implementation, the Bombay Stock Exchange now boasts of being the world’s fastest exchange.⁷

The premiums that telecom service providers can charge for a stable, ultra-low-latency connection are considerable. According to China Telecom Shanghai, one futures trading company was willing to pay eight times their existing private-line connection fee to reduce latency from 4.6 ms to 0.63 ms.^8,9 Deepak Sanghi, vice president of optical networks at Airtel, is on record as saying “customers are willing to pay anything” to obtain incremental improvements in latency using their services.¹⁰

Outside of Asia, Omdia reports that a major, unnamed national European service provider is set to introduce latency-based services, initially targeting the financial sector within its jurisdiction. Major enterprise locations have been identified, along with microsecond latency rings, enabling highly compelling SLAs.¹¹ In North America, where OTN-based services have not been as widely adopted by the financial sector, opportunities abound for telecom service providers and brokerages to exploit and benefit from latency-based SLAs.

OTN intercontinental use case

According to Omdia, a major financial services firm with offices in 70 countries and data centers in New York, Singapore, Tokyo, and London has adopted optical network technology to interconnect its sites, creating a robust infrastructure for their high-speed currency exchange, where every millisecond counts (Figure 4).

Figure 4. A low-latency network example.

Regarding the benefits of its low-latency OTN network, the firm’s vice president of network and security trading has noted, “It is critical that our low-latency financial trading network outperforms competitors during peak periods of intense activity. When peaks in traffic or microbursts occur for a very short time (microseconds), transactions can be lost or require retransmission, which has significant negative impact on the company’s financial performance.”¹² Given the volume of trading this firm does—for example, more than 5,000 currency pairs every millisecond—even millisecond changes in latency can result in millions of dollars a year in lost opportunity—for example, as large currency positions shift slightly during slight delays in trading.

To highlight OTN’s edge over IP/MPLS, consider two connected branches of the same brokerage, located respectively in New York and Tokyo. As HFT traders, the offices look for arbitrage opportunities between the New York Stock Exchange and the Tokyo Stock Exchange. Should, for example, Toyota stocks trading on both exchanges temporarily diverge in price, the brokerage will be in a race against its competitors to sell where the stock is overpriced and buy where it is underpriced. Who will win the race?

Although one may assume that it will be the firm who best optimizes latency where the bulk of it occurs—in the long-haul fiber lines connecting continents where latency is on the order of 100s of milliseconds—the fact is that these competitors are all likely using the same lines, leaving little room for differentiation in that segment of the network. Where latency advantages do accrue is at the terminals of those intercontinental fiber connections, when the data traffic enters a regional or urban switched network. It is there, in the networks of Tokyo and New York, where OTN provides a critical latency advantage over IP/MPLS, particularly within fluctuations in network traffic.

How can brokerages access OTN services?

Brokerages seeking to obtain a competitive edge with an ultra-low-latency OTN connection have several network options at their disposal, each representing a tradeoff between cost, latency, and security.

Figure 5. The most cost-effective option for deploying OTN.

Most cost-effective option. The most cost-effective option is to purchase an Ethernet private-line service that feeds into a OTN transport network belonging to one of the major telecom service providers (Figure 5). Although OTN, as a TDM hard-slicing-based technology, is inherently secure in comparison to a statistical multiplexing-based technology like IP/MPLS, the additional security that comes with Layer 1 OTN encryption is not available with this option. This option also does not provide end-to-end transport visibility.

OTN on-premises option. This is where the brokerage has an OTN CPE at their premises (Figure 6). Although this option is more expensive than the first, it offers lower latency, access to OTN encryption, and full transport visibility from end to end. However, like the first option, the brokerage’s traffic is transported alongside mixed traffic from the telecom service provider’s other customers (for instance, traffic from multiple sources may be multiplexed into an 800G wavelength).

Figure 7. The OTN deployment approach that offers the lowest latency.

Lowest latency option. This is where the brokerage leases dark fiber and builds their own dedicated network (Figure 7). The most expensive of the three options due to the cost of dark fiber, it offers the lowest latency, and is fully dedicated to the brokerage’s own traffic, offering the highest level of security and full visibility.

Pasha Ferdowsi is a product marketing manager with Microchip Technology’s communications business unit, supporting the company’s OTN and Ethernet PHY technologies.