Strategies for video services testing

By Taran Singh, IXIA -- A successful video services roll-out depends on delivering quality of experience, proving reliability, and 'getting it right' the first time.

A successful video services roll-out depends on delivering quality of experience, proving reliability, and 'getting it right' the first time.

By Taran Singh, IXIA

Between 13 billion and 14 billion video streams will be delivered over IP on a monthly basis by 2011, producing an estimated $3.3 billion in revenue, with another $1 billion accrued from downloaded video, say Yankee Group analysts. With so much at stake, savvy telcos looking to successfully jump on the triple-play revenue "gravy train" must ensure their networks and infrastructure can support the bandwidth and throughput requirements of a converged video, voice, and high-speed data offering.

Just as importantly, telcos must be able to deliver a quality of experience (QoE) to their customer base that meets, if not exceeds, the services those customers currently receive. To meet these demands, a robust investment of new IP infrastructure is required. Because a mix of video services will gobble up the lion's share of the bandwidth in a triple-play environment, it is imperative to thoroughly test both this cornerstone of the bundle as well as the new infrastructure—pre-deployment—to ensure a successful service roll-out.

While video over IP and IPTV are both part of the triple-play landscape and thus factor into the total revenue proposition, technically, they are not the same and must be examined in a testing environment both separately and simultaneously to be fully understood.

Delivered across the public Internet, video over IP, sometimes referred to as "over-the-top" video, may be classified as any of three things:

  1. Home-grown video converging on popular web sites like YouTube.
  2. Previously aired content from broadcast television networks that is rebroadcast across the Internet.
  3. Both live and rebroadcast webcasts.
IPTV, on the other hand, is linear and includes on-demand video content served up to subscribers over a separately managed, autonomous IP network.

For telcos, delivering video services over IP presents critical technical challenges, not the least of which is that IP is not designed to transport real-time traffic like ATM. Moreover, telcos do not have a mature proving ground either in the lab or with a large installed base of early adopters. As a result, they have not had the chance to validate their video services offerings. And if those challenges weren't enough, they are faced with new technologies, new platforms, and emerging standards, many of which are still under development.

With the revenue stakes as high as they are, telcos need a comprehensive, well-planned testing program. They must ensure that their IP networks are scalable enough to accommodate future growth and the addition of new services. They also must ensure that their networks will efficiently and reliably carry high-quality services all the way from the core to the customer premises—even during peak hours, under peak load, and when equipment failures occur.

But video services testing does not have to cause a migraine. There is a silver lining for telcos that strategically plan their testing programs. For example, telcos should conduct video services testing from both the service provider and customer points of view.

To begin, telcos must systematically test and verify various network devices in each of the video transport architectures, including video content servers, core and edge routers, access devices, and IP customer premises equipment. Such testing provides an understanding of individual device performance and may determine how much impact each has on the overall system.

Next, telcos should move on to system-level tests that incorporate more than one demarcation point in the transport architecture. In this way, telcos gain a clear understanding of how well the individual systems play with each other.

Finally, telcos should extend this approach to test the network end-to-end. As part of this sequential process, telcos should expect to run most standard routing and forwarding performance tests, looking at packet loss or latency. They should also look at characteristics under different load conditions purely from the packet side to ensure that the packet core will scale. Once the network has proved itself in this arena, telcos should begin testing from an application delivery standpoint.

Now the telco needs to flip its test perspective and analyze what the customers see from their living room couches. Video services need to be tested using a test tool that simulates a mix of real-world traffic using stateful application to prove the network actually delivers user-defined application requests within predetermined standards. Determining video quality—what will be good enough for the customer to see—becomes a critical factor in the test process. However, everyone's quality benchmarks are different.

Telcos should test about 25,000 video services and assess how well the video is delivered and how well it is perceived by the customers. Since video traffic makes high-bandwidth demands, it has little tolerance for jitter and even less patience for packet loss.

For example, a distribution network designed to deliver standard-definition content to 100,000 subscribers at 5% peak rate requires around 19 Gbits/sec of forwarding capacity. No network offers 100% forwarding capacity, though, so networks supporting video must be engineered to meet a very slim drop rate during congestion. A high QoE is critical. Some key ways to ensure QoE include performing channel request validation, monitoring channel change performance, measuring network service quality, and measuring perceived video quality.

It is important to test how the network's characteristics affect a video stream based on network delay and loss. This can be quantified as the media delivery index (MDI), which comprises the delay factor (DF), based on the arrival time of a packet at a given point of measurement, and the media loss rate (MLR), or the number of packets lost per second.

Telcos can objectively assess user perception of video quality using Video Quality Score (VQS) as well as Telchemy Video Quality (TVQ) metrics. VQS delivers a codec-dependent objective quality score between 1 and 5. While not strictly subjective, it is normalized to a mean opinion score (MOS)-like range and considers original video quality as well as its sensitivity to packet loss before encoding and transport. Video quality metrics (VQM) simply provides a score between 0 and 50 of the Real-time Transport Protocol (RTP) channel's capability to support video transmission. Numerous tests are available with variations configured for specific networks that extend beyond the scope of this article, but these suggestions should provide a good starting point.

Triple-play services, including all variants of video driven by increased broadband revenue on a per user basis, are a means of survival for the telcos, not simply top-line revenue. Infonetics Research estimates that with nearly 40% of their capital expenditure budgets going to triple-play service infrastructure, telcos are sending a clear message that the combination of voice, data, and video services is a long-term differentiator for them. According to Infonetics, "Carriers are demanding complete interoperability, full standards compliance, and an open and flexible architecture from their suppliers to ensure the content and services they provide will work right out of the box and far into the future."

The stakes are high and the challenges numerous, but once telecom operators have completed IP video services testing, then successful, timely deployment will become a reality, and they'll be able to take their triple-play services revenue to the bank.


Taran Singhis manager of IPTV at Ixia. He may be reached via the company's web site at www.ixiacom.com.

More in Home