Putting reliability issues in perspective

By Don Gall and Mitch Shapiro

During the past year, Pangrac & Associates has evaluated network plans from several companies that are either new to the communications industry or have plans to position themselves for future requirements. Often these plans involve strategies such as exclusive use of underground plant and electronic and route redundancy without analyzing the impact on capital costs, maintenance issues, conditions in the local environment, and bottom-line reliability. While such techniques can be important in creating a "reliable" network, using them indiscriminately can be a waste of capital dollars.

One of the very first myths to debunk is that "building a network underground will be more reliable than the equivalent aerial plant." Our experience maintaining thousands of route-miles of both fiber-optic and coaxial cable over the last 30 years tells us this adage is not entirely true. There are exceptions, of course. For example, we know of a pipeline turned fiber-transport company that built a route using a large stainless steel pipe buried 5 feet deep. To our knowledge this route has never had an outage from physical damage, which is the exception, not the rule. Most communication networks are buried from 18 to 36 inches deep and in suburban areas need to come to the surface frequently for splicing and distribution.

Speaking from experience, a rock saw with carbide teeth, cutting through 24 inches of concrete does not notice a steel conduit filled with fiber optics. In one case, the rock-saw operator was unaware of the damage he'd caused until he noticed the saw had snagged the fiber and pulled roughly 60 feet of it out of the trench. Not only did the saw destroy the fiber at the actual cut, but it pulled all the storage slack out of the manholes on either side, fracturing the fiber.

Our experience also suggests that, statistically speaking, underground can be even more vulnerable to damage than aerial plant. As an example, in the greater Kansas City area during the early 1990s, we had approximately 100 route-mi of fiber-optic plant. Only 5% of this length was underground. During a five-year period, we had five fiber "cuts." Three of these cuts were in the underground portion of the plant. Calculating mean time to failure (MTTF) for 1 mi using these statistics yields an underground cable MTTF of three years versus 237.5 years for the aerial cable. This calculation was done in a state with an active "one-call" location service that contractors could contact to obtain information on all underground utilities before they began digging. Subsequent experience has led us to believe that this case is extreme, but as a general rule, underground is no safer than the equivalent aerial plant.

Let's also consider the widely assumed benefits of electronic redundancy. The premise here is that using hot standby backup devices will eliminate downtime. While this may always be true in a perfect world, the world we live in is full of "gotchas." One of our first experiences with backup devices involved a totally redundant transport network we designed and built several years ago. The system was designed to automatically switch to the working path in event of a failure. Our first breakdown occurred when the automatic switch failed and shutdown both of the perfectly functioning circuits-once again illustrating the maxim about best-laid plans and the validity of the theory that the weakest link in a chain is the first to break.

We are not arguing that electronic redundancy is always wrong. There are very many areas in a network were it is the proper way to maintain network reliability. But it is not a "cure-all." And it's important to remember that electronic redundancy will inflate network costs and can add significantly to day-to-day maintenance expenses.

Installing fiber-optic rings ubiquitously should also be questioned. The most cost-efficient network in a communications network is a bus, where all the network elements share the same fiber and electronics. In many cases, however, the overall network capacity cannot be supported by a bus network. Not quite as cost-effective but relatively efficient is a star topology. Rings are one of the most expensive ways to build a network, but there are many business and marketing issues that dictate their use. They are almost always found in the transport layers of large networks because the failure groups are so large.

Building fiber-optic rings for a residential hybrid fiber/coaxial (HFC) network service is very questionable for the last mile. It will typically be quite expensive to complete these rings, since in many areas, the subdivisions to be served are filled with short dead end streets. In addition, these subdivisions often have to be built underground, which in most cases means costs are at least twice that of equivalent aerial plant. Second, the failure group size is also quite small, which makes the cost per passing very high.

The third consideration is the reliability of the fiber plant. The average length of the majority of fiber paths used for distribution in an HFC network is 2 to 4 mi. If we use 100 years MTTF per mile (historical data we have observed so far, including cable damage, suggests this is a valid ballpark number), then the average time between failures is about 33 years. Even if it took eight hours to repair the damage, then over the 33 years, the network reliability of this segment would be 0.99997, which adds up to a whopping 44 sec of downtime per year. We would like to suggest that there are other areas of the network where the money could be better spent. Of course, there are commercial business applications where spending additional capital dollars could be justified.

One of the best ways we have found to determine where redundancy is needed is to understand how a company's network ties back to its business case. The business case will define the level of reliability required as well as the available revenue to support network costs. Competition also is an important consideration.

Almost every network can be divided into distinct elements. Each element has components that need to be identified in terms of their impact on the overall network. Things that need to be considered are reliability, network quality, failure group size, and cost.

Reliability of individual network components is usually measured in MTTF. Typically, passive components have much longer lifetimes, which almost disappear in the overall MTTF calculation. As an example, consider a network element with 10 components. Let's assume that nine of these components have very long useful lives, but that one has a very short life span of five years. The result is that the overall MTTF calculation would be very close to five years.

When all of the network's MTTF elements are added together, a figure of merit for reliability is generated that must meet or exceed the requirements set forth in the business plan. The business plan should also set minimum quality limits for the entire network. Staying within the limits required in each network segment is always an important consideration,

Failure group size is a key factor in determining where and when to add redundancy for two reasons. First, the larger the failure group, the more customers will be affected by a failure. Second, financially speaking, will adding redundancy to improve reliability add so much to per-customer costs that it will make the final product noncompetitive?

In conclusion, it is important to understand both your business goals and how your network functions to make the best possible overall decisions. The most elegant solutions are not always the best answer.

Mitch Shapiro has been tracking and analyzing the broadband industry for more than 12 years. He is a consultant with Pangrac & Associates and the founder of Broadband Markets, a supplier of strategic reports, databases, and analytical tools focused on competitive broadband networks and services. He can be reached at [email protected] or http://broadbandmarkets.com.