Lab automation: Transforming test environments

By KAUSHAL K. AGRAWAL, BRYAN KNYSH, and JAMEY MUNROE, Calient Networks--Lab automation with an optical switch system helps lab managers reduce operational expenses, simplify lab planning, and automate test setup and execution--resulting in more efficient and effective operations.

Feb 28th, 2003
Th 119090
Figure 1: A typical verification cycle


Lab automation with an optical switch system helps lab managers reduce operational expenses, simplify lab planning, and automate test setup and execution--resulting in more efficient and effective operations.

By KAUSHAL K. AGRAWAL, BRYAN KNYSH, and JAMEY MUNROE
Calient Networks

When equipment vendors and communication provider labs test optical systems, the step-by-step manual process is labor intensive, time consuming, and prone to human error. A network setup for lab testing requires technicians to handle physical connectivity between systems-under-test and the test equipment. Every piece of equipment has to be continually reconfigured for each set of tests; countless hours are consumed in upfront setup, manual reconfiguration between test sets, and debugging errors. Debugging without knowing whether the obstacle is product-related or the result of a more straightforward problem--bad optical cable or improper patch connection--can cause significant delays.

Automated setup and reconfiguration using a bit-rate-, protocol-, and wavelength-independent transparent optical switch can dramatically shorten test cycle times and increase test coverage, thereby increasing operational efficiency and decreasing operational expenses.

Lab environment issues

During the test execution cycle a significant amount of lab equipment configuration and re-configuration takes place. In almost any product development cycle, much time is spent on designing and verifying the product (see Figure 1). Experience has shown that the biggest problem is setting up and re-creating new test topologies during all phases of verification and certification, ranging from unit tests, integration tests, feature or functional tests, system tests, regression tests, alpha/beta tests, and deployment tests to interoperability tests. Due to the lack of lab infrastructure automation, test equipment has very limited utilization. Static allocation, lack of knowledge management, and minimal remote access capabilities compound the problem. The result? Continued investment in operational-intensive labs with stranded inventory, minimal asset flexibilities, and low asset utilization, which lengthens test cycles and lowers overall product quality. Some of these operational expenses in today's labs include:

System bring-up
The existing fiber management systems (FMS) solution is essentially a big cabinet filled with fiber trays. A highly skilled technician with specialized splicing equipment is required to connect individual fiber to each port on a tray as part of the bring-up of the FMS. This is an extremely delicate and time-consuming process requiring several days of preparation before an ordinary FMS is usable. In addition, each fiber strand inside a fiber bundle is extremely thin and comes without the protective sleeve one would normally find on a regular fiber. Hence pinching or even breaking the fiber is not uncommon. A pinched fiber can create extra loss on the signal, a silent failure that is extremely hard to pinpoint after cabling is complete.

Network connection and setup
During the execution cycle, test beds must be developed for each test plan, each test case, and each test variation. Typical setup/configuration/re-configuration can comprise 40% to 50% of the total test cycle. Factors adversely affecting physical and logical setup may include bad cable quality, bad hardware quality, reversal of polarity on ports, tractability problems, and equipment unavailability. While these impact the execution profile of a test activity, they have a greater detrimental effect on the overall test coverage. Because verification cycles are usually very short (given tight delivery schedules), testing must be focused, targeting very specific functional areas. If 20% to 30% of test time is consumed by setup or reconfiguration, less time is left for testing. Therefore testing must be scaled back, which results in reduced test coverage and a potential reduction in product quality.

Rapid configuration and reconfiguration
Another operational impact is the inability to save and restore physical connectivity on the fly. Test equipment and test nodes are very expensive resources that need to be shared by different groups for various activities. With the existing FMS implementation, old connections (patch cords) need to be removed before establishing new connections. Re-configuration is time consuming, labor intensive, and error prone, particularly during shift work in the late night or early morning. Constant removal and insertion of patch cords can also shorten the lifespan of the patch cords as the fiber can be broken and the ferrule damaged. There is also the possibility of damaging the connector on the interface card itself. Repairing or even replacing the card can be costly and result in extended down time if the card needs to be sent back to the manufacturer for repair.

Fault isolation
People who have worked with FMS know that an optical power meter is indispensable when troubleshooting a problem with a connection. Trying to determine why a link doesn't pass light can be a challenging exercise. Normally one would have to debug each link at a time, which is, again, both labor and time intensive.

Usability and scalability
It is not uncommon to mislabel the ports on the FMS, which can result in mis-configuration of the test bed and require a lengthy and tedious troubleshooting exercise. In the lab environment, a fully filled FMS shelf is usually ordered and installed--though only the necessary number of trays (connectors) on the FMS is fibered up. The remainder would be fibered on an as-needed basis. However, it could take hours or even days to bring up the extra ports.

These are some of the major operational impacts a typical lab faces. Lab managers address them by lengthening product development schedules and hiring additional lab staff to cope with these operational challenges--adding unnecessary expenditure and complexity in the lab.

A different mindset

Test labs can achieve significant operational savings through automation using a transparent optical switch. Physical setup and configuration need only happen once. Cables connect test equipment and systems-under-test through the optical switch. Lab technicians use software to perform logical provisioning of test configurations and topologies.

Software also allows equipment to be easily switched into test configurations, as needed. When tests are complete, the equipment is available for other test configurations or to other test teams. Test configurations can be stored for easy re-creation of test topologies, as part of a standard test suite or for bug fix verification and reproduction.

Reconfiguration can also be done through test scripts for full lab test automation. The use of a scalable, transparent, and non-blocking optical switch allows a test organization to accomplish a lights-off, 24/7 operating environment. An optical switching system enables the realization of Any Topology, Any Time (ATAT) as shown in Figure 2.

Operational savings are the result of a variety of improvements. Automated scripts or stored topologies reduce set up time from days and weeks to virtually minutes. Pre-tested scripts and topologies also eliminate human setup errors and lengthy debug cycles. By reducing setup times, teams can focus on productive testing. Through switch management software, users can easily examine configurations to ensure bugs are related to the system-under-test, not to the incorrect setup of the test bed.

An optical switch with capabilities for mode conversion from singlemode fiber to multimode fiber and vice versa, along with the ability to monitor optical power at input and output for rapid fault isolation and inventory and budget control, provides additional operational benefits.

Such a paradigm change requires integrating a complete system in the lab as part of the lab infrastructure, which would then provide seamless services. Any attempt to piecemeal a solution by using sub-systems or a smaller, non-blocking switch could jeopardize the entire effort.

Where are the savings?

Lab automation using a transparent, bit-rate-, protocol-, and wavelength-independent, large (>100x100), non-blocking optical switching system can dramatically reduce operational costs. Based on case study models, this approach increases expensive equipment utilization from 30% to 65% and beyond, which results in a 50% reduction in capital equipment costs.

By automating test setup and configuration, a lab can reduce setup time from 50% of the test cycle to less than 10%, resulting in a nearly 40% savings in operational expenses that can be used to either shorten the test cycle or increase test coverage by executing more tests.

Kaushal K. Agrawal (agrawal@calient.net) is director of engineering operations and Jamey Munroe (jmunroe@calient.net) is director of sales at Calient Networks, headquartered in San Jose. Bryan Knysh (bryan@knysh.com) serves as a consultant to the company.

Figure 2: Any Topology, Any Time with an optical switch in the test configuration.
More in Design & Manufacturing