MARCH 17, 2010 By Stephen Hardy -- Nestled within the appendices of its National Broadband Plan, the Federal Communications Commission (FCC) offered suggestions about how Congress and others should assess the performance of the Department of Commerce and National Telecommunications and Information Administration (NTIA) Broadband Technology Opportunities Program (BTOP).
The recommendations came at the behest of Congress, which directed the FCC to review the progress of the BTOP. Congress did not include a similar review of the Department of Agriculture’s Broadband Improvement Program (BIP) in its mandate.
In its review, the FCC was careful not to second guess how BTOP money has been allocated so far. Instead, it noted that project funding is ongoing (see "NTIA rewards Level3, others, with broadband stimulus grants" as an example), and therefore that it would focus on how Congress and others should evaluate the effectiveness of the program overall in the future.
In its “BTOP Progress Assessment,” the FCC first outlines what the NTIA is trying to do with the $4.7 billion the American Recovery and Reinvestment Act has charged it to allocate by the end of this September. BTOP focuses on funding three types of enhancement projects:
- infrastructure
- public computing center (PCC) capacity
- sustainable adoption of broadband services
As of the middle of last month, the NTIA had allocated $597 million. Infrastructure programs, as expected, have received the greatest share at $547 million. PCC programs have received $42 million and sustainable adoption programs $8 million.
The FCC noted that in addition to meeting short-term goals in particular applications, the projects can serve as testbeds to answer such questions as
- What leads people and communities to adopt broadband?
- What quantifiable difference does broadband make?
- How does broadband affect economic development?
- How does the “broadband experience” vary among communities, various demographics, and institutions?
With these factors in mind, the FCC made four recommendations for assessing the performance of the BTOP:
- Ensure the assessment tracks program outcomes as well as execution: While assessors should determine whether a particular project met its deployment goals, the FCC said that the project also should be judged by whether it had “a meaningful impact in the context for which funding was specified.” For example, did a middle-mile project break a bandwidth bottleneck and enable broadband service delivery? This may mean developing control groups in unfunded areas for comparison.
- Develop measures that specify the outcomes to be assessed: Metrics need to be established to measure a program’s progress. As much as possible, these metrics need to be applicable across multiple programs.
- Create a panel of experts from the academic and research community to advise on assessment approaches: The FCC notes that there is “little empirical evidence” on the impact of programs such as BTOP. As researchers outside of government measure the impact of the program, the NTIA should take advantage of such growing expertise.
- Employ longitudinal design in assessing programs where possible: Which is a fancy way of saying that programs should be judged over the long term, well beyond when assets have been deployed. Economic effects may take years to develop and assessment programs should take account of this fact, the FCC said.