Detailed Information on the
National Telecommunications and Information Adminstration Assessment

Improve FCC-NTIA coordination

Provide updated cost/benefit analysis of paperless spectrum initiative to OMB in FY2006

Provide marginal cost data in FY2006 for frequency assignment actions.

Measure: Timeliness of Processing--Average number of days to process spectrum assignments

Explanation:This measure will permit NTIA to continue to track improvements in processing time through implementation of automation procedures and logistical procedures as a result of the paperless spectrum initiative. NTIA expects that the target will improve dramatically in FY 2008 (to 3 days or fewer) and beyond as investments in the paperless spectrum initiative are realized. Improvements in processing time benefit the American public by allowing NTIA's federal agency customers to more effectively accomplish their missions in ensuring our homeland security, maintaining public safety and the federal transportation infrastructure, and supporting law enforcement. Achieving these improvements are also part of the President's Spectrum Policy Initiative.

Measure: Certification Request Processing Time (Months)

Explanation:NTIA certifies per OMB Circular A-11 that spectrum will be available in the future for Federal agency planned radiocommunications. NTIA's approval prevents an agency from developing communications in the wrong frequency band and could cause or receive interference from other spectrum users that could result in being unable to implement the system and the loss of all the funding that was necessary to develop the communication system. The performance measure contains the planned average target time to obtain approval, the number of requests for spectrum support, the average time it took to provide approval, and a comparison of actual time for approval versus the target.

Measure: Space System Coordination Request Processing Time (percentage of actions meeting target approval time in days)

Explanation:NTIA provides approval and coordination domestically and internationally in a timely manner for an agency to operate its planned satellite communications. Coordination with other satellite spectrum users is essential to prevent interference to each other in light of the high costs of developing and implementing satellite communication systems.. The performance measure contains the planned average target time to obtain approval for coordination actions within the Space Systems Subcommittee process, the number of space system coordination requested, and the percentage of actions meeting target approval time.

Measure: Spectrum Plans and Policies Processing Time (days)

Explanation:NTIA and the FCC work together to on both an informal and formal basis. Much of the coordination occurs in the development of the various rulemakings and reduces the time it takes to come to an agreement. This performance measure contains the planned average target time to obtain NTIA coordination, the number of requests, the average time it took to provide coordination and a comparison of the actual time for coordination versus the target.

Measure: Number of milestones completed from the Plan to Implement Recommendations of the President's Spectrum Policy Initiative

Explanation:NTIA has been directed by the President in November 30, 2004, to implement his Spectrum Policy Initiative by implementing 24 recommendations contained in two reports submitted by the Secretary of Commerce and coordinated with Federal agencies in the OMB coordination process. NTIA has prepared an implementation plan with 136 milestones to be completed in 7 years, starting in 2005. The performance measure contains the planned target of the number of milestones required to satisfy the 24 recommendations for each fiscal year until completed and for each of the four major goals in the President's spectrum policy initiative. The actual number of milestones completed will be compared with the target.

Measure: Support new telecommunications and information technology by advocating Administration views in FCC docket filings, Congressional proceedings, and proposed legislation.

Explanation:NTIA will continue to examine an array of spectrum management policy issues dealing with innovative approaches to spectrum management including market-based incentives, and the effectiveness of current processes. This examination will be conducted in tandem with the FCC's proceedings on spectrum management policy, in which NTIA will participate on behalf of the Administration and as part of the President's Spectrum Management Policy Initiative. NTIA also will participate on behalf of the Administration in FCC and Congressional proceedings on telecommunications policies, including the development of appropriate regulatory treatment for broadband services. A number of Internet related policy issues will require NTIA action, including ICANN reform and continuing Internet privatization, domain name management both domestically and internationally, address proposals to regulate Internet services and content, and the combination of Internet and telecommunications addressing (ENUM). NTIA will work closely with other Federal entities in pursuit of policies promoting international trade in telecommunications products and services, promoting consistent international approaches to telecommunications policies, and improving relations with Western Hemisphere neighbors.

Measure: Number of website views for research publications

Explanation:NTIA will measure the number of website "hits" on its on-line research publications, replacing the measure of the number of peer-reviewed articles that are published in technical journals and publications. This measure will indicate the reception and utility of research results within the spectrum research and engineering community.

Is the program purpose clear?

Explanation: The National Telecommunications and Information Administration (NTIA) serves as the President's principal adviser on telecommunications and information policy matters and develops forward looking spectrum policies that ensure efficient and effective spectrum access and use. NTIA manages all spectrum use by Federal government departments and agencies and examines how the radio frequency spectrum is used and managed in the United States.

Evidence: NTIA Fiscal Year 2006 Annual Performance Plan www.osec.doc.gov/bmi/budget/06APP/NTIA06APP.pdf

Does the program address a specific and existing problem, interest, or need?

Explanation: As NTIA is responsible for coordinating radio spectrum usage across federal government agencies to ensure the needs of all stakeholders are met, its activities have a significant impact on the public and private sectors.

Evidence: TIA 2005 Telecommunications Market Review and Forecast; The National Telecommunications and Information Administration (NTIA): Budget, Programs and Issues, CRS Report for Congress dated April 11, 2005; Presidential Determination and Memorandum for the Heads of Executive Departments and Agencies (November 30, 2004) at www.whitehouse.gov/news/releases/2004/11/20041130-8.html.

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: As the manager of the Federal spectrum and principal advisor to the Executive Branch on telecommunications policy, NTIA's role is unique. However, the Federal Communications Commission (FCC) also plays a role in developing and implementing telecommunications policy to foster competition and universal access to telecommunications services for all Americans. Both NTIA and the Federal Communications Commission are charged with promoting the efficient and effective use of spectrum, however FCC is responsible for non-Federal spectrum management while NTIA manages Federal spectrum. NTIA's telecommunications research activities are similar to other efforts. FCC's Office of Engineering and Technology conducts spectrum research; the National Institute for Standards and Technology conducts research and sets industry standards for telecommunications applications; and according to a 2004 GAO report, several agencies funded $1.4 billion in research and development projects that may improve spectrum efficiency.

Evidence: NTIA and FCC responsibilities are laid out in Title 47, United States Code sections 301 et.seq. Sec 305 places responsibility for Government systems with the President. These powers currently are delegated NTIA. See Executive Orders 12046 and 12591; DOC DOO 10-10, 25-7. FCC's Office of Engineering Technology responsibilties are available at www.fcc.gov. GAO report GAO-04-666 outlines research and development funding for 2004 that had the potential to improve spectrum efficiency.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: NTIA does not currently have sufficient mechanisms in place to ensure efficient and effective Federal spectrum use. The current regulatory framework and spectrum certification process for federal spectrum use tend to focus on compliance with existing standards to avoid interference, not on overall spectrum efficiency. NTIA lacks the authority to implement market-based or other incentives to promote efficient and effective use of the Federal spectrum among Federal agences. As part of the President's Spectrum Management Initiative for the 21st century, NTIA plans to study incentives to promote more efficient and effective use of spectrum and seek authority to implement incentives, as appropriate.

Evidence: GAO and other studies have recognized the potential for market-based incentives to increase the efficiency of public sector spectrum use. A 2004 GAO report suggests that countries that have implemented mechanisms to reflect the opportunity costs associated with government spectrum use have reported more flexibility in reassigning and allocating spectrum to its most efficient use. See GAO-04-666 and GAO-03-277.

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: In general, NTIA program resources are targeted to address the program's purpose directly. The majority of NTIA program resources are utilized for spectrum management and policy activities, with the goal to ensure allocation of radio spectrum provides the greatest benefit to all people. NTIA works directly with public and non-public partners to ensure that it addresses agency goals for spectrum management, research, and policy activities.

Evidence: FY 2006 President's Budget Submission.

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: NTIA has two long term goals: to ensure that the allocation of radio spectrum provides the greatest benefit to all people; and to promote the availability and support new sources of advanced telecommunications and information services. While it is difficult to measure performance on these goals, the program has a few output-oriented performance measures that serve as proxies to show progress towards these goals. Recently, NTIA has developed a long-term measure based on the percentage of Spectrum Initiative milestones acheived annually. These milestones mark progress towards acheiving the recommendations of the Spectrum Initiative which are geared towards NTIA's goal of optimal spectrum allocation, which in turn also suports the goal of new sources of advanced telecommunications and information services. NTIA could improve measures related to efficiency for agency spectrum assignments. Currently, there is no measure that reflect improvement in Federal spectrum efficiency or cost effectiveness over time. NTIA should develop the cost-per-unit figure cited in its annual report to show more cost-effective assignment processing as a result of the application of advanced information technology. As NTIA develops methodology for measuring spectrum efficiency, the agency should consider a measure that reflects efficient Federal spectrum use in particular. A measure that reflects the knowledge transferred or applications developed and put to commercial use as a result of CRADA agreements, perhaps through the number of new applications or services that result from CRADA agreements, would also be useful.

Evidence: FY 2006 President's Budget and new measure was developed with this PART. See OIG Final Audit Report No. FSD-12856 (corrective actions taken as a result of OIG recommendations). See NTIA FY 2006 Annual Performance Plan at www.osec.doc.gov/bmi/budget/06APP/NTIA06APP.pdf

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: NTIA's long-term goal is to improve spectrum management processes throughout the Federal government so that time for spectrum assignments can be reduced from more than 15 days to 3 days or fewer by 2008, and ultimately to near instantaneously, supporting longterm goals for efficiency and effectiveness of spectrum use. NTIA's research efforts directly support this goal. The Spectrum Initiative for the 21st Century Implementation Plan, developed in coordination with other Federal agencies, provides targets for specific NTIA spectrum management and spectrum policy activities, which serve to advance both of the agency goals. NTIA adopted a new measure of the percent of milestones completed annually to measure progress in achieving these goals.

Evidence: See Performance Measures section, NTIA FY 2006 Annual Performance Plan at www.osec.doc.gov/bmi/budget/06APP/NTIA06APP.pdf, President's Spectrum Management Initiative Implementation Plan (pending publication), and new measure developed with this PART.

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: It is difficult to measure progress in terms of policy activities, as NTIA does not have direct control over regulation and policy of the telecommunications sector. Therefore, the agency relies on output measures that reflect the number of filings and legislative proposals. The recently adopted annual measure of milestones completed annually could show progress towards achieving the programs long term goals. NTIA's measure of unique web hits for ITS research reports does suggest the overall utility of its advanced telecommunications research. Other NTIA measures could be improved to better show progress towards meeting program goals of more efficient and effective spectrum allocation as NTIA develops a methodology for measuring spectrum efficiency. NTIA should also consider an improved efficiency measure for spectrum assignment processing that considers cost per assignment to better understand the cost-benefit of new information technology investments. Given the range of telecommunications issues and reform proposals, a steady target of 5 filings, statements, or legislative proposals to reflect policy activity may warrant reconsideration.

Evidence: See Performance measures section and NTIA FY 2006 Annual Performance Plan at www.osec.doc.gov/bmi/budget/06APP/NTIA06APP.pdf

Does the program have baselines and ambitious targets for its annual measures?

Explanation: NTIA has recently reevaluated baselines and targets for its annual measures, setting more ambitious targets. NTIA will reduce frequency assignment process from more than 15 days to 3 days or fewer by 2008; and seeks to demonstrate the utility of research through an increase in the number of website requests for NTIA research products. These targets are reviewed annually by senior management and are contained in management and employee performance plans.

Evidence: See Performance Measures section, and NTIA FY 2006 Annual Performance Plan at www.osec.doc.gov/bmi/budget/06APP/NTIA06APP.pdf

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: NTIA's government partners work through the IRAC, interagency processes and task forces to support both annual and long-term goals. Federal agencies show support the program through participation in the Policy Planning and Steering Group and staff level participation in the working level groups implementing recommendations of the Spectrum Management Intiative. NTIA is dependent on its Federal partners to provide the data necessary to evaluate efficient spectrum use. GAO reports suggest that the data provided by Federal partners is insufficient to promote efficient and effective use of spectrum. The lack of economic consequence to spectrum decisions mean that agencies do not always consider the value of spectrum when making investment decisions. NTIA should continue working with Federal agencies through the Spectrum Management Intiative to determine and implement economic incentives, where appropriate, to improve the efficiency of Federal spectrum use.

Evidence: GAO-03-277; GAO-04-666; Spectrum Management Intiative, information available at www.ntia.doc.gov/osmhome/spectrumreform/

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: While there are no regularly scheduled independent evaluations of the program, GAO and OIG studies have examined NTIA's role and activities several times in recent years. All evaluations have included recommendations which NTIA has implemented. Also, as part of the President's Spectrum Management initiative, NTIA collected public comment and input on spectrum policy and potential improvements to spectrum management through a notice of inquiry and forums.

Evidence: See GAO-04-1028 Report to Congressional Requestors, GAO-03-277, GAO-04-666, OIG Audit Report No. FSC-9866-8-001, OIG Final Audit Report No. FSD-12856.

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: NTIA's budget requests tie resource requests to the accomplishment of general program areas and new initiatives. However, improvements could be made to incorporate unit costs and to better tie base activities into annual and long term performance goals.

Evidence: See NTIA FY 2006 Annual Performance Plan at www.osec.doc.gov/bmi/budget/06APP/NTIA06APP.pdf See FY 2004 Department of Commerce Performance and Accountability Report (PAR) at www.osec.doc.gov/bmi/Budget/04APPR/KKPAR04.pdf

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: OIG evaluated NTIA performance measures in FY 2000 and made recommendations, which have been implemented. NTIA has continued to refine performance measures through annual reviews with DOC management. NTIA has identified performance measure improvements and has proposed new measures to OMB. As NTIA develops improved methodology for measuring spectrum efficiency as part of the Spectrum Initiative, more improvements could be made. Also, NTIA should consider incorporating unit cost in measuring the efficiency of its spectrum assignment processing.

Evidence: See OIG Final Audit Report No. FSD-12856. See NTIA FY 2006 Annual Performance Plan at www.osec.doc.gov/bmi/budget/06APP/NTIA06APP.pdf See Performance Measures; two new measures developed with this PART.

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: Spectrum assignment data for Federal agency partners is collected weekly through IRAC processes. Data is reviewed quarterly and utilized in establishing resource requirements and IT support directions. At the same time, GAO reports have suggested that information from agencies may not be sufficient for NTIA to ensure efficient and effective use of spectrum.

Evidence: See FY 2004 Department of Commerce Performance and Accountability Report (PAR) at www.osec.doc.gov/bmi/Budget/04APPR/KKPAR04.pdf, ITS Technical Progress Report at www.its.bldrdoc.gov/tpr/2004/, GAO Reports.

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: NTIA managers and staff are held accountable for results through inclusion of criteria for meeting NTIA performance goals and objectives in their annual performance plans. All NTIA employees will have such criteria in FY 2006. NTIA utilizes and reports quarterly on Earned Value Management to enable project managers to assess contract performance for projects and milestones. Federal partners are held accountable, as agencies can lose frequency assignments if not used and must report every 3 years as to continuing need. However, NTIA must rely on agency information and there are no incentives for Federal agencies to seek more efficient spectrum use. GAO has also found that agencies may continue to use the spectrum assignment, even if it has been revoked due to failure to report.

Evidence: See FY 2004 Department of Commerce Performance and Accountability Report (PAR) at www.osec.doc.gov/bmi/Budget/04APPR/KKPAR04.pdf NTIA Manual of Regulations & Procedures for Federal Radio Frequency Management, Chapter 2, p. 6. and GAO-03-277

Are funds (Federal and partners') obligated in a timely manner and spent for the intended purpose?

Explanation: Program funds are obligated consistent with overall program plans. Annual spending plans are established for obligations. Actual expenditures are tracked to intended uses as specified in spending plans. NTIA has never had an audit finding that funds were not spent as intended.

Evidence: See OIG Audit Report No. FSC-9866-8-001 See DOC FMFIA reports to Congress See FY 2004 Department of Commerce Performance and Accountability Report (PAR) at www.osec.doc.gov/bmi/Budget/04APPR/KKPAR04.pdf

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: Major IT investments are directed at improving the efficiency of the spectrum assignment process measure, and NTIA utilizes competitive sourcing for all major investments. At the same time, there is no measure of cost effectiveness associated with the anticipated increased efficiency from IT investments. NTIA plans to develop a unit cost measure in FY 2005.

Evidence: 2006 President's Budget, NTIA FY2006 Annual Performance Plan.

Does the program collaborate and coordinate effectively with related programs?

Explanation: NTIA's spectrum mangement activities are coordinated with its Federal agency partners in weekly IRAC meetings and through the development of annual MOUs. NTIA also coordinates with the FCC in several ways including twice a year meetings with the heads of NTIA and the FCC, monthly spectrum managers meetings, and clearance of any spectrum management changes. NTIA and the FCC jointly manage the 70-80-90 Ghz website, which is an indicator of future directions for U.S. spectrum management. NTIA, with FCC and IRAC participation, is developing a data dictionary for interchangeable xml formatted spectrum data.

Evidence: Annual MOUs with Agency partners. See GAO-04-1028 Report to Congressional Requestors, pp. 3, 8. See White House Fact Sheet on Spectrum Management at www.whitehouse.gov/news/releases/2003/06/20030605-5.html for examples of successes. See Joint NTIA/FCC website for 70-80-90 for frequency approvals (www.ntia.doc.gov/ntiahome/press/2005/708090website_02082005.htm) See ""Acting Assistant Secretary Michael Gallagher and FCC Chairman Michael Powell Meet To Plan and Coordinate Spectrum Policy""at www.ntia.doc.gov/ntiahome/press/2004/ntiafccmeeting_02272004.htm See spectrum training at www.ntia.doc.gov/osmhome/training/spman.html

Does the program use strong financial management practices?

Explanation: NTIA has received clean audit reports with no material internal control weaknesses every year for 17 years. Procedures are in place to ensure proper payments and minimize erroneous payments. NTIA uses NIST's CAMS for its financial management system and financial information is accurate with improving timeliness. No non-compliance with laws or regulations for financial management has ever been found.

Evidence: See DOC FMFIA reports. See FY 2004 Department of Commerce Performance and Accountability Report (PAR) at www.osec.doc.gov/bmi/Budget/04APPR/KKPAR04.pdf

Has the program taken meaningful steps to address its management deficiencies?

Explanation: NTIA's Office of Spectrum Management was recently reorganized completely to more clearly align with program needs identified through the President's Spectrum Management Initiative. The organization used business process reengineering principles to align domestic, international, policy, strategic and processing programs along with IT support. Annual performance plans for managers are established and reviewed to determine program effectiveness and needed changes.

Evidence: See Spectrum Initiative implementation plan, NTIA FY2006 Budget and Annual Performance Plan.

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: Because the agency only recently adopted long term measures and targets, it cannot get a yes. In developing the Spectrum Initiative Implementation Plan, NTIA has set a framework to better measure progress going forward. In addition to the new measure developed for this work, NTIA will also be held accountable for results through an annual report to the President on overall progress in achieving the recommendations for spectrum management reform. NTIA has shown progress in reducing the time to process frequency assignment actions from more than 15 days to less than 10 and is on track to meet its long-term goal of 3 days or less. NTIA efforts include technical research and policy activities to facilitate the introduction of new technologies and greater access to advanced telecommunications services and promote greater spectrum efficiency; examples include ultra-wideband systems, broadband over powerlines, additional spectrum at 5 GHz, and expanded opportunities for spectrum sharing at higher frequencies (70/80/90 GHz). NTIA also plays a large role in facilitating the adoption of 3G/Advanced wireless services, including playing a lead role in the reallocation of 45 MHz of Federal spectrum for commercial use.

Evidence: See Performance Measures and NTIA FY 2006 Annual Performance Plan at www.osec.doc.gov/bmi/budget/06APP/NTIA06APP.pdf, new measures associated with this PART, and Spectrum Initiative Implementation Plan.

Does the program (including program partners) achieve its annual performance goals?

Explanation: Because the answer to question 2.5 is no, this answer cannot be yes. However, NTIA has met its past annual performance goals as established through the GPRA process, and has refined its performance measures to better reflect progress towards long-term goals.

Evidence: See Performance measures and FY 2004 Department of Commerce Performance and Accountability Report (PAR) at www.osec.doc.gov/bmi/Budget/04APPR/KKPAR04.pdf, Question 2.5, and the new measure developed with this PART.

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: NTIA has reduced the time to process frequency assignment actions from more than 15 days to 10 days in 2004. IT investments in spectrum management process are expected to reduce the cost per assignment, however NTIA has not yet developed a unit cost baseline or target associated with this activity.

Evidence: See Performance Measures, FY 2004 Department of Commerce Performance and Accountability Report (PAR) at www.osec.doc.gov/bmi/Budget/04APPR/KKPAR04.pdf

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: For Federal spectrum management, no comparable federal, state, local government, or private sector programs exist. The GAO found that other countries' approaches to spectrum management may not be applicable in the U.S. because of the unique nature of the United States' military needs, and the President's responsibility for defense and Executive Branch missions.

Evidence: See GAO-03-277 and Performance Measures.

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: A GAO study found that while NTIA has several oversight activities to encourage accountability and efficient use of the spectrum by Federal agencies, the effectiveness of these activities is hindered. NTIA's performance measures have changed several times over the last few years as the agency struggled to find ways to measure results of activities, making it difficult to determine program results over time. Suggested improvements contained in GAO and OIG reports have been implemented or are in progress as part of the President's Spectrum management initiative, and NTIA is working towards improving its performance measures.

Evidence: See GAO-03-277, GAO-04-666, OIG Audit Report No. FSC-9866-8-001, OIG Final Audit Report No. FSD-12856 Presidential Memo on Spectrum Policy (Released by the White House June 5, 2003) at www.whitehouse.gov/news/releases/2003/06/20030605-4.html, White House Fact Sheet on Spectrum Management at www.whitehouse.gov/news/releases/2003/06/20030605-5.html