Detailed Information on the
National Park Service - Technical Assistance Assessment

Set more ambitious targets with stronger and clearer definitions of success.

Increase national oversight of operations by: tracking project data at the national level, setting regional performance goals, tracking regions' performance, and holding personnel accountable.

Clarify national project selection criteria and require all regions to use the national criteria to effectively target the highest priority projects.

Measure: Average lifetime cost of projects completed each year

Explanation:This long-term efficiency measure shows RTCA's average (per project) cost (in 1999 dollars) over the lifetime of projects closed out during a given fiscal year. This is the per-unit cost of RTCA output (completed projects). These projects are expected to contribute to the annual output measure of miles/acres and the long-term outcome of successful projects. Baseline was set for 2003 and reflects the first year that reliable data are available. Results are expected to be somewhat variable, depending on the characteristics of projects that are completed in a given year, but over the long-term, increased efficiency is anticipated as indicated by trends for the annual efficiency measure. Actual results for 2004 are substantially lower than the baseline, yet this was an unusual year where a high number of smaller projects were closed, thus the target is set closer to the baseline level.

Measure: Average cost (per project) of projects worked on each year.

Explanation:This annual efficiency measure shows the average cost of working on a project in a given year (calculated as the total RTCA appropriation (in 1999 dollars) divided by the number of projects worked on that year). It can be used to calculate the number of projects RTCA can undertake for the funding it receives. This measure provides feedback on annual project costs that allows managers to make adjustments in the near-term and provides an indicator of performance on the long-term efficiency measure of the average lifetime costs of projects.

Measure: Percentage of RTCA projects that conserve natural resources and/or create outdoor recreation opportunities within 5 years after RTCA completes technical assistance to build, enhance, or protect trails, rivers, or open space.

Explanation:This measure is used to demonstrate that the program is getting results, but must be combined with an additional measure to demonstrate the quality or long-term sustainability of our technical assistance to partners. It is the percentage of our partnership projects that are successful in achieving on-the-ground results that contribute to achievement of GPRA goals. Data are reported 5 years after RTCA completes technical assistance.

Measure: Percentage of RTCA projects that result in organizations with increased capacity to undertake ambitious on-the-ground conservation and recreation projects, measured biennially as part of the Customer Satisfaction Survey.

Explanation:This measure is used to demonstrate that the program provides quality technical assistance to our partners and enhances their organization's ability to accomplish conservation and recreation projects in the future. It must be combined with an additional measure to demonstrate that our projects achieve conservation and recreation results. It is the percentage of our projects that our partners are satisfied with project assistance to develop organizational capacity to undertake the project. Actual percentages are +/- about 7%.

Measure: Percentage of communities served that are satisfied with NPS Partnership assistance in providing recreation and conservation benefits on lands and waters.

Explanation:The RTCA measure for Community Satisfaction is directly related to NPS Strategic Plan Performance Long Term Goal 3b2. Data is collected by an independent contractor who administers an OMB-approved survey of primary project partners that have received at least 3 pay periods of RTCA assistance in the previous year, or at least 4 pay periods in the previous two years. This measure is used for evaluating quality of technical assistance, which in turn contributes to the success of partners in achieving on-the-ground miles/acres results. Long-term targets are adjusted periodically based on previous performance. Actual percentages are +/- about 4%.

Measure: Percentage of communities served that are satisfied with NPS Partnership assistance in providing recreation and conservation benefits on lands and waters.

Explanation:SEE EXPLANATION for MEASURE 5. This is a statement of biennial targets necessary to ensure continued high level of performance necessary to achieve long-term goal 5. RTCA measures biennially, as a cost-saving measure, due to consistently high level of achievement. Actual percentages are +/- about 4%. Targets are not expected to rise above 95%; the goal will be to maintain high level of satisfaction.

Measure: Acres of park, wildlife habitat and open space preserved with NPS partnership assistance

Explanation:Baseline was established in 1999 and equals 11,013 acres. This is a statement of annual targets necessary to achieve DOI/NPS strategic plan goals. Targets are adjusted to reflect past performance when the DOI/NPS strategic plan and/or the available budget and FTE are revised. Consideration is given so exceptional projects that contribute extraordinarily high results (see actuals for 2000), do not create unachievable expectations.

Measure: Miles of protected river corridor conserved with NPS partnership assistance

Explanation:Baseline was established in 1999 and equals 418 miles of rivers conserved. SAME EXPLANATION AS FOR MEASURE 7.

Measure: Miles of trails are conserved with NPS partnership assistance

Explanation:Baseline was established in 1999 and equals 1409 miles of trails. SAME EXPLANATION AS FOR MEASURE 7.

Is the program purpose clear?

Explanation: The National Park Service (NPS) External Technical Assistance (TA) programs "help create local, regional, and State networks of parks, rivers, trails, greenways, and open spaces by collaborating with community partners and NPS areas in every State." The programs implement the natural resource conservation and outdoor recreation mission of the NPS and contribute to two NPS goals: 1)natural and cultural resources are conserved through formal partnership programs; and 2) through partnerships with other Federal, State, and local agencies and nonprofit organizations, a nationwide system of parks, open space, rivers, and trails provide educational, recreational, and conservation benefits for the American people.

Evidence: FY 2006 National Park Service Congressional Budget Justification; Outdoor Recreation Act of 1963 - 16 U.S.C. 4601; National Trails System Act (1968) - 16 U.S.C. 1241; Wild & Scenic Rivers Act (1968) - 16 U.S.C. 1271; RTCA Strategic Plan (2004); What Is the Rivers & Trails Program? (2005); 40 USC § 550 (2003) Federal Lands to Parks.

Does the program address a specific and existing problem, interest, or need?

Explanation: The TA programs address existing problems and needs. Programs support the NPS vision of a national network of parks, rivers, trails, greenways, and open spaces. For the Rivers, Trails, and Conservation Assistance (RTCA) program, Congress explicitly recognizes the need for river conservation, open space preservation, and trail development through each of the program's principal legislative authorities. RTCA represents a cooperative model for achieving natural resource conservation and accessible outdoor recreation. NPS units and other Federal lands help address the problem but NPS recognizes that parks cannot conserve resources in isolation. The RTCA program addresses this need with a goal to "help create local, regional, and State networks of parks, rivers, trails, greenways, and open spaces by collaborating with community partners and NPS areas in every State," with a majority of projects contributing to this goal. For Federal Lands to Parks (FLP), Congress stated a need for recreation resources for all Americans, which FLP directly addresses by transferring surplus lands for park and recreation use.

Evidence: Outdoor Recreation Act (1963); National Trails System Act (1968); Wild & Scenic Rivers Act (1968); RTCA Strategic Plan (2004); What Is the Rivers & Trails Program? (2005); 40 USC § 550 (2003) (Federal Lands to Parks); Summary FY-04 Network Projects Survey (2004).

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: The TA programs are not redundant or duplicative. For example, DOI, through the FLP program, is uniquely responsible for park and recreation public benefit transfers. Several other Federal agencies provide technical assistance, but only RTCA provides technical assistance that combines natural resource conservation objectives with outdoor recreation development. RTCA's assistance is complementary, not duplicative, of Federal, State, and local efforts, positioning beneficiaries to more effectively use other services. The program occupies a distinct niche, as evidenced by the mission statements of private organizations that offer certain common services. Private groups concerned with similar issues do not freely share services with all applicants, and they principally address the providers' agendas rather than the recipients'. The program serves some populations not served by private programs.

Evidence: Federal Resources to Support Community Parks (2004); Mission statements and descriptions (2005); Catalog of Federal Domestic Assistance (2005); Chart: Public Benefit Transfers (submitted as evidence 6/7/05).

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: No major design flaws have been identified that would prevent the TA programs from meeting their goals and objectives. The programs are designed to respond to communities' requests for technical assistance in conserving natural and recreational resources throughout the country. The RTCA program allows NPS to conserve natural and recreational resources without additional Federal land acquisition and management costs. Projects compete for RTCA assistance at the regional level. All TA programs consistently receive high "customer" satisfaction ratings. Still, the program managers' flexibility to prioritize and adjust national staff and funding allocations in response to changing needs could be improved. RTCA is making progress in improving flexibility as a result of the new RTCA strategic plan.

Evidence: Budgeting Guidelines (2004); Accountability Measures for Online Database (2005); Chart: FY2000-2005 Repeat Cooperators List; FY05 Project Review Scorecard from Pacific West Region; Resource Allocation Policy (2004).

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: RTCA's process for allocating funding to the regions and the regions' processes for selecting projects do not demonstrate that the program effectively targets the highest priority projects. OMB is also concerned that the program's decentralized structure does not allow for adequate national oversight and management to ensure targeting of resources. Finally, RTCA has limited ability to respond to changes in regional or national conservation priorities (see question 1.4) once staff are allocated to the field and must wait for vacancies to occur to adjust funding allocations. NPS has made progress in addressing these issues, including a proposal to increase the flexibility of funding provided to the regions. RTCA has also improved national project selection criteria in FY05, though the criteria could be further clarified to ensure a nationally consistent project selection process. Program resources are directly applied toward assisting beneficiaries, with most of the budget and the FTE for the TA programs assigned to provide technical assistance.

Evidence: Catalog of Federal Domestic Assistance (2005); Application Guidelines and Criteria (2005); Project Information Sheet (2005); Typical Project Workplan (2002); Letter: City of Bamberg biennial report (July 1997); Application to Acquire Surplus Federal Property, DOI NPS, FLP (rev. April 2000); RTCA - Budget Allocation Process, Section C4.

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: RTCA, the largest TA program, has new long-term outcome measures that OMB and DOI/NPS agree align witht he NPS strategic plan and will be used to track program performance. RTCA's Long-term Outcome Measure is tied to the purpose of the program: to provide technical assistance to partners so they can conserve natural resources and provide outdoor recreation opportunities. A combination of two measures is used because either alone would be insufficient; together they measure RTCA's program impact: a) Percentage of RTCA projects that conserve natural resources and create outdoor recreation opportunities, measured 5 years after RTCA completes technical assistance. b) Percentage of RTCA projects that result in organizations with increased capacity to undertake ambitious on-the-ground conservation and recreation projects, measured biennially as part of the Customer Satisfaction Survey.

Evidence: PART Performance Measures; DOI Strategic Plan 2003-2008; DOI to NPS End Outcome Measures and Intermediate Outcome Measures; NPS Strategic Plan, 2001-2005; RTCA Strategic Plan 2004-2009; Serving Communities 2002 Community Satisfaction; Budget Manual extractions of definitions for projects and measurable results; Customer Satisfaction Survey evidence for 1999, 2000, 2002, 2004 (June 2, 2005); Memo and Chart: Selection of Projects with Greater Resource Significance (June 21, 2005).

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: The RTCA program has used existing data to set targets and timeframes for the new long term measures. Data are available for the project success measure beginning with projects completed in 1998 and 1999 that reported GPRA results in 2003 and 2004, respectively. The 1998/2003 data provides a baseline and targets for improving performance to increase the percentage of successful projects each year. Although these targets are based on work that has been completed, the program expects performance to improve based on management measures and training implemented in the past 5 years. Data are also available for the strengthened organization measure from the Customer Satisfaction Surveys in 1999, 2000, 2002, and 2004. NPS set a target of 85% for the "strengthened organization" measure, above levels achieved in 1999, 2000 and 2002. The target was exceeded in 2004. NPS will raise the target levels if the 2004 data are not anomolous. RTCA plans to modify the question for measuring strengthened organizational capacity in 2006 but has used data based on the current question to set baselines and targets.

Evidence: PART Performance Measures; Summary Comparing NPS Strategic Plan Goals; See Evidence for Question 2.1 Chart: Proposed RTCA Long-Term Outcome Measures, Targets and Data (June 6, 2005).

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: TA program partners contribute to annual measures which lead to achievement of the programs' long-term goals. These measures focus on outputs and include quantification of the miles and acres of rivers, trails, and open space protected/created through partnership projects. For example, RTCA's partner contributions are measured in a survey five years after RTCA program assistance is concluded (because in general on-the-ground results are accomplished several years after RTCA assistance is concluded). Biennial satisfaction surveys of current project partners allow NPS to assess the quality of assistance provided. In addition to a question about overall satisfaction with RTCA services, key measures included in the Community Satisfaction Survey include project partners' views of the effect of RTCA involvement on the achievement of project goals and whether RTCA has fulfilled commitments made to the project in a timely manner.

Evidence: PART Performance Measures; Greenbook Entries from 2001-2006; Serving Communities 2002 Community Satisfaction.

Does the program have baselines and ambitious targets for its annual measures?

Explanation: RTCA, FLP, and HRA have baselines and targets for their annual measure of miles/acres but OMB is concerned that not all of the targets are ambitious. Specifically, the TA programs often exceed the goals for open space/habitat conservation by large margins. OMB is also concerned that the programs do not set goals for these measures at the regional level, and that the lack of such regional goals decreases accountability below the national level.

Evidence: PART Performance Measures; Greenbook Entries from 2001 - 2006

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: Partners in FLP and RTCA all report to the NPS miles and acres performance goals. For example, RTCA partners commit to and work toward the annual miles and acre goals of the program through multiple steps. Applications, which include responding to RTCA's project criteria (which reflect program goals), are required for all RTCA assistance. Once partners are chosen, agreements and work plans are developed between RTCA staff and partners to assure project success. Internal documents and public web pages identify proposed project outcomes. Finally, completing miles/acres surveys upon project close-out and five years later allows partners to demonstrate accomplishment of RTCA goals.

Evidence: 2005 RTCA Application and criteria for selection; Sample Workplan; PART Performance Measures; Annual Reporting process (2004); Letter: Shoreline Greenway Trail Application (2004); Shoreline Greenway Trail Task Menu (work plan) (2004); RTCA 2004 Projects website extraction (2004); RTCA project information sheet (2005); Sample page of FY 2004 (mile/acre) Performance measure report (2004).

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: The set of TA programs have never undergone comprehensive evaluations and have not received sufficient program-specific evaluations on a regular basis. Specific programs have undergone repeated evaluations (though not sufficient to qualify for a "Yes" to this question); for example, RTCA undertakes a strategic planning process every five years that includes hiring a consultant to conduct an independent review of the program.

Evidence: Materials related to hiring an independent consultant for RTCA's Strategic Planning process (2004); Situation Analysis: RTCA Strategic Planning April 27, 2004.

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: The TA programs do not yet have a budgeting system in place that defines the relationship between new performance targets and budget resources. The Congressional justification does not adequately make clear the impact of funding, policy, or legislative decisions on expected performance. NPS expects budget and performance integration to improve with implementation of the Department's Financial and Business Management System (FBMS).

Evidence: National Park Service Congressional Budget Justification (FY 2003-2006); OFS Funding Request: Representing the Administration's FY06 Request; OFS Funding Request: Representing the Administration's FY '00 Request for a $500,000 increase; PMDS pages for 2001-2005 Strategic Plan.

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: The TA programs have made progress in addressing strategic planning deficiencies. For example, RTCA finalized a strategic plan in 2004 and has made progress in developing long term and annual measures. The new plan is leading to development of information systems for improving program management and monitoring partners' progress in achieving goals. FLP has revised its application to ensure full disclosure of the program's requirements in order to improve long-term compliance and enforcement ability.

Evidence: "Evidence for 2.1 - 2.7 Strategic Planning Meeting (Program Leaders, Dec. 6-9, 2004); Application instructions for Surplus Federal Real Property; Application to Acquire Surplus Federal Property (Revised 2004).

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: The TA programs collect data from program partners and use it to manage the programs. For example, RTCA surveys projects five years after the end of NPS involvement and uses that data to assess success and program priorities. The five-year time lag (between RTCA project completion and collection of data) is a challenge to "timeliness" but the data is still useful to identify needed management actions. For example, RTCA managers respond to weak performance on annual targets and needs identified in the customer satisfaction survey with training and other management actions. FLP and HRA collect and report to similar annual and long-term performance measures. For these programs, the results are often obtained sooner than five years.

Evidence: Manual: Biennial Community Satisfaction Survey (2004); Survey: Community Satisfaction Survey (2004); Manual: Annual Reporting (2004); Report: 2004 RTCA Annual Report (2004); Survey Results: Conservation Impact's "Situation Analysis" (2004); RTCA Strategic Plan (2004); Manual: Annual Reporting Process (2004).

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: The TA programs have not demonstrated adequate means to hold federal managers accountable for cost, schedule, and performance information, particularly at the regional level. All performance targets and goals are set at the national level. The programs have not established clearly defined or quantifiable performance targets for the regions and are unable to hold regional or local program managers accountable for performance. OMB is also concerned that the programs' decentralized structures do not allow for adequate national oversight and management. Program partners are held accountable for performance results. For example, FLP managers review partners' biennial reports to ensure that the partners are meeting requirements of the conveyance of Federal surplus property. If partners fail to meet all conveyance requirements, FLP managers pursue revocations of the deeds for noncompliance.

Evidence: NCRC AD performance standards (FY 2004); Program Leader performance standards (FY 2005); National Program Management (2004); FY 2005 Budget Justification (FY 2005); Annual (NPS) Performance Goals; Application Guidelines (2005); Intermountain Santa Fe letters (2001); FLP: Example of Biennial Report - City of Bamberg; FLP: Example of process for noncompliance - City of Nassau.

Are funds (Federal and partners') obligated in a timely manner and spent for the intended purpose?

Explanation: Operating funds for the TA programs are one-year appropriations with no carryover. For RTCA, annual budget plans required of all regional offices are reviewed in the Washington office to assure that funds are expended for staff assigned to RTCA and for their intended purposes. In the past, Congress has expressed concerns about RTCA's use of cooperative agreements that effectively provide financial assistance, given that RTCA is supposed to be a technical assistance program. The program has corrected the problem. All RTCA work on the agreements of concern will be completed by the end of FY 05, the agreements will not be renewed, and RTCA will not enter into such agreements in the future.

Evidence: Status of Funds AFS Report (2003/2004); Comptroller memo Fiscal Year Closing (2003); Procurement Cutoff email (2003); RTCA Budgeting Guidelines (2004); Appendix E: Project Criteria (2004); Intermountain Region Budget memo & Program Budget Report (2004); Allocation memo Philadelphia Support Office (2004).

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: The TA programs do have procedures in place designed to improve efficiency. For example, RTCA performs periodic development and revision of the RTCA strategic plan, frequently reviews non-personnel costs and travel expenses, and holds regular program leader meetings reviewing program operation issues. The new Strategic Plan emphasizes the development of networks with the goal of amplifying conservation and recreation outcomes. RTCA has both long-term and annual efficiency measures (see measures). The long-term measure shows the program's average expenditures (per project) over the lifetimes of projects completed in a given year. The cost varies depending on the characteristics of projects that are completed that year. The annual measure shows the average annual cost of projects worked on in a given year and can be used to calculate the number of projects RTCA can expect to assist for a given appropriation level.

Evidence: PART Performance Measures; New Project criteria (2005); New "Network" goal (2004); Efficiencies and cost effectiveness due to new "network" goal (2004); Strategic Planning Meeting notes (2004); Appropriation Expenditure History (2005); IMDE MT Field Office Justification (2003).

Does the program collaborate and coordinate effectively with related programs?

Explanation: The TA programs collaborate extensively with related programs. RTCA works collaboratively and coordinates with many nonprofit, State, and Federal partners, including the U.S. Forest Service, U.S. Fish & Wildlife Service, the Bureau of Land Management, the Centers for Disease Control and Prevention, and the Environmental Protection Agency. Some examples of collaborations are: a Memorandum of Understanding to Promote Public Health and Recreation; the Interagency Wild and Scenic Rivers Coordinating Council; the EPA/NPS Interagency Agreement; and a General Agreement with the Conservation Fund. Key partners were interviewed for the situation analysis while developing the new RTCA strategic plan and were also given the opportunity to respond to a draft. Other TA programs also collaborate extensively and effectively with many of the same partners, as well as with FERC, DOD, Army, Air Force, Navy, Education, Health and Human Services, Coast Guard, General Services Administration, and State and local agencies.

Evidence: Survey Results: Conservation Impact's "Situation Analysis" (2004); MOU to Promote Public Health and Recreation (FY 2002-2006); Interagency Wild & Scenic Rivers Coordinating Council (1995); EPA/NPS Interagency Agreement (FY2005); General Agreement with The Conservation Fund (2004); River Projects: Hydro (Re)Licensing (2005).

Does the program use strong financial management practices?

Explanation: The TA programs exercise effective financial practices. There are agency procedures in place (which each program follows) to ensure that payments are made properly for the intended purposes. The RTCA Budgeting Guidelines provide an example. The Independent Auditor's reports assure systems meet statutory requirements and receive clean audit opinions. No reference was made to any of the TA programs in the Auditor's report. Implementation of the Financial and Business Management System will allow for increased integration of financial and performance systems.

Evidence: RTCA Budget Guidelines (FY 2004); RTCA FY 2004 Program Budget Ledger (FY 2004); Internal RTCA Allocation memo (FY 2003); AFS Document Summary (FY 2004); Annual Report FY 2004 Accounting Operations Center Independent Auditor's report; AOC report (2005); NPS Comptroller's email (2005).

Has the program taken meaningful steps to address its management deficiencies?

Explanation: RTCA engages in a new strategic planning process every five years. The recent RTCA strategic planning process helped identify national management needs. A revised organizational design is one example of progress on a more efficient program management structure. The strategic plan and several other components of the program's Budgeting Manual were created or revised, and meaningful steps have been taken to address management deficiencies noted in the Associate Director March 2004 memorandum.

Evidence: Associate Director (Partnerships, Interpretation and Education, Volunteers, and Outdoor Recreation) Memorandum to Regional Directors (March 2004); RTCA Chief update memorandum to Associate Director (2005); RTCA Chief to Associate Director on Cooperative Agreements (2004); RTCA Strategic Plan (2004); Decision-making for RTCA (2005); Program Review Process (2002); Program Review Report (2000); Response to Program Review Report (June 2001); Program Review Report (2002); Organizational Design Workgroup Report (2004); Strategic Plan Meeting notes (2004).

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: The RTCA program's long term performance measures are new but the baseline data indicate that the program is making progress toward achieving its long term goals. The percentage of projects that successfully conserve natural resources and/or create recreation opportunities is increasing gradually. NPS expects recent management changes to show results in future surveys. The program has collected baseline data for the partner capacity building measure and has shown signficant progress in the 2004 survey. NPS will modify the customer satisfaction survey for 2006 to better address this question.

Evidence: PART Performance Measures

Does the program (including program partners) achieve its annual performance goals?

Explanation: The RTCA program generally achieves its annual goals and often exceeds them but there are years when some performance measures were not achieved. Because not all of the targets are ambitious (see 2.4), the program can receive no higher than "Small Extent" for this question. In FY 1999 and 2000 all performance measures for miles and acres of river, trails, and open space were achieved or exceeded. In FYs 2001-2004, at least one of the three performance measures were met. Although all of the annual performance measures may not be met each year, over time the long-term goals of the program are met successfully. Partners are committed to the contribution to these annual targets, as described in 2.5.

Evidence: PART Performance Measures; Table: GPRA Scorecard Report; Manual: Annual Reporting Process (2004) - see also 3.1 evidence.

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: The RTCA program has demonstrated increased efficiencies in the past several years, as demonstrated in the "cost per project" efficiency measures (see question 3.4). Because the data are new and only available going back a limited number of years, the program does not yet warrant a full "Yes" on this question. However, the data do demonstrate increased efficiency in the delivery of technical assistance.

Evidence: PART Performance Measures; See Evidence from 3.4

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: The RTCA program's dual focus on conservation and recreation, combined with its national scope, makes a direct comparison to other programs difficult. The program consistently receives overall customer satisfaction ratings above 90% in satisfaction surveys. The target for future years is 95%.

Evidence: Chart: Program Comparisons; NRCS Program Description from website; Customer Satisfaction Survey Data.

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: The TA programs have not undergone evaluations of sufficient scope, quality, and independence to satisfy this question.

Evidence: None provided.