ExpectMore.gov


Detailed Information on the
National Park Service - Visitor Services Assessment

Program Code 10003723
Program Title National Park Service - Visitor Services
Department Name Department of the Interior
Agency/Bureau Name National Park Service
Program Type(s) Direct Federal Program
Assessment Year 2005
Assessment Rating Moderately Effective
Assessment Section Scores
Section Score
Program Purpose & Design 100%
Strategic Planning 62%
Program Management 57%
Program Results/Accountability 75%
Program Funding Level
(in millions)
FY2007 $716
FY2008 $794
FY2009 $933

Ongoing Program Improvement Plans

Year Began Improvement Plan Status Comments
2006

Examine cost and benefits of different interpretive techniques, such as waysides, guided walks, and exhibits.

Action taken, but not completed The performance standards workgroup has been established as a part of the I & E Renaissance. The FMSS work group is developing products to test for Summer 2008. Implementation scheduled for 2010 and draft templates are based on FMSS standards.
2006

Determine a process and schedule for an independent program evaluation.

Action taken, but not completed Blue Ribbon Evaluation Panel met 10/06 and produced a report. Independent Volunteer Program Evaluation completed 2/07 Seventeen pilot park evaluation studies identified and asked for $875,000 and 1 FTE in FY 2010.
2006

Develop (jointly with the Forest Service) a measure on how parks contribute to societal health by encouraging outdoor recreation.

No action taken
2006

Demonstrate how NPS holds superintendents and other managers accountable for achieving program results, such as by incorporating into a manager's personnel evaluation a set of specific performance targets that "cascade" down from program goals.

Action taken, but not completed Performance goals have been added to each employee's evaulation. These goals are tied to the supervisor's performance plan. Goals will be used in FY 2008.

Completed Program Improvement Plans

Year Began Improvement Plan Status Comments
2006

Demonstrate how NPS uses performance measures to improve management, adjust priorities, and allocate resources.

Completed Performance goals added to each employee evaluation. Tied to the supervisor's performance plan.
2006

Establish baseline and targets for measure on customer satisfaction with value for fee paid.

Completed A baseline of 92% was established in FY05. Targets and actuals reported in FY2008 Green book.

Program Performance Measures

Term Type  
Long-term Outcome

Measure: Visitor Understanding - Percent of visitors that understand and appreciate the significance of the park they are visiting


Explanation:Average result of annual visitor survey card (VSC) surveys.

Year Target Actual
2003 86% 86%
2004 86% 88%
2005 86% 88%
2006 86% 89%
2007 87% 86%
2008 89%
2009 90%
2010 90%
2011 91%
2012 92%
Annual Output

Measure: Customer satisfaction with value for fee paid


Explanation:Baseline and targets to be established FY05

Year Target Actual
2005 Est. Baseline 92%
2006 92% 92%
2007 92% 92%
2008 92%
2009 92%
2010 92%
2011 92%
2012 92%
Long-term Outcome

Measure: Visitor Safety (fatalities) - Number of fatalities annually in national parks


Explanation:From 1971-2002, parks had an average of 154 visitor fatalities annually. Goal is to reduce by 3% annually.

Year Target Actual
2000 110 110
2001 110 108
2002 110 128
2003 110 116
2004 110 106
2005 107 180
2006 175 148
2007 193 171
2008 190
2009 206
2010 210
2011 200
2012 200
Annual Outcome

Measure: Visitor Satisfaction - Percent of visitors satisfied with appropriate park facilities, services and recreational opportunities


Explanation:Average result of annual visitor survey card (VSC) surveys.

Year Target Actual
2003 95% 96%
2004 95% 96%
2005 95% 96%
2006 95% 96%
2007 95% 96%
2008 96%
2009 96%
2010 96%
2011 97%
2012 97%
Long-term Outcome

Measure: Public Health -- Percent of park visits that involve physical activity.


Explanation:Demonstrates how parks can contribute to better societal health by encouraging outdoor recreation. Need to establish baseline and targets.

Year Target Actual
2005
Annual Output

Measure: Increased visitation levels at targeted sites.


Explanation:Demonstrates increased public awareness of lesser known recreation areas. Need to establish baselines and targets

Year Target Actual
2005
2006
Annual Efficiency

Measure: Cost of collection as a percent of total revenue collected


Explanation:Compiled annually for NPS fee management purposes

Year Target Actual
1997 15-20% 13.90%
1998 15-20% 19.60%
1999 15-20% 20.40%
2000 15-20% 20.70%
2001 15-20% 21.50%
2002 15-20% 21.90%
2003 15-20% 22.09%
2004 15-20% 21.79%
2005 25% 21.3%
2006 25% 19.9%
2007 25% 18.3%
2008 24%
2009 23%
2010 23%
2011 22%
2012 21%
Annual Efficiency

Measure: Cost per interpretive visitor contact


Explanation:The cost of providing an interpretive contact per per person is adjusted annually for inflation.

Year Target Actual
1999 none $0.34
2000 $0.35 $0.38
2001 $0.39 $0.42
2002 $0.42 $0.54
2003 $0.55 $0.59
2004 $0.59 $0.60
2005 $0.60 $0.74
2006 $0.60+inflation $0.75
2007 $0.75 $0.60
2008 $0.75
2009 $0.75
2010 $0.75
2011 $0.75
2012 $0.75
Annual Output

Measure: Facilitated Programs - Attendance at facilitated programs in millions


Explanation:Compiled annually for Servicewide Interpretive Report (SIR) since FY99

Year Target Actual
2003 118 118
2004 118 147
2005 150 155
2006 153 155
2007 156 155
2008 161.7
2009 170.0
2010 174.0
2011 178.0
2012 181.0

Questions/Answers (Detailed Assessment)

Section 1 - Program Purpose & Design
Number Question Answer Score
1.1

Is the program purpose clear?

Explanation: The organic act of 1916 created the National Park Service with the purpose to conserve park resources and "provide for enjoyment of the same in such manner and by such means as will leave them unimpaired for future generations." The Visitor Services program fulfills this purpose by providing safe recreational opportunities and helping the public understand the meaning and relevance of park resources.

Evidence: NPS Organic Act (1916). NPS Management Policies (2001). NPS Director's Orders 6 and 83. Harpers Ferry Center's mission statement.

YES 20%
1.2

Does the program address a specific and existing problem, interest, or need?

Explanation: Parks have almost 300 million visitors annually. This program provides visitors the information they need to have a safe, enjoyable visit and a better appreciation of park resources. Public health and safety programs have helped to reduce visitor fatalities, even while visitation increases. Visitor services have contributed to consistently high visitor satisfaction levels.

Evidence: NPS Organic Act (1916). NPS budget justification (FY06). NPS Director's Order 52A. National Leadership Council's Mission Statement for the National Park Service

YES 20%
1.3

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: No other program has responsibility for serving visitors in national parks. Other Federal agencies may provide similar services, but on different lands with different objectives and responsibilities (e.g., multiple-use management). Some State and local entities may have a similar mission to the NPS, but not for Federal lands, for the Nation as a whole, or for resources deemed nationally significant by law or proclamation.

Evidence: Document: "Excellence in NPS Interpretation. NPS Organic Act (1916). NPS Budget justification (FY06). NPS has sole jurisdiction over national parks and sole responsibility for providing services to national park visitors.

YES 20%
1.4

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: Program is free of major design flaws, as illustrated by consistently high visitor satisfaction ratings. NPS successfully leverages resources through the use of volunteers and seasonal employees.

Evidence: Visitor satisfaction surveys. Volunteer in Parks (VIP) reports. Comprehensive Interpretive Plans.

YES 20%
1.5

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: Program is generally designed well, with services provided through a variety of means to park visitors and, via internet, to non-visitors. Health and risk management programs focus on providing as safe a visiting environment as possible. Interpretive and educational programs focus on helping visitors to understand the significance of park resources. NPS can do more to examine the costs and benefits of different interpretive techniques, such as waysides, guided walks, and exhibits.

Evidence: NPS visitor surveys show consistently high satisfaction levels and overall understanding of park resources. Website recorded nearly 400 million hits in 2004, up from 112 million in 2001. Visitor fatalities have declined from an average of 154 annually from 1971 to 2002 to 106 in 2004.

YES 20%
Section 1 - Program Purpose & Design Score 100%
Section 2 - Strategic Planning
Number Question Answer Score
2.1

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: Existing long-term measures focus on three basic outcomes: (1) visitor satisfaction; (2) visitor safety; and (3) visitor understanding for park resources. NPS is developing a fourth measure on how parks contribute to societal health by encouraging outdoor recreation. All support the overarching goal of making the parks fully accessible for the public to enjoy safe and meaningful visits.

Evidence: Visitor satisfaction and visitor understanding are measured through annual surveys (see Visitor Survey Card annual summaries) and compiled in various reports (e.g., Servicewide Interpretive Report). All measures are tracked through the Performance Measure Database System (PMDS).

YES 12%
2.2

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: NPS has established targets for all three existing long-term measures. Improvement in visitor satisfaction will be difficult, given the high levels currently achieved, but NPS does have targets to increase visitor safety and visitor understanding. A new reporting system in 2005 will capture visitor accident/incident information at 388 park units more accurately and allow NPS to set targets for reducing visitor injury and fatality rates more effectively.

Evidence: Visitor Survey Card (VSC) annual summaries. PMDS reports. Servicewide Interpretive Report (SIR). Visitor Services Project (VSR) overview. New reporting system for visitor safety is the Incident Management Analysis and Reporting System (IMARS).

YES 12%
2.3

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: Annual measures directly support the long-term goals listed above. Although NPS can measure outputs (e.g., number of visitors contacted by park staff or participating in facilitated programs), it needs to better demonstrate that it uses these measures to improve management. Visitor satisfaction and visitor understanding measures would be more effective as annual measures (in addition to serving as long-term measures) if they could more responsively demonstrate the link between specific NPS actions and subsequent changes in visitor responses.

Evidence: Visitor Survey Card (VSC) annual summaries. PMDS reports. Servicewide Interpretive Report (SIR). Visitor Services Project (VSR) overview. NPS crosswalk to DOI strategic plan measures. (Visitor satisfaction and visitor understanding measures are both annual and long-term measures, but are shown as only one for the purposes of this PART.)

YES 12%
2.4

Does the program have baselines and ambitious targets for its annual measures?

Explanation: NPS has established targets for all annual measures, except customer satisfaction with value for fee paid. Although NPS can measure outputs, it needs to better demonstrate that it uses these measures to improve management. Visitor satisfaction and visitor understanding measures would be more effective as annual measures (in addition to serving as long-term measures) if they could more responsively demonstrate the link between specific NPS actions and subsequent changes in visitor responses.

Evidence: Visitor Survey Card (VSC) annual summaries. PMDS reports. Servicewide Interpretive Report (SIR). Visitor Services Project (VSR) overview. NPS crosswalk to DOI strategic plan measures. (Visitor satisfaction and visitor understanding measures are both annual and long-term measures, but are shown as only one for the purposes of this PART.)

YES 12%
2.5

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: Program has not demonstrated how it links partner activities to program goals through performance requirements, such as performance-based service contracts. NPS has many partners, including more than 140,000 volunteers, 64 cooperating associations, and various universities and associations. Yet, NPS has not shown how partners measure and report on their performance as it relates to accomplishing program goals.

Evidence: No sufficient evidence provided.

NO 0%
2.6

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: NPS has not conducted non-biased evaluations on a regular basis to fill in gaps in performance information or improve the effectiveness of the program. Although the program collects survey information, that constitutes its primary performance measure and is not an evaluation of program effectiveness. Park-specific evaluations lack sufficient scope to evaluate the program overall. NPS should contract or authorize an outisde entity to conduct an independent evaluation.

Evidence: Visitor Service project conducts park-specific evaluations of visitor services, but no evidence was provided of program evaluations that are of sufficient scope, quality, and independence.

NO 0%
2.7

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: NPS lacks effective budgeting that defines the link between performance targets and budget resources. Although budget justifications cite performance goals, they do not make clear the impact of funding decisions on expected performance. Nor does NPS report all the direct and indirect costs needed to attain projected results.

Evidence: NPS 2006 Budget justifications

NO 0%
2.8

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: Steps to correct strategic planning deficiencies include: (1) requiring each park to have a current Comprehensive Interpretive Plan with annual and long-term work plans and targets; (2) developing additional measures to track efficiency and promote outdoor recreation; (3) creating a new IT system (IMARS) for incident reporting; and (4) a long-term strategic education plan and the formation of an educational council.

Evidence: IMARS overview. Comprehensive Interpretive Planning. Rethinking the National Parks for the 21st century. Renewing our Education Mission. A Comprehensive Study of Visitor Safety in the National Park Service, pp71-72.

YES 12%
Section 2 - Strategic Planning Score 62%
Section 3 - Program Management
Number Question Answer Score
3.1

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: NPS collects credible performance information, but has not demonstrated how the agency uses it to manage the program and improve performance. NPS has not shown how it systematically uses visitor satisfaction and visitor understanding measures to adjust program priorities, allocate resources or take other appropriate management actions. Nor has it provided examples of steps taken to enact improvements cited by a specific evaluation. NPS is developing systems, such as IMARS, to better track and evaluate progress in visitor safety, but that has not been implemented.

Evidence: GPRA Survey results (FY03). Comprehensive Study of Visitor Safety in the NPS (2002). IMARS announcement (2004). NPS FY05 and FY06 Budget justifications. Visitor Service Progam summary report. Servicewide Interpretive Report summary (SIR)(2004).

NO 0%
3.2

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: NPS has not demonstrated that it holds superintendents and other managers responsible for achieving program results. Personnel performance evaluations do not incorporate program performance with clearly defined and quantifiable performance targets for each manager. Some evaluations have workload measures, but they do not "cascade" down from program goals. NPS has not shown how agreements with volunteers or cooperating associations incorporate and directly contribute to program performance goals.

Evidence: Illustrative personnel evaluations site program performance goals and list specific workload measures for a given manager, but they do not show how one links to the other. Illustrative cooperative agreements list some requirements, but do not show how they contribute to program performance goals.

NO 0%
3.3

Are funds (Federal and partners') obligated in a timely manner and spent for the intended purpose?

Explanation: All visitor service functions prepare annual financial plans to allocate and track expenditures against intended use. Nearly all funds are available for only one year, so obligations are made in a timely manner. Program uses a project agreement system to outline the products, schedule, budget, and roles and responsibilities involved with projects, such as exhibits built by Harpers Ferry Center.

Evidence: Table showing enacted vs. obligated history (FY00-FY04). Servicewide Interpretive Report summary (FY04). Example of Harpers Ferry Center project tracking sheet.

YES 14%
3.4

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: NPS efficiency measures show that the costs per visitor contact and the cost of fee collections are going up (mainly due to higher labor costs), but NPS is using volunteers and other steps to leverage resources and limit the rate of growth. NPS also gathers data annually to track the cost of full and part time NPS personnel in providing interpretive services, and makes appropriate personnel adjustments. NPS has also made IT investments, such as Harpers Ferry Center's project tracking system, to better measure project performance. NPS still needs to do more benchmarks against similar programs and competitive sourcing or other cost comparisons. It must also scrutinize those parks with above-average fee collection costs to find ways to allocate more fee receipts toward projects.

Evidence: Volunteer-in-Parks annual activity and expense report. Harpers Ferry Center project tracking information system. Servicewide Interpretive Report (SIR) (2003-2004). See efficiency measures on cost per interpretive visitor contact and cost of fee collection.

YES 14%
3.5

Does the program collaborate and coordinate effectively with related programs?

Explanation: NPS collaborates frequently with other Federal, State, and local providers of visitor services. Examples that involve meaningful actions include: Recreation One-Stop, an e-gov initiative to improve access to recreation information; a consolidated reservation system for Federal agencies providing campgrounds and other services; Volunteer.gov, a web-based recruitment resource for public agencies; Lewis and Clark Bicentennial celebration; and Blue Ridge Parkway regional destination center. All of these involve joint planning or resource allocations.

Evidence: See www.recreation.gov or www.volunteer.gov. Project Agreement for Blue Ridge Parkway regional destination center. Lewis and Clark webpage at www.nps.gov/lecl.

YES 14%
3.6

Does the program use strong financial management practices?

Explanation: NPS has "several weaknesses" in its financial management systems and has reportable conditions on its financial reporting controls, according to its FY04 financial audit. NPS lacks the ability to integrate financial and performance systems to support day-to-day operations, so these weaknesses have a direct relation to the visitor services program.

Evidence: FY04 NPS financial audit. NPS historically has had weak financial management practices, which adversely affects management in many program areas.

NO 0%
3.7

Has the program taken meaningful steps to address its management deficiencies?

Explanation: The program has had various internal and external reviews, including a 2002 review by private consultants of the Harpers Ferry Center core business and processes and a 2003 GAO report on cooperating associations and other non-profit partners. NPS has under review a draft Director's Order for risk management and a new strategic plan for public health programs. NPS has also created a new certification program for interpreters.

Evidence: "Aiming for Excellence" HFC Strategic Plan (2002). "Connecting People to Parks" Interpretation and Education FY05 work plan. GAO report on cooperating agencies and nonprofit partners (GAO-03-585, 2003). Public Health Service strategic plan (2004). Director's Order #50 on public risk management programs (2005 draft).

YES 14%
Section 3 - Program Management Score 57%
Section 4 - Program Results/Accountability
Number Question Answer Score
4.1

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: NPS is on track to meet its long-term goals for visitor services. The current high levels of visitor satisfaction (95%) and visitor understanding (86%) demonstrate that nearly all visitors are pleased with the services being offered and appreciate the significance of the park resources being protected. Visitor fatalities are also below historic averages, which suggests safer visits as well.

Evidence: Table on visitor satisfaction levels and visitor survey card reports (1998-2004). Crosswalk on NPS/DOI measures. Table on number of NPS fatalities (1971-2004)

YES 25%
4.2

Does the program (including program partners) achieve its annual performance goals?

Explanation: NPS met the annual goals for facilitated programs, visitor satisfaction, and visitor understanding, but still needs to develop output measures for increasing visitation levels at targeted sites and customer satisfaction with value for fee paid. NPS also needs to implement its planned IT system (IMARS) for collecting data on visitor fatalities, injuries, and illnesses. It must also establish performance goals for the recently established risk management program.

Evidence: Table on visitor satisfaction levels and visitor survey card reports (1998-2004). Crosswalk on NPS/DOI measures. Table on number of NPS fatalities (1971-2004). Interpretation and Education annual measures (2004-2008). IMARS announcement (2004). (Visitor satisfaction and visitor understanding are both annual and long-term measures, but are shown as just one for the purposes of this PART.)

LARGE EXTENT 17%
4.3

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: The cost per visitor contact and the cost of fee collections have gone up, mainly due to higher labor costs, but NPS has taken steps to control the rate of growth, such as increasing by 5% annually the number of volunteer hours donated. NPS estimates (based on independent estimates of the value of donated labor) that contributions from volunteers have increased from $72 million in FY02 to $86 million in FY04. NPS still needs to do more competitive sourcing or other cost comparisons. It must also scrutinize those parks with above-average fee collection costs to find ways to allocate more fee receipts toward projects.

Evidence: Facilitated program summary (FY03-04). Volunteer-in-Parks summary (FY02-08). NPS visitation summary. See efficiency measures on cost per interpretive visitor contact and cost of fee collection.

LARGE EXTENT 13%
4.4

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: NPS pioneered the delivery of visitor services by land management agencies. A comparison of visitor satisfaction surveys with other US and Canadian agencies shows that the national park visitor generally is more satisfied with the quality of services, although this could also reflect the relatively higher satisfaction with national park resources. This is difficult to determine, given that no benchmark comparison has been completed by an independent entity. NPS has also developed the Interpretive Development Program (IDP), a course on professional standards for interpretation, that has drawn the interest of other Federal and state agencies, universities, professional and international organizations.

Evidence: Visitor satisfaction surveys for the U.S. Fish and Wildlife Service, U.S. Forest Service, British Columbia Provincial Parks and Parks Canada. Interpretive Development Program summary (www.nps.gov/idp/interp). In 2000 the National Association of Interpretation adopted language from the IDP standard as its definition of interpretation.

YES 20%
4.5

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: NPS has not conducted non-biased evaluations on a regular basis to improve the effectiveness of the program. Although the program collects survey information, that constitutes its primary performance measure and is not an evaluation of program effectiveness. Park-specific evaluations lack sufficient scope to evaluate the program overall. A 2004 GPO report did evaluate the relationship between NPS and cooperating agencies, but that is only a small piece of visitor services overall. NPS should contract or authorize an outside entity to conduct an independent evaluation.

Evidence: Visitor Service Project conducts park-specific evaluations of visitor services, but no evidence was provided of program evaluations that are of sufficient scope, quality, and independence. GAO report on cooperating agencies and nonprofit partners (GAO-03-585, 2003)

NO 0%
Section 4 - Program Results/Accountability Score 75%


Last updated: 09062008.2005SPR