Detailed Information on the
Rural Health Activities Assessment

Program Code 10001065
Program Title Rural Health Activities
Department Name Dept of Health & Human Service
Agency/Bureau Name Health Resources and Services Administration
Program Type(s) Competitive Grant Program
Assessment Year 2003
Assessment Rating Adequate
Assessment Section Scores
Section Score
Program Purpose & Design 60%
Strategic Planning 75%
Program Management 70%
Program Results/Accountability 58%
Program Funding Level
(in millions)
FY2007 $160
FY2008 $167
FY2009 $17

Ongoing Program Improvement Plans

Year Began Improvement Plan Status Comments

Develop health and quality-related annual performance measures to demonstrate program accomplishments.

Action taken, but not completed Data collection instruments for performance measures for the Community-based programs are pending OMB Paperwork Reduction Act approval. The program is preparing Flex and State Office of Rural Health program material for submission to OMB for approval. (June 08 update)

Explore development of a possible alternative to the current approved efficiency measure for Rural Health Activities.

Action taken, but not completed Such exploration will be dependent upon new data to be collected from grantees, following approval of data collection by OMB. These data will be assessed to determine if an alternative efficiency measure can be developed. (June 08 update)

Completed Program Improvement Plans

Year Began Improvement Plan Status Comments

The Administration will continue to monitor progress toward data gathering for the newly developed long-term and annual performance goals.


Articulate long-term and annual performance measures to all partners to achieve a commitment toward goals.


Program Performance Measures

Term Type  
Long-term Output

Measure: Increase percentage of critical access hospitals with positive operating margins.

Explanation:The overarching goal is to increase the financial viability/sustainability of small rural hospitals.

Year Target Actual
1999 NA 10%
2010 58%
2013 60%
Annual Output

Measure: Increase by 0.5 percentage point annually the average operating margin of critical access hospitals

Explanation:To be a CAH, a hospital must: 1) Maintain no more than 15 acute care beds and up to 10 swing beds; 2) Keep patients hospitalized no longer than 96 hours; 3) Provide 24 hour emergency care; and 4) Be designated by the state. Some targets may be shown as NA (not applicable) because the Rural Hospital Flexibility Grants program is not proposed to continue. The FY 07 target was changed to reflect funding provided in the full-year Continuing Resolution for FY 07.

Year Target Actual
1999 NA -14.05%
2002 +0.5 percentage pt. -9.37%
2003 +0.5 percentage pt. -10.5%
2004 +0.5 percentage pt. -10.2%
2005 +0.5 percentage pt. -9.6%
2006 +0.5 percentage pt. -8.8%
2007 +0.5 percentage pt. Sept. 08
2008 +0.5 percentage pt. Sept. 09
2009 NA NA
2010 +0.5 percentage pt.
Long-term Outcome

Measure: Reduce the proportion of rural residents of all ages with limitation of activity caused by chronic conditions

Explanation:The overarching goal is to address health disparities in rural areas by increasing the health and wellness of people living in rural communities.

Year Target Actual
2002 NA 14.6%
2010 13.5%
2013 13.0%
Annual Output

Measure: Increase the number of people served through outreach grants

Explanation:Some targets may be shown as NA (not applicable) because the Outreach Grants program is not proposed to continue. The FY 07 target was changed to reflect funding provided in the full-year Continuing Resolution for FY 07

Year Target Actual
2002 NA 673,700
2003 NA 668,810
2004 675,498 655,257
2005 682,253 776,880
2006 675,300 627,120
2007 777,000 Oct. 08
2008 785,000 Oct. 09
2009 NA NA
2010 798,000
Annual Efficiency

Measure: Increase the return on investment of funds by the Rural Hospital Flexibility (FLEX) grant program, as measured by change in total operating margin of critical access hsopitals in relation to FLEX dollars invested. (New measure, added February 2007).


Year Target Actual
2004 Baseline 23.36%
2005 24% 14.8%
2006 24.5% Sept. 08
2007 25% Sept. 09
2008 26% Sept. 10
2009 NA NA
2010 28%

Questions/Answers (Detailed Assessment)

Section 1 - Program Purpose & Design
Number Question Answer Score

Is the program purpose clear?

Explanation: The Health Resources and Services Administration's (HRSA) Office of Rural Health Policy advises the Secretary on the effects of current policies and proposed statutory, regulatory, administrative, and budgetary changes on rural areas. The Office also oversees Outreach Grants expanding access to, coordinate, and improve quality of health care services. Rural Health Network Development Grants encourage providers to partner in formal networks to integrate administrative, clinical, financial, and technological functions across organizations. State Offices of Rural Health funds operation of these offices. Rural Access to Emergency Devices provides grants to community partnerships to purchase equipment and provide defibrillators and basic life support training. Rural Hospital Flexibility Grants to states help stabilize and improve access to services and develop and implement state rural health plans. The Small Hospital Improvement Program helps these hospitals implement the prospective payment system, comply with HIPAA, and improve hospital performance. Denali Commission funds are used to construct primary health care facilities in Alaska.

Evidence: Section 711 of the Social Security Act (42 USC 912) authorizes HRSA's Office of Rural Health Policy. Included is the authorization for the programs it oversees: 1) Outreach Grants Section 330A of the Public Health Service Act (42 USC 254c) 2) Rural Health Network Development Grants Section 330A of the Public Health Service Act (42 USC 254c) 3) State Offices of Rural Health Section 338J of the Public Health Service Act (42 USC 254r) 4) Rural Access to Emergency Devices Public Law 106-505 Subtitle B, Section 411-413 5) Rural Hospital Flexibility Grants Section 1820(c) of the Social Security Act (42 USC 1395i-4) 6) Small Hospital Improvement Program Section 1820(g)(3) of the Social Security Act (42 USC 1395i-4) 7) Denali Commission Public Law 105-277, Section 304

YES 20%

Does the program address a specific and existing problem, interest, or need?

Explanation: Approximately 65 million Americans reside in rural areas, of which the Rural Policy Research Institute (see Evidence/Data column) estimates that approximately 7 million live in poverty (25 percent higher than in urban areas). Non-elderly people living in rural poverty are more likely than their urban counterparts to lack health insurance. Population shifts over the last decade from urban to rural areas has changed the racial and ethnic makeup of communities. Many growing rural counties are experiencing concurrent growth in the diversity of its residents and in general rural areas have a higher proportion of elderly residents, primarily in the South and Midwest. Minorities often move to distinct rural communities where poverty is high and opportunity is low and in general the elderly use more health services than the non-elderly. Cigarette use by adolescents ages 12-17 in 1999 is higher in rural areas (19%) than urban areas (11%), adults living in rural counties are most likely to smoke (27% of women and 31% of men in 1997-1998), and the percent of women with obesity is highest in rural counties (23%). These trends illustrate the health disparities that exist in rural areas.

Evidence: 1) www.rupri.org The Rural Policy Research Institute provides objective analysis and facilitates public dialogue concerning the impacts of public policy on rural people and places. 2) CDC/NCHS Urban and Rural Health Chartbook 2001

YES 20%

Is the program designed so that it is not redundant or duplicative of any Federal, state, local or private effort?

Explanation: Redundancy and duplication exist. More than one program across the Department addresses the same problem, interest, or need--rural health. In July 2001, the Secretary of HHS charged all agencies to examine ways to improve and enhance health care in rural areas. HHS created a Rural Task Force, which identified more than 225 health and social services programs within HHS of which: 33% provide grants for which rural communities can directly apply (including IHS programs), 25% are block grants or other funding to States, and 42% are funding to national organizations, academic institutions, and Congressionally-mandated projects. Within this array of programs there are clearly some programs that consistently reach into rural communities, most notably the HRSA Community Health Centers (27% in rural zip codes), IHS, CMS, and programs administered by SAMHSA and the AoA. Efforts are in place to help minimize duplication. Applicants are required by law to note any other sources of federal funding and to distinguish how it is being used in a manner that would alleviate concerns about duplicate or redundant financial support. The majority of the Office's funding (75%) is used for activities that would not overlap with other HHS resources.

Evidence: HHS Rural task Force Report to the Secretary, July 2002

NO 0%

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: The major flaw of the Office's portfolio stems from the programs' authorization. The Office's portfolio consists of seven programs that each focus on a small part of the total. A less stovepipe and more seamless effort in rural areas could help maximize access, generate effectiveness, yield cost efficiencies, and reduce the number of specific projects and geographically targeted projects funded each year.

Evidence: HHS Rural task Force Report to the Secretary, July 2002

NO 0%

Is the program effectively targeted, so program resources reach intended beneficiaries and/or otherwise address the program's purpose directly?

Explanation: The Office's programs are specifically designed to address health needs in rural communities. Through demonstrations the Office supports creative models of outreach and offers flexibility for rural communities to identify needs. The Office also focuses on the smallest most vulnerable rural hospitals through the Flex and Small Hospital Improvement programs.

Evidence: The Offices Small Hospital program has assisted more than 700 of the smallest, most vulnerable hospitals

YES 20%
Section 1 - Program Purpose & Design Score 60%
Section 2 - Strategic Planning
Number Question Answer Score

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: OMB and HRSA recently developed two long-term output goals that link to the mission of the program.

Evidence: 1) FY 2005 GPRA Plan 2) See "Measures" tab for the long-term goals

YES 12%

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: When developing these long-term goals, specific attention was paid to highlighting baseline data and ensuring ambitious targets.


YES 12%

Does the program have a limited number of specific annual performance measures that demonstrate progress toward achieving the program's long-term measures?

Explanation: OMB and HRSA recently developed two annual output and outcome goals that demonstrate progress toward achieving the long-term goals for patient safety activities.

Evidence: 1) FY 2005 GPRA Plan 2) See "Measures" tab for the annual goals

YES 12%

Does the program have baselines and ambitious targets and timeframes for its annual measures?

Explanation: When developing these annual goals, specific attention was paid to highlighting baseline data and ensuring ambitious targets.


YES 12%

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, etc.) commit to and work toward the annual and/or long-term goals of the program?

Explanation: The overarching long-term goals have not been articulated in RFAs, contracts, cooperative agreements, or interagency agreements. RFAs are written to include themes, but themes are not identical to those goals laid out for the program. Project Officers use these themes as they perform their annual site visits with each grantee.


NO 0%

Are independent and quality evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: In 2002, the University of Minnesota Rural Health Research Center conducted an evaluation of the long-term success of the Rural Health Outreach Program. It evaluated 104 former grantees whose projects started in 1994 or 1996 and examined whether services implemented with Outreach program funds continue to be provided three-five years after funding ended. In addition, three program assessments have been conducted on the Network Development Grant Program. The assessments studied network organizational structure, management, financing services, leadership, and sustainability.

Evidence: 1) University of Minnesota Rural Health Research Center Evaluation 2) TA Contractor for Rural Health Network Development Grant Program

YES 12%

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: Prior to the recent development of overarching long-term and annual goals, the program did not have clear and articulated performance goals they drove the budget formulation process. As a result, budget requests were not developed to request funding levels designed to achieve performance.


NO 0%

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: HRSA attempts to hold all parties accountable by specifying annual goals in contracts goals negotiated with the contractor as part of their performance based contract plans. Contractors are required to commit to tasks contributing to those performance goals and file reports by phone weekly, and written monthly and annual reports. If progress is judged as insufficient agreements may be terminated. In addition, the Office will add to all of its program guidance for the 2005 cycle information about its strategic plan and its long-term and annual performance goals for the program. This will provide grantees the necessary context to understand the Office's overarching goals of increasing the health and wellness of people living in rural communities and ensuring the viability and sustainability of rural hospitals.


YES 12%
Section 2 - Strategic Planning Score 75%
Section 3 - Program Management
Number Question Answer Score

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: The Office independently evaluates all of its programs once they have been implemented long enough to gain experience and uses that information to revise and improve program guidance and management. Program guidance for all programs is assessed annually and refined to reflect compliance with the authorizing statutes, address any valid concerns of grantees over administrative burden and to protect program integrity. In addition, the Office regularly convenes project officers at the conclusion of each funding cycle to review the past year's activities, identify program strengths and weaknesses and develop strategies for addressing weaknesses. The Office then works with Grants Management personnel in making any needed changes. In making contracts, the Office reviews each contract quarterly and requires project officers to ensure that tasks are carried out in a timely manner consistent with the contract requirements.


YES 10%

Are Federal managers and program partners (grantees, subgrantees, contractors, cost-sharing partners, etc.) held accountable for cost, schedule and performance results?

Explanation: For the first time in FY 2002, each Office grant program manager created a strategic plan. As part of each employee's mid-year and annual performance review, they are assessed on their administration of the particular grant program they work with and on any contracts for which they served as Project Officer. This includes compliance with timelines developed jointly by management and staff and for use of resources and ensuring that grants are awarded appropriately. Staff performance ratings also hinge on their work as Project Officers. The Office is required to adequately review all contracts on a quarterly basis to ensure contractors are meeting deadlines and adhering to the requirements of the contract. For each of the Office's grant programs, Project Officers are required to perform non-competing continuation reviews of grantees annually. In those situations where a problem with a grantee arises, the Office conducts an inquiry into whether or not the problem should have been identified in the course of the annual non-competing continuation review and corrective actions are taken as necessary.


YES 10%

Are all funds (Federal and partners') obligated in a timely manner and spent for the intended purpose?

Explanation: Since the inception of these programs all funds were obligated and disbursed in a timely manner, following specific legislative requirements. HRSA monitors grantee expenditures to ensure compliance with legislation, regulation and policies.

Evidence: 1) Estimated obligations by quarter in apportionments for FYs 2001-2003 2) Actual obligations by quarter for FYs 2001-2003

YES 10%

Does the program have procedures (e.g., competitive sourcing/cost comparisons, IT improvements, approporaite incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: The program does not have procedures in place to measure and achieve efficiencies and cost effectiveness. In addition, the program's performance plan does not include efficiency measures and targets that address such things as per unit cost of care and/or treatment or other measures directly linked to the mission of the program.


NO 0%

Does the program collaborate and coordinate effectively with related programs?

Explanation: The Office works with HRSAs Health Professions training programs and CMS on options for providing technical assistance or potential grants to rural communities interested in using the Medicare PACE model (Program of All-Inclusive Care for the Elderly). The Office will jointly issue a contract to provide some technical assistance on this issue to rural communities in August 2003. The Office also works with HUD in its administration of the 242 Capital program to provide an avenue for Critical Access Hospitals (CAHs) to gain access to the capital markets. As a result of this collaboration, the HUD program has created special rules that take into account the small scale of CAHs with a refined application process. HRSA also works cooperatively with IHS to assist with the predominant number of American Indian and Alaska Natives living in isolated rural areas.


YES 10%

Does the program use strong financial management practices?

Explanation: The September 30, 2002 and 2001 independent auditor's report identifies five reportable conditions. 1) Preparation and analysis of financial statements - HRSA's process for preparing financial statements is manually intensive and consumes resources that could be spent on analysis and research of unusual accounting. 2) HEAL program allowance for uncollectible accounts ' HRSA's financial statements indicate limited success in collecting delinquent HEAL loans. 3) Federal Tort Claims Liability ' HRSA is unable to estimate its malpractice liability under the Health Centers program. 4) Accounting for interagency grant funding agreements ' HRSA's interagency grant funding agreement transactions are recorded manually and are inconsistent with other agencies' procedures. 5) Electronic data processing controls ' HRSA has not developed a disaster recovery and security plan for its data centers. Although HRSA's rural health programs have not been cited specifically by auditors for material weaknesses, the above reportable conditions constitute weaknesses within HRSA and its Office of Financial Integrity. The Office reports directly to the Administrator and is intended to ensure procedures are in place to provide oversight of all of HRSA's financial resources.

Evidence: 1) CORE Accounting Form 2) HRSA Office of Financial Integrity description 3) HRSA FY 2001-2002 Annual Reports

NO 0%

Has the program taken meaningful steps to address its management deficiencies?

Explanation: HRSA is streamlining its grants operations and increasing efficiency through an electronic grant application process; the Office will be part of that transition. In addition, for the 2004 cycle for Outreach and Network grants, the Office has begun an initial letter-of-intent requirement. The previous requirement only asked applicants to let the State Office know an applicant was applying at the time of submission. State Office representatives noted that this was too late in the process to identify situations where applicants from the same community might be applying for funds for similar or overlapping projects. State Offices can now provide more assistance on the front end in and identify potential areas of overlap in terms of proposals. HRSA also developed a corrective action plan to address the reportable conditions identified in the September 30, 2002 and 2001 independent auditor's report. For each aspect of the five reportable conditions, HRSA assigned an office responsibility. The plan also outlines milestones and target completion dates.

Evidence: HRSA Corrective Action Plan for FY2002 Financial Statement Audits as of 4/30/2003.

YES 10%

Are grants awarded based on a clear competitive process that includes a qualified assessment of merit?

Explanation: Program applications for nationally announced competitive grant cycles are reviewed by objective review committees. The committees review the project plan and budget based on criteria announced publicly in the application guidance. Funding decisions are made based on committee assessment, relative need, announced funding preferences, program priorities, and, beginning in FY 2004, periodic on-site reviews. The Outreach and Network development grants are time-limited demonstration grants for three years. The Office announces new grants under the HRSA Preview announcement and encourages new and first-time applicants to apply. State Offices of Rural Health encourage communities to apply for these grant programs. Technical assistance is made available through the State Offices and directly to any entity seeking assistance with the process.


YES 10%

Does the program have oversight practices that provide sufficient knowledge of grantee activities?

Explanation: Program and project officers review grantee continuation applications. Award recipients submit audits that are appropriate for their type of organization and level of funding. All grantees submit quarterly cash transaction reports indicating the current amount of cash spent to the Payment Management Office. Grantees also provide a yearly Financial Status Report to the Office of Grants Management Operations which identifies the amount of Federal funds spent for the budget period and how much is unobligated. The original application and progress reports are reviewed for information on how grant funds will be spent. The program staff identifies areas where problematic expenditures are noted and contacts the grantee for explanation and correction if necessary. There have been very few instances where funds have been expended outside of their intended purpose. The Agency is developing an integrated performance review program for all of its programs, which will include site-evaluation of selected rural health grantees.


YES 10%

Does the program collect grantee performance data on an annual basis and make it available to the public in a transparent and meaningful manner?

Explanation: Data are not made available to the public in a transparent and meaningful manner. The Office does post some key data about the performance of its grantees on the web. In the past two years, the Network Development Grant program and the Outreach program developed source books of all grantees that include financial and narrative information. The Office will also begin systematically reviewing the number of hits on its web site and use that information to help refine the type and format of information that is available.


NO 0%
Section 3 - Program Management Score 70%
Section 4 - Program Results/Accountability
Number Question Answer Score

Has the program demonstrated adequate progress in achieving its long-term outcome performance goals?

Explanation: New measures have been developed and the Office will begin establishing baselines and quantifying the progress of rural hospitals. However, the Office has been monitoring for three years the financial performance data for its 353 hospitals that have been converted to critical access hospitals. Reports show that average operating margins for these hospitals has improved since 1996. Profit margins have increased from -4.1% in 1996 to 1% in 2000.

Evidence: The Rural Hospital Flexibility Program Tracking Project (February 2003 Report)


Does the program (including program partners) achieve its annual performance goals?

Explanation: The annual GPRA measures for the Outreach and Network grants established in FY 98 demonstrate incremental progress towards the long-term goal by providing access to services. From the base year of FY 98, when the program served 630,000 rural residents, the program has served more than 670,000 every year, with a peak year in 2000. The program has received level funding during that period. In FY 2002, the program served 673,700 rural residents.



Does the program demonstrate improved efficiencies or cost effectiveness in achieving program performance goals each year?

Explanation: The Flex, Outreach, Network, and Research grant programs have received level funding for the past three years. Despite this and increased expenses for grantees, these programs continue to maintain or expand services. Capacity building and infrastructure development are key Office activities. The Office maximizes its technical assistance capacity by working with the 50 State Offices of Rural Health to 'train the trainer' in grant writing, small hospital performance improvement, and economic modeling. In turn, these State Offices assist local communities to prepare grant applications, improve local hospital performance and networking, and determine those services that might be offered through local resources. In addition, the grant programs seek to develop networking and sustainable partnerships. Projects funded through the outreach grant program have demonstrated sustainability; nearly 90 percent of the grantees continues a significant portion of their activities three years after the end of the grant project period.

Evidence: University of Minnesota Rural Health Research Center Evaluation


Does the performance of this program compare favorably to other programs, including government, private, etc., that have similar purpose and goals?

Explanation: No other programs fund the wide array of activities funded by the Office.


NA 0%

Do independent and quality evaluations of this program indicate that the program is effective and achieving results?

Explanation: Evaluations of several office programs indicate that the Office is effective and achieving results in increasing access to services in rural communities. For example, the 2002 evaluation by the University of Minnesota Rural Health Research Center indicates that the majority of Outreach grantees surveyed continue to provide health services in rural communities. These services were made possible by initial support from the Office. In addition, ongoing assessments in the Network and Flex programs indicate that the strength and viability of rural health organizations and infrastructure increases.

Evidence: University of Minnesota Rural Health Research Center Evaluation

YES 25%
Section 4 - Program Results/Accountability Score 58%

Last updated: 09062008.2003SPR