Detailed Information on the
Administration on Aging Assessment

Conduct quality program evaluations of key AoA aging services programs by 2012.

Measure: By 2012, all States achieve a targeting index greater than 1.0 for rural, minority, and poverty clients.

Explanation:The purpose of this measure is to demonstrate continuous program improvement in targeting services to vulnerable elderly as required by the Older Americans Act. The targeting index is calculated as follows: rural targeting index = percent of Title III services recipients in State who live in rural areas divided by the percent of 60+ residents of State who live in rural areas. If the targeting index for rural clients is greater than 1.0 then the percentage of OAA rural clients in the State is greater than the percentage of rural elderly in the State, meaning that services have been effectively targeted to rural clients. For the baseline calculation, data for the numerator come from the FY 2001 State Reporting System (SPR) and data for the denominator come from Census 2000 data. Baseline (year 2001) targeting indices for all States have been developed as follows: poverty targeting (44 States achieved targeting index greater than 1.0); rural targeting (41 States achieved rural targeting greater than 1.0); and minority targeting (37 States achieved minority targeting greater than 1.0). The 2012 targets for this measure are: 1) 52 State poverty targeting indices >1; 2) 51 State rural targeting indices >1; and 3) 52 State minority targeting indices >1. Each State will be working toward 3 indices but the targets will be added for simplicity of presentation.

Measure: Increase the number of States that serve more elderly living below the poverty level then the prior year.

Explanation:The purpose of this measure is to increase the number of States that increase the percent of below poverty elderly that they serve. The data source is the SPR. Note: revisions to the SPR were implemented in 2005 which affected the manner in which poverty data is reported. States were previously required to report poverty data for all clients, which involved estimation for unregistered clients. States are now reporting actual poverty data for registered clients. This explains the apparent decline between FY 2004 and FY 2005. Ambitious performance targets have been established for this measure since poverty status can be a predictor of vulnerability to loss of independence.

Measure: The percentage of OAA clients living in rural areas who receive services funded by AoA will exceed the percentage of all U.S. elderly who live in rural areas by 10%.

Explanation:The purpose of this measure is to assure that the Aging Network continues to target services to rural residents, as required by the OAA. Percentages for this measure are calculated using SPR data and special tabulations of the 2000 Census data for the 60+ population. Living in rural areas may make access to needed services more difficult but it is not generally a predictor of vulnerability to a loss of independence. Therefore, AoA has established a national performance target that, while suitably ambitious, should not encourage States to divert resources away form more vulnerable populations.

Measure: Increase the number of elderly persons with severe disabilities (3+ Activities of Daily Living (ADL) limitations) receiving select home and community-based services.

Explanation:Since people with 3+ ADLs limitations are generally considered nursing home eligible this population is at great risk for loss of independence. The measure demonstrates AoA's successful efforts to promote the ability of frail and disabled elderly to continue living at home. Results for this performance measure are currently calculated using home-delivered meals client data from the annual performance measurement surveys. Eventually, using annual SPR data, this measure will be expanded to include all recipients of in-home services.

Measure: Increase the number of people served per million dollars of AoA's Title III home and community-bases services funding.

Explanation:The purpose of this measure is to demonstrate the success of the Aging Network in employing available tools (see Section 3.4) to enhance the use of AoA funds. This measure will be monitored in conjunction with consumer assessment of service quality to assure that increased efficiency does not result in declining service quality. Performance for this measure has improved significantly over the past three years as a result of information and performance measurement management improvements. Performance through 2010 is expected to improve at more modest rates with substantial increases resulting from successful program innovations by 2012.

Measure: Maintain high percentage of home-delivered meal, transportation and family caregiver services clients rating quality of services good to excellent.

Explanation:This measure, in conjunction with the efficiency measure above, will monitor service quality as increased efficiencies are realized, to assure that there is no decline in service quality. Data for this measure are obtained from the national performance measurement surveys. The confidence interval is 91.9% through 95.1% (93.3% +/- 1.6%). 91.7% is the lower confidence limit. Assuming future samples will have similar standard errors, an estimate would have to fall below 90% (actually 90.1%) before there would be no overlap in the confidence intervals. Therefore, the target of 90% is point at which a decline in quality can be detected.

Measure: Improve well-being and prolong independence for elderly individuals as a result of AoA's Title III State and Community-based services.

Explanation:The purpose of this measure is to demonstrate the success of State and Community-Based Services and program innovations in developing tools that enable the Aging Services Network to improve access to services for vulnerable populations and maintain those vulnerable elderly in their homes. Demonstrating improved well-being and prolonged independence from year to year is accomplished through the use of an OAA nursing home predictor score. To construct this score, characteristics predictive of nursing home placement were selected for four key OAA services. An increase in the score means an increase in the prevalence of selected nursing home predictors in the OAA service population. The components of the composite score for the well-being and independence measure are as follows: 1. Percentage of caregivers reporting that services help them provide care longer. Rationale: This variable from AoA's Annual National Surveys of OAA Service Recipients was validated as a nursing home predictor for the Family Caregiver Support Program by the Performance Outcome Measurement Project (POMP) grantees. 2. Percentage of transportation clients who are transportation disadvantaged. (Defined as unable to drive or use public transportation). Rationale: This variable from AoA's Annual National Surveys of OAA Service Recipients was validated as a nursing home predictor by the POMP grantees. Further, data from the Third National Survey of OAA Service Recipients show that older persons receiving transportation services who are "transportation disadvantaged" are more disabled and vulnerable and less likely to receive the information and assistance they need. Specifically, they are more likely to exhibit Activities of Daily Living/Instrumental Activities of Daily Living (ADL/IADL) limitations, to have stayed overnight in a hospital in the past year, to have stayed overnight in a nursing home or rehabilitation facility, and to be socially isolated (see "Predicting Elderly People's Risk for Nursing Home Placement" by William G. Weissert). They are also less likely to know how to contact their case manager and to understand an explanation of their services. 3. Percentage of congregate meal recipients who live alone. Rationale: Living alone is a predictor of nursing home placement (see "Predicting Elderly People's Risk for Nursing Home Placement, Hospitalization, Functional Impairment and Mortality" by Edward Alan Miller and William G. Weissert) and congregate meal recipients who live alone exhibit numerous other characteristics than can make them more vulnerable to loss of independence. For example, data from the Second National Survey of OAA Service Recipients show that they are more nutritionally vulnerable. They are less likely to eat three meals a day, are in poorer health, are less likely to socialize, and are more likely to be low income. 4. Percentage of home-delivered meal recipients with 3+ IADL limitations. Rationale: Multiple IADL limitations is a predictor of nursing home placement (see Miller and Weissert and the Urban Institute's 2003 study, "Estimates of the Risk of Long Term Care - Assisted Living and Nursing Home Facilities" at http://aspe.hhs.gov/daltcp/reports/riskest.htm) and data from the Third National Survey of OAA Service Recipients show that home-delivered meal recipients with three or more IADL limitations exhibit numerous other characteristics that make them vulnerable to loss of independence. For example, they are more likely to have ADL limitations, to exhibit numerous health conditions, to be homebound, and to suffer from food insecurity. Further, improved nutrition can help manage many of the diseases that they suffer from (e.g. heart disease, diabetes, and osteoporosis). AoA calculated the composite score using OAA Title III expenditures as reported in the State Progam Report to weight the four components.

Is the program purpose clear?

Explanation: The Older Americans Act created the Administration on Aging (AoA) in 1965 as the Federal focal point and advocate agency for older persons and their concerns. AoA was recently reauthorized by the Older Americans Act of 2006 (P.L. 109-365). The purpose of AoA's Aging Services Program is clearly defined in this Act as ten distinct objectives, which may be summarized as follows: To help elderly individuals maintain their independence and dignity in their homes and communities through comprehensive, coordinated, and cost effective systems of aging services with priority given to the most vulnerable elderly with economic and social needs. Examples of supportive aging services include meals programs, transportation, caregivers support, health promotion/disease prevention activities, an ombudsmen program for long-term care, and activities to prevent elder abuse and neglect.

Evidence: 1) Older Americans Act: Title I - Declaration of Objectives: Definitions, Section 101. http://www.dads.state.tx.us/providers/AAA/OAA_unofficial_compilation.pdf; and 2) AoA Strategic Plan FY 2007-2012.

Does the program address a specific and existing problem, interest, or need?

Explanation: The elderly, ages 60 and older, suffer higher levels of disease and disability than any other population age group, along with higher rates of institutionalization. Among older adults, racial and ethnic minority populations such as Native Americans, Hawaiian Islanders and Alaska Native populations experience even higher levels of disease and disability. AoA's aging services include a wide array of services to reduce the ill effects of disease and disability so that vulnerable elderly individuals may remain in their homes. These services include nutritious meals to elderly individuals in congregate and home settings; transportation to senior centers and other vital trips including medical appointments; supportive services to enable family members to continue caring for the elderly; protection and advocacy for people who are living in nursing homes; and evidenced-based health promotion activities such as exercise programs in senior centers. Finally, AoA programs are still relevant to current conditions as the problems that AoA was created to address still exist. For example, the Agency on Healthcare Research and Quality (AHRQ) projects that the number of functionally disabled elderly resulting from arthritis, stroke, diabetes, coronary artery disease, cancer, or cognitive impairment is expected to increase at least 300 percent by the year 2049.

Evidence: 1) Statistics on the aging population, including data on disabilities and well-being: http://www.aoa.gov/prof/Statistics/statistics.asp; 2) Government Accountability Office. (May, 1999). "Adults with Severe Disabilities: Federal and State Approaches for Personal Care and Other Services" http://www.gao.gov/archive/1999/he99101.pdf; 3) The Center for Rural Health at the University of North Dakota. (2005). "Prevalence of Chronic Disease Among American Indian and Alaska Native Elders." http://www.med.und.nodak.edu/depts/rural/nrcnaa/pdf/chronic_disease1005.pdf; 4) Center for Disease Prevention and Control. (2007). "State of Aging and Health in America." http://www.cdc.gov/aging/pdf/State_of_Aging_and_Health_in_America_2004.pdf; 5) Agency on Health Research and Quality. (April 2002). "Preventing Disability in the Elderly with Chronic Disease." http://www.ahcpr.gov/research/elderdis.htm.

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: No other federal or non-federal program provides the combination of services provided by AoA. AoA integrates funding from State and local sources with Federal funds and fills in the service gaps by working with State, local, and private home and community-based supportive services programs. Also, its unique program design provides for an infrastructure of aging services providers called the "aging network." This aging network is comprised of State Units on Aging (SUA), Area Agencies on Aging (AAA), Tribal Organizations, and their contractors. SUAs are located in every State and U.S. territory and are influential in setting policy, legislative and program agendas for the governor and State legislature. In fact, about half of the 56 State and territorial units on aging are of cabinet rank. AAAs are agencies designated by the State to be the focal point for implementing AoA programs and reside in different governance structures as follows: 41% are non-profits; 32% are a part of city/county governments; 25% are a part of councils of government; 1% are Indian reservations; and 1% are other. Further, the work of the aging network does not duplicate, and instead complements the work of other State and local efforts because it is statutorily charged with 1) identifying the service needs of the elderly by conducting needs assessments; 2) assessing the capacity of public and private programs to detect gaps in service areas; and 3) ensuring that funding from AoA's Aging Services Program is not used to supplant Federal, State, or local funds. The aging network is able to maximize the impact of its targeted investments by using its infrastructure of SUAs and AAAs to leverage resources and information for older adults and their families. Finally, the network provides leadership to ensure that local support continues as supportive services systems for the elderly evolve.

Evidence: 1) For needs assessment requirement, see Older Americans Act (OAA) Section 307(a). http://www.dads.state.tx.us/providers/AAA/OAA_unofficial_compilation.pdf; 2) For collaboration and coordination of service provisions, see OAA Section 306 http://www.dads.state.tx.us/providers/AAA/OAA_unofficial_compilation.pdf;3) For prohibition of supplanting of funds, see OAA Sections: 321(d), 374, 705(a) and 752(e) http://www.dads.state.tx.us/providers/AAA/OAA_unofficial_compilation.pdf; 4) National Association of State Units on Aging. (2004.) "40 Years of Leadership." http://www.nasua.org/40YearsofLeadership.pdf; 6) National Association of State Units on Aging (2006) State Aging Information System Management Study http://aoa.gov/about/results/Information%20Systems%20Management%20Study.pdf p.2. 7) National Association of Area Agencies on Aging (N4A) Conference. Presentation of 2006 Survey of Area Agencies on Aging Preliminary Results.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: Funding for most of Aging Services grant programs (95% of total AoA under the Older Americans Act) are determined by formula based on the number of persons ages 60 and older in the State. The program design also provides sufficient flexibility to States, Indian Tribal Organizations and local entities to target the unique needs of the elderly across the different communities. It accomplishes this by not being overly prescriptive in mandating a limited list of allowable supportive services. The program design also allows for shifting of funds between AoA programs so that communities can prioritize the need of individual communities and appropriately target resources to fill in the service gaps. This approach has generated positive results in program efficiency as evidenced by leveraging of funds and participation by volunteers. For example, States leverage nearly $3 for every $1 of Federal AoA funding. Also, AoA programs document a high participation of volunteers whereby volunteers comprise over 40% of Area Agency on Aging (AAA) staff.

Evidence: 1) For leveraging of funds, see AoA's Congressional Justification for FY 2008. http://aoa.gov/about/legbudg/current_budg/docs/AoA%20FY%202008%20CJ%20Final.pdf p. 7 2) For statistics on volunteers, see AoA's Congressional Justification for FY 2008. http://aoa.gov/about/legbudg/current_budg/docs/AoA%20FY%202008%20CJ%20Final.pdf p. 42 & 57 and 3)State Program Report. http://www.aoa.gov/prof/agingnet/NAPIS/SPR/2005SPR/profiles/2005profiles.asp

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: The Older Americans Act (OAA) authorizes AoA's aging services programs. OAA defines the service population as persons ages 60 and over and requires that there be a special effort to target services to the vulnerable elderly defined as individuals of low income, low income minority, rural areas, and the disabled and frail. The program design uses the aging network infrastructure to distribute Federal AoA dollars and to target services to the elderly in our communities. Entities in the aging network, such as senior centers, ensure that the right beneficiaries are served by using age as a criterion for determining eligibility for services. Additionally, each State develops a State plan which identifies techniques and strategies for reaching the most vulnerable elderly. States review the State plan against results from the State Program Reports (SPR) and other data sources. The data collected annually from the SPR includes the information on clients served who are low income, low income minority, living in rural areas and disabled or frail based on commonly used measure of limitations of activities of daily living (ADL) and independent activities of daily living (IADL). For example, the 2005 State Program Report showed that of the 9 million seniors served, demographic data on frailty was collected on those registered to receive primarily in-home services (N=1.2 million) . Of this, 72% had 3 or more limitations that hinder instrumental activities of daily living showing that services reached the most frail and disabled. 36.7% of registered clients resided in rural areas showing that dollars reached isolated rural communities and 29% were those living below the poverty level. The program design also provides the aging network with the flexibility to transfer Federal dollars among AoA programs so that resources can best serve the elderly as needs change and thus, best meet AoA's program purpose. Finally, AoA monitors to ensure that Federal funds authorized by OAA do not supplant any other federal, state or local funds such that there are no unintended subsidies.

Evidence: 1) For targeting of intended population, see Section 307 of OAA. http://www.dads.state.tx.us/providers/AAA/OAA_unofficial_compilation.pdf; 2) For prohibition of supplanting of funds, see OAA Sections: 321, 374, 705 of OAA. http://www.dads.state.tx.us/providers/AAA/OAA_unofficial_compilation.pdf; 3) State Plans. are created by the each State's Unit on Aging in 2-4 year cycles to help each state identify their direction and priorities for the upcoming period. All state plans require community input and public hearings, many of them are available on State's websites. See Florida's plan at http://elderaffairs.state.fl.us/english/StatePlan/docs/05.pdf or Washington State's process at askgeorge.wa.gov/dshs/highlight/index.html?url=http%3A//www1.dshs.wa.gov/mediareleases/2006/pr06061.shtml&fterm=state&fterm=plan&fterm=state+plan&la=en&search=../query.html%3Fcharset%3Diso-8859-1%26nh%3D7%26style%3Dsow%26col%3Ddshs%26origin%3Ddshs%26qt%3Dstate%2Bplan 4) For statistics on individuals served, including vulnerable population as defined by OAA, see 2005 State Program Reports which include data supplied annually by each state on service recipients' demographic characteristics, expenditures, and amount of services provided. http://www.aoa.gov/prof/agingnet/NAPIS/SPR/2005SPR/profiles/2005profiles.asp These state reports also contain census data from the American Communities Survey and the decennial Census. The reports are used by community planners, interested members of the public, and researchers to inform their understanding of senior services.

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: AoA has three clear long-term outcome measures that directly and meaningfully support the program's purpose. AoA's mission is to help elderly individuals maintain their independence and dignity in their homes and communities through comprehensive, coordinated, and cost effective systems of aging services with priority given to the most vulnerable elderly with economic and social needs. To this end, AoA has the following two targeting measures: 1) All States will achieve a targeting index >1 for rural, minority and poverty clients; and 2) Increase the number of elderly persons with severe disabilities (3+ Activities of Daily Living (ADL) limitations) receiving select home and community-based services. AoA also developed a new measure to improve well-being and prolong independence as a result of AoA's Title III home and community-based services. These measures have outyear long-term targets to 2012 and support the following AoA strategic goals: 1) Empower older people and their families to make informed decisions about, and be able to easily access, existing home and community-based options; and 2) Enable seniors to remain in their own homes with high quality of life for as long as possible through the provision of home and community-based services, including supports for family caregiver. Finally, the two targeting measures are already a part of AoA's strategic plan and the plan will be updated to reflect the newly devised well-being and independence long-term measure.

Evidence: 1) FY 2005, 2006, 2007 and 2008 Congressional Justifications; 2) Office of Evaluation website at http://www.aoa.gov/about/results/; 3) Performance measurement website at www.gpra.net; 4) AoA Strategic Plan http://www.aoa.gov/about/strategic/strategic.asp; 5) Older Americans Act: Title I - Declaration of Objectives: Definitions, Section 101. http://www.dads.state.tx.us/providers/AAA/OAA_unofficial_compilation.pdf

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: The program has a baseline for each of the long-term measures with outyear targets to 2012. Also, there are clear, ambitious targets that are set at a level to promote continued improvement. For example, the targeting measure for elderly with disabilities has ambitious goals to serve 10% more clients each year so that the 2012 target will serve about 80% more clients above the baseline. Similarly, the targeting index measure for poverty, minority, and poor clients sets an ambitious goal for all States to achieve an index greater than 1 by 2012. Finally, there are incremental, ambitious targets set for the composite score of the well-being and independence measure.

Evidence: 1) FY 2005, 2006, 2007 and 2008 Congressional Justification; 2) Office of Evaluation website at http://www.aoa.gov/about/results/; 3) Performance measurement website at www.gpra.net; 4) AoA Strategic Plan http://www.aoa.gov/about/strategic/strategic.asp.

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: AoA's targeting measure for elderly with disabilities and the performance measure for well-being and independence both serve as long-term and annual measures. As such, there are annual targets set for each of the measures to assess the progress made in reaching the long-term targets. As for the long-term targeting index measure for rural, minority, and poverty clients, there are separate annual measures for service recipients living in poverty and in rural areas. The two annual measures align with the long-term measure to determine whether annual progress is being made in targeting the most vulnerable of the elderly population. Finally, AoA has an efficiency measure that is measured on an annual basis. The efficiency measure aims to serve more individuals per each million dollars of AoA funding. To avoid serving more individuals at lower quality, AoA has an annual service quality measure to ensure continued high quality aging services.

Evidence: 1) FY 2005, 2006, 2007 and 2008 Congressional Justification; 2) Office of Evaluation website at http://www.aoa.gov/about/results/; 3) Performance measurement website at www.gpra.net; and 4) AoA Strategic Plan at http://www.aoa.gov/about/strategic/strategic.asp

Does the program have baselines and ambitious targets for its annual measures?

Explanation: All annual measures have baselines and annual outyear targets that are ambitious with incremental improvements. Ambitious targets were set based on the trend data, which has shown a history of continuous performance improvement. The trend data was then adjusted to reflect anticipated improvements in performance resulting from management improvement and program innovation investments. In other words, the targets are ambitious because they reflect levels above and beyond the historical trend data and anticipated program investments. The only technical exception is the service quality measure, which does not have targets reflecting incremental increases. This is because the quality measure complements the efficiency measure and is in place to guard against any declines in service quality. Targets are set at 90% for all fiscal years because estimates would have to fall below 90% in order to detect decline in service quality. In other words, it is not statistically meaningful to set incrementally higher targets because 90% is the threshold for detecting any changes in service quality.

Evidence: 1) FY 2005, 2006, 2007 and 2008 Congressional Justification; 2) Office of Evaluation website at http://www.aoa.gov/about/results/; 3) Performance measurement website at www.gpra.net; and 4) AoA Strategic Plan at http://www.aoa.gov/about/strategic/strategic.asp

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: All grantees are committed to achieving performance goals which support AoA's strategic goals. AoA's strategic goals, in turn, are supported by AoA's long-term and annual measures. Specifically, formula grantees are required to develop measurable performance objectives (i.e. performance measures) that are tied to the strategic goals and to report performance information on AoA performance measures using the National Aging Program Information System (NAPIS). Recipients of AoA's competitive grants identify performance goals, as a part of their grant application, and monitor their progress in semi-annual progress reports. In addition to these efforts, the Performance outcome Measurement Project (POMP) has helped AoA develop new performance measures and improve upon existing measures by working directly with States and other entities in the aging network. Grantees at all levels are committed to AoA's strategic goals, as evidenced by the data that the aging network submits for the performance measures in the NAPIS and the POMP surveys which demonstrates a continuous record of improved program performance.

Evidence: 1) The State Plan instructions Section II discuss inclusion of AoA Strategic goals and establishment of "measurable performance objectives". 2) State Plan review protocol; 3) Discretionary Grant Template Section 2b requires goals, objectives and outcome measures; 4) Office of Evaluation website at http://www.aoa.gov/about/results/; 5) POMP website at www.gpra.net; 6) POMP Program Announcements; 7) AoA Program Performance Analysis 2008 CJ; 8) Section 202(f) of the OAA requires the establishments of performance measures; and 9) Section 307(a) of the OAA established the Assistant Secretary's authority to establish State plan criteria.

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: AoA evaluates all of its programs on a 10 year cycle using independent research firms. While these evaluations inform program operations and management decisions, they do not met the quality criteria. For example, an on-going study of AoA's Title III home and community-based programs lacks a comparison group or a pre- and post- assessment of program participants to understand the program's impact. Instead, the Title III evaluation uses a one-time survey and analyzed quantitative program data. AoA also faces challenges in designing the most rigorous evaluation due to the nature of some of their programs. For example, the advocacy nature of the ombudsman program requires strict confidentiality for resident complainants and precludes the use of randomized control groups, comparison groups and pre-post methodologies. The ombudsman study was also mandated by Congress and employed the Institute of Medicine (IOM). IOM formed a committee to study the ombudsman program and conducted site visits and interviews, commissioned papers, held a symposium and technical panel meetings. To date, evaluations of AoA's nutrition programs are the only evaluations that meet the quality and scope criteria. For example, the 1995 national evaluation of the elderly nutrition program by Mathematic Policy Research included a comparison group of non-participants to examine program impact. AoA is also in the process of re-assessing the nutrition programs. This multi-year evaluation will include a comparison group to assess true program impact, in addition to studying the process, outcome, and cost-benefit analysis. However, nutrition programs comprise about 50% of AoA funding and it needs to conduct regular evaluations of high quality and scope to assess the impact of other key programs.

Evidence: 1) Serving Elders at Risk: The Older Americans Act Nutrition Program, National Evaluation of the Elderly Nutrition Program, 1993-1995, Mathematica Policy Research, Inc. See Vol. I Ch. I. Section C. Objectives of the Evaluation and Section D. Study Methods: <http://www.aoa.dhhs.gov/prof/aoaprog/nutrition/program_eval/eval_report.as> and Executive Summary for key findings: <http://www.aoa.dhhs.gov/prof/aoaprog/nutrition/program_eval/EXECUTIVE%20SUMMARY.doc>; 2) Real People, Real Problems: An Evaluation of the Long-Term Care Ombudsman Programs of the Older Americans Act, 1995, IOM. See Final Report, Chapter 2 Overview of the Older Americans Act Long-Term Care Ombudsman Program; Chapter 5 Effectiveness of the State Long-Term Care Ombudsman Programs; and Appendix C Study Activities <http://books.nap.edu/catalog/9059.html> and Executive Summary section "Effectiveness of the Ombudsman Program" p.11-13 <http://books.nap.edu/readingroom/books/rprp/contents.html>; 3) Title III-C Nutrition Services and VI Native American, Hawaiian Islander, and Alaska Native Services evaluation, Scope of Work; AoA Program Activities Evaluation Schedule; National Ombudsman Reporting System (NORS) website: <http://www.aoa.gov/prof/aoaprog/elder_rights/LTCombudsman/National_and_State_Data/national_and_state_data.asp>; and 4) Title III-B: Supportive Services Evaluation, ongoing, RTI International. Interim Quantitative Report, 2006: <http://www.aoa.gov/about/results/TitleIIIB_quantitative-report_6-1-06_psgFINAL.pdf>.

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: AoA presents budget and performance information in public documents such as the annual congressional justification; however, there is no clear and defined linkage between annual and long-term performance targets and resources. AoA is taking steps to make improvements in this area as evidenced by its draft marginal cost analysis document.

Evidence: 1) FY 2008 AoA Congressional Justification. <http://aoa.gov/about/legbudg/current_budg/docs/AoA%20FY%202008%20CJ%20Final.pdf>; and 2) AoA Draft Marginal Cost Analysis (March, 2007).

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: The 2002 PART assessment identified the lack of a strategic plan and outcome oriented performance measures as deficiencies. As evidenced in the 2003 PART re-assessment, AoA corrected these weaknesses by 1) developing its first strategic plan in accordance with GPRA requirements and consistent with the HHS Strategic Plan; and 2) establishing new annual and long-term performance measures. AoA was able to make these corrections quickly, largely due to the Performance Outcome Measurement Project (POMP), a sophisticated data collection project that AoA implemented in 1999. As reflected in the 1999/2000 GPRA plans, AoA's initial performance measures were output-oriented, such as units of service. However, AoA worked to further strengthen performance measurement by implementing the advanced POMP project to measure program impact. Specific program impact areas include testing of nursing home predictor models; analyzing hospital utilization data; and conducting longitudinal studies on physical/emotional health and social well-being for senior center/congregate meals participants. AoA also continues to explore opportunities to improve and refine performance measures that assess program impact. For example, AoA developed a new outcome measure for wellness and independence in this year's PART re-assessment, and is in the process of developing a frailty index to better assess its effectiveness in targeting the vulnerable elderly. In addition to AoA's work on performance measures, AoA developed an evaluation plan to assess all of its programs in a 10 year cycle. AoA first developed this evaluation plan in 2003 and has adhered to this plan to date. The supportive services evaluation, which commenced in 2004, is nearly completed with the final report expected to be issued in the summer of 2007. Also, the contract to evaluate the nutrition and Native American services was awarded in 2006 and is expected to be completed in 2009.

Evidence: 1) See 2003 PART Assessment and evidence; 2) AoA Strategic Plan (see 2.1 evidence) http://www.aoa.gov/about/strategic/strategic.asp; 3) Office of Evaluation website at http://www.aoa.gov/about/results/; 4) Federal Register Announcement for 2004-2006 Performance Outcome Measurement Projects http://www.archives.gov/federal-register/the-federal-register/indexes.html; 5) Performance outcomes Measurement Project. www.gpra.net; 6) draft statewide reporting protocol, including frailty index; 7) Advanced POMP documentation notebook including research references and some preliminary findings; 8) evaluation schedule (see 2.6); 9) statement of work for nutrition/native Americans evaluation (see 2.6); 10) FY 2006, 2007 and 2008 Congressional Justifications (see 2.1).

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: AoA uses National Aging Program Information Systems (NAPIS) to collect high-quality performance data for formula grant programs, which make up 95% of AoA's total funding. NAPIS includes profiles on elderly clients and caregivers; service utilization and expenditure data; aging network profiles; and grantees' accomplishments such as number of resolutions reported by the Ombudsman program. In addition, a national survey of service participants helps AoA to better measure the impact of its programs on participants' lives. AoA administers this survey to a random sample of program participants on an annual basis and the survey measures consumer-reported program impacts such as improved health or nutrition, increased socialization, caregiver stress reduction and prolonged independence. Data from the NAPIS and the national survey, in turn, are used to develop program outcome measures and to determine baselines and targets for the measures. This helps AoA track the performance of its program partners and also helps the program partners identify and improve program weaknesses. AoA's competitive grant programs, which make up the remaining 5% of AoA's overall funding, include Program Innovation for demonstration projects; Alzheimer's Disease Demonstration Grants; and Aging Network Support Activities. Grantees are held to individual evaluation criteria, which are reported in the semi-annual program progress reports. Finally, AoA uses the information from the NAPIS, national survey, semi-annual progress reports, and performance measures to adjust program priorities and to reallocate resources. For example, AoA used performance information from existing long-term care demonstration projects to develop Choices for Independence, a new demonstration project which takes a holistic approach to addressing the problems in our fragmented long-term care system.

Evidence: 1) Data fields in National Aging Program Information Systems (NAPIS). http://www.aoa.gov/prof/agingnet/NAPIS/docs/SPR-Modified-Form-11.08.04.pdf; 2) Annual evaluation results from state program reports. http://aoa.gov/about/results/; 3) Results from national survey. http://aoa.gov/about/results/index.asp; 4) Annual Ombudsman data from NAPIS. http://aoa.gov/prof/aoaprog/elder_rights/ltcombudsman/national_and_state_data/2005nors/2005nors.asp.

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: It is the responsibility of AoA managers to pursue improvements in program management and performance. Under the Performance Management Appraisal Program (PMAP), all employees are held accountable for results through written performance plans that cascade down from senior management's performance plans. These plans have links to program measures, as well as AoA's overall goals and objectives. Program partners of formula AoA grants are held accountable for results through a variety of means. This includes the State planning process, submission of periodic program and financial reports, annual NAPIS data collection and survey activities, and Single Audit Act audits. Failure to comply with statutory or regulatory requirements may result in restrictions or withholding of funds. If program partners fall short of their goals, AoA provides technical assistance to identify and fix deficiencies. AoA also works with grantees to implement corrective action for any deficiencies identified through the audit process. While funding for most programs is determined by formula as specified in the Older Americans Act, there are incentives to encourage better performance. For example, AoA provides additional funds based on the number of meals served by the nutrition programs. Also, there is funding for States to improve performance measurement through the POMP project and the State Planning Grant Project. Grantees of competitive AoA grants include measurable outcomes as part of grant applications and report on their progress semi-annually. Program officers review semi-annual and final performance reports and complete grant monitoring statements to ensure that program objectives are being achieved.

Evidence: 1) AoA manager performance contracts; 2) HHS Performance Management Appraisal Program (PMAP) guidance document. (Revised 1/17/2006); 3) HHS Senior Executive Service and Organizational Performance Management System guidance document. (August 2006); 4) The State Plan instructions; 5) State Plan review protocol http://www.aoa.gov/prof/agingnet/NAPIS/SPR/2005SPR/2005spr_reports.asp; 6)State Planning Grant Project Program Announcement ; 7) POMP Announcement. http://a257.g.akamaitech.net/7/257/2422/01jan20061800/edocket.access.gpo.gov/2006/E6-10641.htm; 8) Grantee progress report template for competitive grants; and 9) Grant applications for Indian Tribal Organizations.

Are funds (Federal and partners') obligated in a timely manner, spent for the intended purpose and accurately reported?

Explanation: Funding is provided to AoA through annual appropriations and funds are obligated in the year in which they are provided. For example, in 2006, AoA was appropriated $1,361,921,563 and obligated $1,361,722,344 by September 30, 2006. At the Federal level, internal control procedures ensure that monies are spent for their intended purposes. This includes the use of apportionments, allotments, and allowances in accordance with OMB Circular A-11, Part IV. Also, grants are awarded promptly and accurately using the GATES system and paid through the HHS Payment Management System (PMS). Accountability at the grantee's level includes: 1) submission of the semi-annual Financial Status Reports (SF-269) to show that the funds are obligated properly, disbursed, spent for the intended purposes; 2) funding match and program income reported in accordance with the Older Americans Act; and 3) submission of the quarterly Federal Cash Transaction Reports (SF-272) to the Program Support Center's Division of Payment Management, which operates PMS. Additionally, recipients of formula grants are required to submit: 1) a semi-annual SF-269 supplemental form that provides more detailed reporting on Federal and non-Federal expenditures; and 2) annual maintenance of effort and inter-fund transfer reports. Based on the information from these financial reports, future grants are not awarded unless the grantees comply with expenditure requirements. Finally, if any monetary findings are identified as a result of Single Audit Act audits (A-133), the regional office and central office fiscal staff review the findings and take appropriate follow-up action, which may include recovery of funds where appropriate.

Evidence: 1) Financial Status Reports (SF-269) http://www.aoa.gov/doingbus/grantrep/forms/sf269.pdf; 2) Apportionment and Reapportionment Schedule (SF-132); 3) State Audit Reports; 4) Supplemental reports for inter-fund transfers; 5) State Maintenance of Effort Reports; and 6) Quarterly 272 reports sent to Payment Management System to document cash draw down.

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: Since Community-based programs are administered at the local level, by Area Agencies on Aging (AAAs), efforts to achieve efficiencies must be directed toward the AAAs. AoA monitors performance on key aging network systems measures, such as leveraged funding and the use of volunteers. AoA's efficiency measure, number of people served per million dollars, demonstrates the efficiency of the aging network in delivering home and community-based services. AoA established the baseline for this efficiency measure in 2002, saw an improvement of 7% from FY 2004 to FY 2005, and has continued to set ambitious targets in the outyears. AoA also engages in on-going activities to enhance performance at the State/local level including: 1) the Performance Outcomes Measures Project (POMP) to develop performance measurement tools for State/local agency to use in assessing and improving program performance; 2) IT improvements including a) State Reporting Tool (SRT)- the software system States use for performance data submission to AoA; and b) Comprehensive Aging Reporting Data System (CARDS) - the AoA system under development for housing, reviewing and reporting on NAPIS data; and 3) a cooperative agreement with National Association of State Units on Aging (NASUA) to assist in the development of information systems for the collection of program information.

Evidence: 1) FY 2005, 2006, 2007 and 2008 Performance Budgets; 2) AoA Office of Evaluation website. http://www.aoa.gov/about/results/; 3) Performance Outcome Measurement Project website at http://www.gpra.net; 4) State Agency Information Systems Management Study. http://www.aoa.gov/about/results/index.asp; and 5) Cooperative agreement with National Association of State Units on Aging (NASUA).

Does the program collaborate and coordinate effectively with related programs?

Explanation: AoA coordinates with Federal agencies as well as State and local agencies to enhance services for seniors. At the national level this includes: 1) collaborating with the USDA to administer the Nutrition Services Incentive Program (NSIP) which rewards State Units on Aging (SUA) and Indian Tribal Organizations that efficiently deliver nutritious meals to older adults with USDA commodities or cash allotment for the purchase of additional food; 2) partnering with the Centers for Medicare and Medicaid Services to improve access to long-term care by funding the establishment of Aging and Disability Resource Centers and to provide enrollment assistance for Medicare Part D prescription drug and health screenings benefits; 3) using best practices from National Istitute of Health, Centers for Disease Control and Prevention, and Agency for Healthcare Research and Quality to implement evidence-based disease prevention and health promotion programs at the community level; and 4) working with the Federal Transit Administration to enhance seniors' access to transportation. The Older Americans Act (OAA) also supports the infrastructure of the aging network, which encourages collaboration at the State and local level. As noted in testimony to the Senate Special Committee on Aging (February 15, 2007), two-thirds of SUAs funded by the OAA now have responsibilities in implementing Medicaid Waiver programs. These collaborations have resulted in a greater number of seniors obtaining services, including prescription drug coverage and in-home health care services. It has also extended the breadth and depth of services at the local level where Federal funds are leveraged with local resources.

Evidence: 1) USDA and AoA Interagency Agreement FY 2006 NSIP; 2) CMS and AoA jointly funded ADRC Program Announcement FY 2006. http://www.aoa.gov/doingbus/fundopp/announcements/2006/Attachment_1392.doc and CMS and AoA Medicare Part D Memorandum Of Understanding; 3) FY 2003 and 2006 Evidence-Based Disease Prevention Grant Program Announcement.http://www.aoa.gov/doingbus/fundopp/announcements/PA1-2-A.doc see section 1.C p 12 and http://www.aoa.gov/doingbus/fundopp/announcements/2006/Attachment_1399.doc, AoA and AHRQ Inter-Agency Agreement FY 2006; 4) Memorandum of Understanding with Federal Transportation Administration signed 1/9/2003. ; 5) A toolkit promoting transportation collaboration, developed by AoA: "Seniors Benefit from Transportation Partnerships: Promising Practices from the Aging Network." http://aoa.gov/prof/transportation/transportation.asp

Does the program use strong financial management practices?

Explanation: AoA exercises sound financial management practices and complies with the Federal government and Departmental standards for financial management as established in OMB circulars (A-11, A-123, A-127, A 134, and A-136) and HHS' accounting manual. To ensure compliance with standards, AoA conducts annual assessments of its internal controls, including its internal controls over financial reporting, as required under OMB Circular A-123. AoA utilizes the financial management services of Program Support Center (PSC) for the many of its financial management processes and activities, including accounting and payment management. In October 2006, AoA implemented the Unified Financial Management System (UFMS) which is designed to bring HHS into compliance with legislative mandates and regulatory requirements for financial management systems. AoA is part of the HHS top-down audit and in FY 2006, HHS earned its eighth consecutive clean opinion. Also, a 2005 risk assessment of the Home and Community-Based Supportive Service and Congregate Nutrition Services programs determined that AoA was at low risk for improper payments. Program partners of formula AoA grants are held accountable for results through a variety of means. This includes the State planning process, submission of periodic program and financial reports, annual NAPIS data collection and survey activities, and Single Audit Act audits. Failure to comply with statutory or regulatory requirements may result in restrictions or withholding of funds. If program partners fall short of their goals, AoA provides technical assistance to identify and fix deficiencies. AoA also works with grantees to implement corrective action for any deficiencies identified through the audit process.

Evidence: 1) AoA Financial Statement Audit Memos; 2) HHS Performance and Accountability Report. http://www.hhs.gov/of/reports/account/index.html; 3) AoA's 2005 Improper Payment and Information Act Risk Assessment; and 4) OMB Circulars: a) A-11: Preparation, Submission and Execution of the Budget; b) OMB Circular A-123: Accountability and Control; c) OMB Circular A-127, Financial Management Systems; d) OMB Circular A-134, Financial Accounting Principles and Standards; e) OMB Circular A-136, Financial Reporting Requirements. /omb/circulars/index.html

Has the program taken meaningful steps to address its management deficiencies?

Explanation: The Administration on Aging continues to employ the government-wide tools made available by Congress and OMB to systematically identify and correct deficiencies in Older Americans Act programs. These tools include 1) risk assessment and management control review; 2) oversight of internal controls by the Senior Assessment Team (SAT); and 3) audit follow-up, monitoring and corrective action. A more detailed explanation of each tool follows: 1) Risk assessments and management control reviews: AoA conducts annual reviews of its program and administrative functions to identify weaknesses and reportable conditions and to plan corrective actions. AoA monitors and evaluates its internal controls on an ongoing basis to ensure compliance with the standards established by the GAO. These evaluations are conducted in accordance with OMB circular A-123, which formally incorporated the five internal control standards established by GAO into the assessment process and outlined an integrated framework for developing, assessing, correcting, and reporting on internal controls. 2) Senior Assessment Team (SAT): AoA established this team to provide oversight and accountability for the annual review and assessment of internal controls, as required by the FMFIA and Circular A-123. In accordance with the procedures outlined in the AoA Management Control Plan, the SAT reviews controls using an internal control evaluation tool developed by AoA that addresses the five internal control standards enumerated by the GAO, including the key factors identified in the GAO Internal Control and Management Evaluation Tool. The SAT also assesses AoA's internal controls over financial reporting for its key budget execution, grants management, fund balance, and financial reporting transaction cycles in accordance with the procedures outlined in the HHS Guidance Manual to Implement Appendix A of OMB Circular A-123. 3) Audit follow-up monitoring and corrective action: AoA regularly ensure sthat findings of both financial and management audits are addressed and corrected as required by Federal audit follow-up standards. While no material weaknesses were identified, these reviews identified cases where the design and/or operation of an internal control was less than fully effective. For example, in June 2006 the SAT found that operating procedures for obligations and payments related to certain types of commercial invoices, such as transportation of goods and services, did not include a mechanism for secondary review. By the end of August, AoA took corrective action by implementing new operating procedures requiring secondary review of all commercial invoices by the Director of the Office of Budget and Finance prior to submitting documents to obligate funds.

Evidence: 1) OMB Circular A-123; 2) AoA FMFIA Internal Control Evaluation Tool; 3) AoA's 2005 Improper Payment and Information Act Risk Assessment; 4) See evidence under 3.2 and 3.B.F1 for performance monitoring

Does the program have oversight practices that provide sufficient knowledge of grantee activities?

Explanation: States, which are the primary recipients of the AoA formula grants, are required to submit a State plan on a periodic basis (2 to 4 year cycles). States, in turn, distribute the Federal funding to localities through the aging network, and local providers are held accountable through contracts and audits. AoA regional staff are in regular contact with States with at least one annual on-site visit to each State. AoA staff also engages in frequent communication with State grantees during the development, review, and approval process for the State plans, which are reviewed by AoA staff and approved by the Assistant Secretary for Aging. After a plan is approved, States submit program performance data (State Program Report (SPR) and National Ombudsman Reporting System) each year through the NAPIS system. Examples of data collected include expenditures by allowable service category, persons served, targeting data, and resolution status for nursing home complaints. When States submit their annual SPR data, AoA provides feedback. This includes a comparison of the current year's SPR data to the prior year, percentage change in services, expenditures, and people served. In response to the feedback, States review and revise their reports and certify the quality of their final submission. In addition to performance data, States provide semi-annual financial status reports (SF-269 and supplemental form) to show that funds are spent for the designated purposes. Additional reports and audits include inter-fund transfer reports and A-133 audits. Similar to the State processes, Indian Tribal Organizations (ITOs) submit funding applications on a three year basis, report on the program's progress annually, and submit financial status reports. These reporting and auditing procedures provide AoA with a thorough knowledge of grantees activities, as well as help AoA track actual expenditures and verify that the funds are used for their designated purpose.

Evidence: 1) Copies of state plans and ITO applications are maintained at AoA for review by internal and external groups; 2) Central and regional staff use a written protocol for reviewing State Plans; 3) Sample site visit schedule for Region IX via their 4th Quarter Travel Plan. 4) Data Submission and Review Timeline; 5) State Program Reports http://www.aoa.gov/prof/agingnet/NAPIS/SPR/2005SPR/profiles/2005profiles.asp; 6) Financial Status Report (SF-269) http://www.aoa.gov/doingbus/grantrep/forms/sf269.pdf; 7) Supplemental inter-fund transfers; 8) A-133 Audit.; 9) For a sample of a State Plans see Florida's plan at http://elderaffairs.state.fl.us/english/StatePlan/docs/05.pdf

Does the program collect grantee performance data on an annual basis and make it available to the public in a transparent and meaningful manner?

Explanation: AoA collects, compiles and disseminates program performance data on an annual basis through the National Aging Program Information System (NAPIS). NAPIS data can be aggregated on a program-wide level and disaggregated at the grantee level, as demonstrated by the national and State level reports and tables on the AoA website. It also includes standardized electronic submission and formal verification, as well as validation and certification processes to ensure high data quality. Upon certification, data for all States are disseminated to the public via the Internet and other mechanisms, including the Performance Outcomes Measurement Project (POMP) report and various performance and strategic planning reports required by the Government Performance and Results Act (GPRA).

Evidence: 1) AoA Office of Evaluation website. http://aoa.gov/about/results/index.asp; 2) State Program Reports. http://www.aoa.gov/prof/agingnet/NAPIS/SPR/2005SPR/2005spr_reports.asp; 3) POMP project reports. https://www.gpra.net/reports.asp; 4) National survey of AoA program participants. http://aoa.gov/about/results/Final-Highlights-2nd-natioinal-survey.pdf; 5) Performance Outcomes Measures Project (POMP). https://www.gpra.net/default.asp.

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: AoA has the following three long-term performance measures with ambitious targets. 1) All States achieve a targeting index greater than 1 for rural, minority and poverty clients; 2) Increase the number of elderly persons with severe disabilities (3+ Activities of Daily Living (ADL) limitations receiving select home and community-based services; and 3) Demonstrate improved well-being and prolonged independence as a result of AoA's Title III home and community based services. AoA has made progress in achieving its long-term goals, to a large extent. The targeting index measure to reach more poor, rural, and minority elderly individuals relies on the decennial census and as such, the next target year is set at 2012 to allow for data availability and analysis. In the interim years, AoA has annual measures to assess progress made in targeting rural and poor clients and is exploring potential measures for targeting minority clients. Next, AoA has made sufficient progress in meeting the long-term goal to target more elderly with disabilities as evidenced by the annual targets that it has met in recent years. Finally, trend data shows improved performance for the new well-being and independence measure. As AoA fully implements this new measure, outyear data will show the aging network's contributions to helping the elderly achieve an enhanced sense of well-being and independence.

Evidence: 1) FY 2005, 2006, 2007, and 2008 Performance Budgets (Congressional Juistification); and 2) 2007 PART Measures Table.

Does the program (including program partners) achieve its annual performance goals?

Explanation: Two of AoA's long-term measures also serve as annual measures. Specifically, AoA met the annual targets to serve more elderly disabled individuals and trend data shows annual improvement for the new well-being and independence measure. AoA also achieved the annual targets to serve more elderly persons living below the poverty threshold; however, there was a decrease from 2004 to 2005. This decrease does not reflect lower performance. Whereas grantees reported estimated numbers of unregistered clients in 2004, AoA tightened up the data collection by requiring grantees to report actual numbers of registered clients served in 2005 and beyond. Similarly, AoA revised its annual measures to serve more elderly living in rural areas in 2006. While there is no trend data available for the new rural measure the old measure showed positive gains where targets were met by 2005. AoA also met the annual targets for the efficiency measure and the service quality measure. Finally all grantees are committed to achieving the annual performance goals. Grantees submit individual performance measures supporting these performance goals in either the State plan or the grant application. They also feed the data for AoA's performance measures using the National Aging Program Information System (NAPIS).

Evidence: 1) FY 2005, 2006, 2007, and 2008 Performance Budgets Congressional Justification; and 2) 2007 PART Measures Table

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: AoA has an established efficiency measure with a baseline and trend data that shows improved efficiencies each year. For example, AoA provided home and community-based services and nutrition services to 6,103 people per million dollars of AoA funding in 2002. In 2005, AoA served 7,492 people per million dollars, an increase of 23% from 2002. Had AoA not achieved any efficiency since 2002, it would have had to spend an additional $288 million to provide services to the same number of people that it served in 2005. AoA also has a service quality measure to complement the efficiency measure so that positive gains in efficiency are not at the expense of lower service quality. AoA achieved efficiencies, in part, by setting up systems to help grantees monitor their performance so that they may better identify strengths and improve upon weaknesses. For example, AoA 1) engaged grantees at the State/local level on the Performance Outcomes Measures Project (POMP) to develop performance measurement tools; 2) made IT improvements to the reporting tool for the annual State program reports; and 3) developed the Comprehensive Aging Reporting Data System (CARDS) for States to establish baselines and gather comparative data with linkages to data sets such as the Census. AoA also realized efficiency gains by 1) providing grantees with tools to enable them to leverage additional funding and increase their volunteer recruitment; 2) forming partnerships with organizations at the national, state and local levels to expand service capacity, most notably for transportation and health promotion; and 3) using its aging network infrastructure to provide outreach and assistance to thousands of seniors navigating Medicare Part D and seeking other aging services.

Evidence: 1) The efficiency measure is calculated as follows: (A/B) = (x/ 1,000,000) A= Total number of elderly individuals who received caregivers and senior services, as reported in the State Program Report. B= Total funding for AoA's Title III home and community-based services. X= Number of elderly individuals served per million dollars of AoA's Title III home and community-based services funding; 2) FY 2005, 2006, 2007 and 2008 Congressional Justifications. http://aoa.gov/about/legbudg/current_budg/docs/AoA%20FY%202008%20CJ%20Final.pdf; 3) AoA Office of Evaluation website. http://www.aoa.gov/about/results/.

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: AoA's mission is to help elderly individuals maintain their independence and dignity in their homes and communities through comprehensive, coordinated, and cost effective systems of aging services with priority given to the most vulnerable elderly with economic and social needs. There are no other units of government, or private entities that offer the comprehensive package of services and approaches of a national scope that AoA is able to offer to a targeted population of not just the elderly, but the most vulnerable of the elderly. Also, no other entity has the national infrastructure to challenge AoA's leadership status as "the nation's primary vehicle for organizing, coordinating, and providing community-based services, protections and opportunities for older Americans and their families."

Evidence: 1) National Association of State Units on Aging. "Four Decades of Leadership." http://www.nasua.org/40YearsofLeadership.pdf; and 2) Older Americans Act. http://www.dads.state.tx.us/providers/AAA/OAA_unofficial_compilation.pdf.

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: To date, a study of the Elderly Nutrition Services (1995) is the only evaluation that meets all of the rigorous standards for quality, scope, frequency, and independence. However, nutrition programs comprise only 50% of AoA funding and additional evaluations of sufficient quality and scope are needed to assess the impact of other supportive services that AoA provides to the elderly. The 1995 evaluation of the nutrition program was conducted by an independent entity, Mathematica Policy Research. This study was a national, multi-leveled study that showed that the nutrition program is effective and achieving results. Many of the evaluation's analytic objectives were descriptive in nature and required compiling detailed information about the organization and the persons involved with the program. To address these descriptive issues, interviews and/or observations were conducted with program participants and with personnel from organizations at all levels of the hierarchy. To examine program impact, a comparison group of eligible non-participants was identified by screening a sample of persons receiving Medicare. The final report included several major findings. First, the nutrition of program participants was better than that of non-participants. People who received meals from the Elderly Nutrition Program (ENP) had higher daily intakes of key nutrients than non-participants. and ENP meals provided 40% to 50% of participants' daily intake of most nutrients. Next, participants experienced improved social interaction compared to non-participants. The evaluation also found that most participants were satisfied with the services that ENP provides; the program efficiently leveraged AoA funding; the program coordinated service access and delivery with health and social services; and the program effectively targeted the vulnerable elderly. In addition to these outcomes, the evaluation helped AoA gain a better understanding of its population with the following demographics information: 1) Between 80% and 90% of participants had incomes below 200% of the DHHS poverty level, which was twice the rate for the overall elderly population in the U.S; 2) Approximately two-thirds of participants were at increased risk for nutritional and health problems; and 3) Home-delivered meal participants had more than twice as many physical impairments, compared to the overall elderly population. Finally, AoA is in the process of re-assessing the nutrition programs. This multi-year evaluation will include a comparison group to assess true program impact.

Evidence: 1) Serving Elders at Risk: The Older Americans Act Nutrition Program, National Evaluation of the Elderly Nutrition Program, 1993-1995, Mathematica Policy Research, Inc. See Executive Summary for key findings and conclusion: <http://www.aoa.dhhs.gov/prof/aoaprog/nutrition/program_eval/EXECUTIVE%20SUMMARY.doc>; and 2) Vol. 1 Ch. III. Dietary Intake and Social Contacts: <http://www.aoa.dhhs.gov/prof/aoaprog/nutrition/program_eval/eval_report.asp>.