Detailed Information on the
Workforce Investment Act - Dislocated Worker Assistance Assessment

Change the basis for financial reports and reallotments from obligations to actual spending.

Consolidate the program with three other State grant programs, to increase the number of workers trained, improve services, and eliminate unnecessary overhead.

Increase the Secretary's and Governors' flexibility to target resources to address special, local layoff situations in a timely manner.

Conducting an evaluation to determine WIA services' impact on employment, retention, and earnings outcomes for participants. An evaluation using existing administrative data will be completed by December 2008.

Improving reporting efficiency, program management and accountability through the collection of new information with WISPR, a common reporting system for WIA, TAA, and Wagner Peyser Programs.

Adopting efficiency measures that are linked to performance outcomes, account for all costs, and facilitate comparisons across Department of Labor training and employment programs.

Strengthen accountability by instituting common performance measures that will allow comparison between various job training programs. Measures will track employment, retention, and earnings.

Measure: Entered Employment Rate

Explanation:Percent of participants employed in the first quarter after exit. This is a federal job training program common measure, which enables the Workforce Investment Act (WIA) Dislocated Worker Program to describe in a similar manner the core purposes and results of the program compared to other education, employment and job training programs. For example, while the target population for the program is dislocated workers as defined by WIA, the ultimate outcomes for this program are the same as for all other employment and training programs. Common measures remove a barrier to service integration among programs by ensuring that programs no longer have different definitions and methodologies for measuring performance. In this case, the performance indicator measures how many participants got a job according to the following formula: Of those who are not employed at the date of participation - The number of participants who are employed in the first quarter after the exit quarter divided by the number of participants who exited during the quarter. This definition for entered employment was adopted in PY 2001 at the onset of WIA reporting and was later adopted by other programs under the common measures initiative. Experience shows actual performance results for this measure tend to fluctuate from year to year. A one percent target increase for PY 2007 would equate to an estimated 1,432 more people who enter employment in the first quarter after exit in PY 2007 over PY 2006 results, if the number of people exiting in PY 2007 remained the same as PY 2006.

Measure: Employment Retention Rate.

Explanation:Percent of participants employed in the first quarter after program exit still employed in the second and third quarters after exit This is a federal job training program common measure, which enables the Workforce Investment Act (WIA) Dislocated Worker Program to describe in a similar manner the core purposes and results of the program compared to other education, employment and job training programs. For example, while the target population for the program is dislocated workers as defined by WIA, the ultimate outcomes for this program are the same as for all other employment and training programs. Common measures remove a barrier to service integration among programs by ensuring that programs no longer have different definitions and methodologies for measuring performance. In this case, the performance indicator measures how many participants retained their employment once placed in a job, according to the following formula: Of those who are employed in the first quarter after the exit quarter - The number of participants who are employed in both the second and third quarters after the exit quarter divided by the number of participants who exited during the quarter. This definition for employment retention was adopted in PY 2005, and made the measure more challenging by including an additional quarter in which a participant needs to be employed. Based on the PY 2005 result of 88%, the Department established a goal of 91% retention by PY 2011, and targets for the intervening years have been set in a progression to reach this goal. A one percent target increase from PY 2006 to PY 2007 would equate to an estimated 1,158 more people who retained employment in the second and third quarters after exit in PY 2007 over PY 2006 results, if the number of people exiting in PY 2007 remained the same as PY 2006.

Measure: Average Earnings.

Explanation:The average six-month earnings This is a federal job training program common measure, which enables the Workforce Investment Act (WIA) Dislocated Worker Program to describe in a similar manner the core purposes and results of the program compared to other education, employment and job training programs. For example, while the target population for the program is dislocated workers as defined by WIA, the ultimate outcomes for this program are the same as for all other employment and training programs. Common measures remove a barrier to service integration among programs by ensuring that programs no longer have different definitions and methodologies for measuring performance. In this case, the performance indicator measures participants' average six-month earnings once placed in a job (note: the average earnings for a year can be obtained by doubling the performance measure result), according to the following formula: Of those participants who are employed in the first, second, and third quarters after the exit quarter - Total earnings in the second quarter plus total earnings in the third quarter after the exit quarter divided by the number of participants who exited during the quarter. The common measure definition for average earnings was adopted in PY 2006, while results for PY 2005 were for an earnings change, and prior year's results were for a wage replacement rate. In PY 2008, participants evaluated for the earnings measure are expected to earn 2.5% more per year than participants represented in the PY 2007 result. This means that an individual would earn $29,540 annually, $2,462 monthly, or $615 on a weekly basis.

Measure: Cost per participant

Explanation:The program calculates the average cost for each participant by dividing the total annual appropriation for the program by the number of participants.

Is the program purpose clear?

Explanation: This program is intended to help laid-off and incumbent workers return to work quickly at wages as close as possible to those that they received prior to layoff. The mechanism is formula grants to states and substate areas.

Evidence: Workforce Investment Act of 1998 (WIA; Public Law 105-220), Sections 131-134 [codified at 29 U.S.C. 2861-2864]; and DOL's FYs 2003-2008 Strategic Plan. Strategic plans are accessible at www.dol.gov/dol/aboutdol/main.htm.

Does the program address a specific interest, problem or need?

Explanation: The program is intended to help laid-off and incumbent workers return to work quickly at wages as close as possible to those that they received prior to layoff. Even during periods of economic growth, there are worker dislocations within certain communities and industries, so the need for assistance is ongoing. The program addresses this problem.

Evidence: The 2000 Displaced Worker Survey, conducted by the Bureau of Labor Statistics (BLS), found that during 1997-1999, there were 7.6 million displaced workers, of which 3.3 million had 3 or more years of tenure prior to displacement. During the same period, there were 5,675 mass layoffs (affecting 50 or more workers for 30 days or more) resulting in 1,149,267 layoffs, according to the Mass Layoffs Statistics data from BLS. In 2000, 5,620 mass layoff events resulted in 1,170,427 layoffs; in 2001, 8,352 layoff events resulted in 1,750,643 layoffs; and in 2002, 7,293 layoff events resulted in 1,545,340 layoffs.

Is the program designed to have a significant impact in addressing the interest, problem or need?

Explanation: Federal resources of approximately $1.2 billion annually are allocated to all states by a formula that accounts for states' numbers of unemployed people and their length of unemployment, compared to other states. States allocate these funds to local workforce investment boards for services that include job search assistance, job placement, and training. Federal funding finances services for a significant portion of dislocated workers and may leverage additional funding from states and local governments and businesses.

Evidence: DOL estimates that, for Program Year (PY) 2002, which ran from July 1, 2002, through June 30, 2003, these grants financed services for 874,400 total participants, including 379,798 registrants. As noted in the evidence/data for question #1.2, BLS estimates that there were 1,750,643 layoffs in 2001 and 1,545,340 layoffs in 2002.

Is the program designed to make a unique contribution in addressing the interest, problem or need (i.e., not needlessly redundant of any other Federal, state, local or private efforts)?

Explanation: The program is intended to focus on dislocated workers, a subset of all unemployed adults, by providing services--including job search assistance, job placement, and training--to help them return to work quickly at wages that are as close as possible to their previous wages. However, WIA's requirement for one-stop delivery of services to adults and youth means that it is difficult to separate this program's funding and services from those provided through the WIA Adult formula grants program, which provides services without regard to employment status (although some services are targeted based on need with priority given to low-income individuals if resources at a given time are limited). Additionally, Trade Adjustment Assistance (TAA) provides training and income support to a subset of workers who are dislocated due to increased imports or plant relocation. DOL has addressed redundancy, program structure, and the relationship among the Adult and Dislocated Worker programs in the 2004 and 2005 Budgets and the Administration's WIA reauthorization proposal. Rapid response services do constitute a unique contribution largely unduplicated.

Evidence: WIA Sections 131-134 [codified at 29 U.S.C. 2861-2864]; Trade Act of 1974, as amended; and DOL's FYs 2003-2008 Strategic Plan. Strategic plans are accessible at www.dol.gov/dol/aboutdol/main.htm.

Is the program optimally designed to address the interest, problem or need?

Explanation: The program's design does not allow it to help states and localities quickly and sufficiently address dislocated workers' needs. Also, as noted in the explanation of the answer to question #1.4, there is overlap with the WIA Adult program and TAA. The 2004 and 2005 Budgets and the Administration's WIA reauthorization proposal include substantial changes to address program design shortcomings. Some of the major problems that DOL is addressing include: (1) The formula for allotments to states may not always accurately reflect a state's current dislocated worker population. (Reauthorization is intended to address this problem with a new formula and new flexibility for states to adapt to their needs.); (2) Many states experience similar problems to DOL's in the data used to allocate formula funding to localities. (Reauthorization is intended to address this problem, too.); and (3) Participation estimates are inaccurate, excluding many participants because people are not required to register to receive basic services (In December 2003, DOL's Employment and Training Administration (ETA) issued guidance addressing these estimates and performance measurement.).

Evidence: WIA Section 132(a)(2)(B); General Accounting Office (GAO) Reports GAO-03-636, "Workforce Investment Act: Issues Related to Allocation Formulas for Youth, Adults, and Dislocated Workers," April 2003, and GAO-02-274, "Workforce Investment Act: Better Guidance and Revised Funding Formula Would Enhance Dislocated Worker Program," February 2002; ETA Training and Employment Guidance Letter (TEGL) No. 15-03: Common Measures Policy, December 10, 2003. TEGLs are accessible at http://wdr.doleta.gov/directives/.

Does the program have a limited number of specific, ambitious long-term performance goals that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: The program is part of the Job Training Common Measures initiative. Accordingly, it has adopted four long-term goals that will better measure the impacts of the program than current measures and allow comparisons across similar programs. DOL is implementing the common measures and will establish numerical targets in 2004. The Administration's WIA reauthorization proposal identifies the common measures and applies them to a new comprehensive adult program to be created by consolidating this program with the current WIA Adult and Employment Service State grant programs.

Evidence: WIA Section 136 (29 U.S.C. 2871), including proposed amendments to the same in the Administration's WIA reauthorization proposal; DOL's FYs 2003-2008 Strategic Plan; and ETA TEGL No. 15-03: Common Measures Policy, December 10, 2003. TEGLs are accessible at wdr.doleta.gov/directives/. Strategic plans are accessible at www.dol.gov/dol/aboutdol/main.htm.

Does the program have a limited number of annual performance goals that demonstrate progress toward achieving the long-term goals?

Explanation: The program is part of the Job Training Common Measures initiative. Accordingly, the program has adopted four annual goals that will better measure the impacts of the program than current measures and allow comparisons across similar programs. DOL is implementing the common measures and will establish numerical targets in 2004. The Administration's WIA reauthorization proposal identifies the common measures and applies them to a new comprehensive adult program to be created by consolidating this program with the current WIA Adult and Employment Service State grant programs.

Evidence: WIA Section 136 (29 U.S.C. 2871), including proposed amendments to the same in the Administration's WIA reauthorization proposal; DOL's FY 2004 Annual Performance Plan; and ETA TEGL No. 15-03: Common Measures Policy, December 10, 2003. TEGLs are accessible at wdr.doleta.gov/directives/. Performance plans are accessible at www.dol.gov/dol/aboutdol/main.htm.

Do all partners (grantees, sub-grantees, contractors, etc.) support program planning efforts by committing to the annual and/or long-term goals of the program?

Explanation: States are mandated to adopt specific goals established by WIA, negotiate with DOL to establish individual state performance levels, and report on their progress on achieving these levels. States comply with this requirement. The statute also provides an incentive for participation in which states that meet or exceed negotiated levels are eligible to receive incentive grants. To be eligible to receive the award, states must provide complete performance reports that meet DOL's data validation standards. DOL is establishing a more rigorous data validation process. These new validation requirements will provide an additional basis for incentives or sanctions (in addition to performance levels).

Evidence: TEGL No. 14-00, Change 1: Guidance on the WIA Management Information and Reporting System, October 1, 2001, Attachments E, F, and G; Field Memo No. 30-00 WIA Financial Reporting; June 23, 2000; TEGL No. 8-99: Negotiating Performance Goals; and Incentives and Sanctions for the Negotiation and Goal Setting Process; March 3, 2000, Sec. 4-5; TEGL No. 7-99; Core and Customer Satisfaction Performance Measures for the WIA System; March 3, 2000; TEGL No. 19-02: Sanctions Policy for Failure to Meet State Negotiated Performance Levels under Title I of the Workforce Investment Act (WIA); TEGL No. 3-03: Data Validation for Employment and Training Programs; and FYs 2003 and 2004 Annual Performance Plans. TEGLs are accessible at wdr.doleta.gov/directives/. Performance plans are accessible at www.dol.gov/dol/aboutdol/main.htm.

Does the program collaborate and coordinate effectively with related programs that share similar goals and objectives?

Explanation: DOL works with mandatory and other federal WIA partners, as well as states and localities, to implement the program through the nationwide system of One-Stop Career Centers through which localities deliver services. Key issues of collaboration include federal guidance for state planning and partners' sharing of the costs for operating the centers. Also, within DOL, this program and the similar TAA program are managed in the same national office. DOL has urged states to manage these programs similarly to improve customer service and program performance. The Trade Act of 2002 provides rapid response services under this program to firms that request TAA services for their displaced workers.

Evidence: DOL consults with federal WIA partners, including the Departments of Education, Health and Human Services, Housing and Urban Development, and Agriculture, and the Social Security Administration. It also has been working with the Department of Commerce on economic development strategies that incorporate this program. Additionally, DOL regularly meets with states. Unified Planning Guidance: Federal Register (FR) Vol. 65, No. 10, January 14, 2000 (to date, twenty-three states have submitted a Unified Plan); Cost Allocation/Resource Sharing Guidance: FR Vol. 66, #105, Thursday May 31, 2001; Toll-Free referral system. Training and Employment Information Notice (TEIN) No. 2-94; February 27, 1995: Dual Enrollment of Trade Impacted Workers in JTPA; Trade Act of 2002 (Public Law 107-210). FR notices and TEINs are accessible at www.doleta.gov/usworkforce/documents/

Are independent and quality evaluations of sufficient scope conducted on a regular basis or as needed to fill gaps in performance information to support program improvements and evaluate effectiveness?

Explanation: Efforts to initiate an impact evaluation of the program were curtailed in 1998. No impact or outcome evaluations of this program have been completed, nor are any financed under the WIA long-term research plan. (Process evaluations of WIA implementation have been conducted.) However, DOL developed a revised evaluation plan that established a regular cycle of evaluation for all major job training and employment programs. A new multi-year impact evaluation of this program will begin after WIA reauthorization.

Evidence: Though they did not evaluate the program's impact or outcomes, DOL's Office of the Inspector General (OIG) reported on the program in June 2000 (Report No. 04-00-002-03-340, "The Dislocated Worker Program is Not Predominantly Serving Layoff Victims"), and GAO reported on it in February 2002 (GAO-02-274, cited in Section I, #5, above). The OIG's report expressed concerns about eligibility determination and documentation and fund allocation. (OIG's reports are accessible at www.oig.dol.gov/public/reports/oa/main.htm) GAO's report expressed concerns about the formula for allocations to states.

Is the program budget aligned with the program goals in such a way that the impact of funding, policy, and legislative changes on performance is readily known?

Explanation: There has been little evidence that program funding and achievement of goals are related. Funding changes are roughly linked to participation levels but not linked to performance outcomes. The rough funding estimates appear to have far exceeded what has been needed in recent years due, in part, to poor reporting by states and localities and challenges in measuring participants who receive minimal services and, therefore, may not register. In preparing its 2004 budget request, DOL moved closer to providing full program costing and began to integrate performance goals with budget information. However, this program performance and cost integration does not yet permit measurement of the full costs of each program or an assessment of the impact of budget levels, as well as policy and legislative changes, on performance outcomes.

Evidence: DOL's FY 2003 and FY 2004 Budget Requests. Budget requests are accessible at www.dol.gov/dol/aboutdol/main.htm.

Has the program taken meaningful steps to address its strategic planning deficiencies?

Explanation: WIA specifies the performance measures on which states will report and requires that DOL negotiate performance levels with each state. As specified in statute, these negotiations occurred at the beginning of WIA implementation and established goals for the first three years of implementation and then again in year three for the final two years of authorization. This statutory requirement hampers DOL's ability to develop long-term goals. DOL is taking two steps to address this obstacle. First, the Administration's WIA reauthorization proposal would change the state negotiation process to allow for longer-term strategic planning. Second, as part of the Job Training Common Measures initiative, DOL has adopted four long-term and annual goals for the program and will establish numerical values for those goals in 2004.

Evidence: DOL's FYs 2003-2008 Strategic Plan and FY 2004 Annual Performance Plan. Strategic and performance plans are accessible at www.dol.gov/dol/aboutdol/main.htm.

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: DOL uses performance data to award incentive grants and identify areas needing technical assistance, which DOL provides to states that fail on one or more performance measures. However, this information is not reported uniformly to DOL in as timely a manner as would be useful for effective program management. States are supposed to report some information to DOL quarterly, but there may be a lag of several months from the end of a quarter to the time that DOL receives, validates, and publishes the information. Also, though data quality has been improving, DOL has some concerns. In 2003, DOL issued policy guidance for a data validation project that will create more precise programming specifications and standards for use in validating that grantee-reported data. ETA anticipates that states/grantees will finish validating 2002 data and begin submitting validation output reports to ETA in March 2004.

Evidence: DOL's Annual Reports on Performance and Accountability for FYs 2002 and 2003 (Outcome Goal 2.3); TEGL No. 3-03: Data Validation for Employment and Training Programs; and DOL's OIG's Report No. 06-02-006-03-390, "Workforce Investment Act Performance Outcomes Reporting Oversight," September 30, 2002. Performance reports are accessible at www.dol.gov/dol/aboutdol/main.htm. TEGLs are accessible at wdr.doleta.gov/directives/. Information on the data validation initiative is accessible at www.uses.doleta.gov/dv/. DOL's OIG's reports are accessible at www.oig.dol.gov/public/reports/oa/main.htm

Are Federal managers and program partners (grantees, subgrantees, contractors, etc.) held accountable for cost, schedule and performance results?

Explanation: WIA awards incentive grants to states that meet or exceed negotiated levels of performance. In addition, WIA authorizes the Secretary to apply sanctions to states that fail to meet agreed upon performance levels. Performance ratings for federal program managers are tied to ETA and program performance goals and to grants management responsibilities. All ETA managers and staff are held accountable through incorporation of relevant elements and performance standards in the appraisal process. Also, ETA's Office of Financial and Administrative Management routinely provides to managers and regions reports and analysis of cost and performance results.

Evidence: WIA section 136(g); States' Annual Progress Reports on WIA program activities; and revised performance management plans for senior executives (Form DL 1-2059, Rev. 10/2001) and for supervisors and managers (Form DL 1-382, Rev. 10/2001). States' Annual Reports are accessible at www.doleta.gov/usworkforce/documents/

Are all funds (Federal and partners') obligated in a timely manner and spent for the intended purpose?

Explanation: Although the federal program fully obligates the new funds of approximately $1.2 billion annually, states and localities carry over between $0.6 and $0.8 billion of unspent funds annually. Further, even though states and localities claim to have obligated most of these funds, the nature of the commitments is unknown, except case by case, so it is extremely difficult to say confidently how available the funds are to provide services. Tracking of spending for intended purposes is insufficient. The Administration's WIA reauthorization proposal would consolidate the Adult, Dislocated Worker, and Labor Exchange programs into a single funding stream and allow DOL to recapture unexpended funds in states that have not expended at least 70% of all funds on an annual basis. WIA currently allows recapture of unobligated funds from states that have obligated less than 80% of funds annually. The removal of any ambiguity by usage of the commonly defined and understood term 'expend' will improve the ability to track funds and ensure that they are spent in a timely manner.

Evidence: DOL's quarterly reports on states' WIA expenditures; DOL's estimates of unspent carry-in for PYs 2000-2003, as compared to expenditures and annual appropriations of $1.2 billion (PY 2000: Carry-in $302 million, Expenditures $766 million; PY 2001: Carry-in $776 million, Expenditures $1.1 billion; PY 2002: Carry-in $752 million, Expenditures $1.3 billion; and PY 2003: Carry-in $635 million, Expenditures $1.3 billion); GAO Report GAO-03-239, "States' Spending Is on Track, but Better Guidance Would Improve Financial Reporting"; and DOL's OIG's Report No. 04-02-004-03-390 (September 20, 2002) on the State of Ohio's application of agreed-upon procedures for WIA grant obligations, expenditures, and payments. DOL's OIG's reports are accessible at www.oig.dol.gov/public/reports/oa/main.htm

Does the program have incentives and procedures (e.g., competitive sourcing/cost comparisons, IT improvements) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: The program does not rely on information and tools to measure and achieve efficiencies and cost-effectiveness in state and local implementation of this and related WIA programs through One-Stop Career Centers. For example, since the inception of WIA, DOL has had difficulty producing a reliable estimate of the unit cost of serving a program participant or comparing program execution among states and localities to improve efficiency and cost effectiveness. As part of the Job Training Common Measures initiative, this program is adopting a common measure of efficiency.

Evidence: DOL's FY 2004 Budget Request and FY 2004 Annual Performance Plan. Budget requests and performance plans are accessible at www.dol.gov/dol/aboutdol/main.htm.

Does the agency estimate and budget for the full annual costs of operating the program (including all administrative costs and allocated overhead) so that program performance changes are identified with changes in funding levels?

Explanation: Like the rest of DOL, ETA does not have an integrated accounting and performance management system to identify the full cost of achieving this program's performance goals and support day-to-day operations. In preparing its FY 2004 Budget request, DOL moved closer to providing full program costing and began to integrate performance goals with budget information. However, this program performance and cost integration does not yet permit measurement of the full costs of each program or an assessment of the impact of budget levels on performance outcomes.

Evidence: DOL's FYs 2003 and 2004 Budget Requests. Budget requests are accessible at www.dol.gov/dol/aboutdol/main.htm.

Does the program use strong financial management practices?

Explanation: The program is free of material internal control weaknesses as identified by auditors. However, DOL has not completed its evaluation of the level and nature of erroneous payments in its WIA programs, including this one, and how to minimize them. This evaluation is a high priority for ETA.

Evidence: DOL's Annual Reports on Performance and Accountability for FYs 2002 and 2003, which are accessible at www.dol.gov/dol/aboutdol/main.htm.

Has the program taken meaningful steps to address its management deficiencies?

Explanation: DOL has tied its performance goals to performance ratings for managers. Also, DOL developed a strategy for program reporting that will substantially address issues of timely, accurate, and valid program information. It will address local, state, and federal levels of performance and financial reporting, and validation of data, and will improve compliance tools. These steps will help DOL to manage the program more effectively and support compliance and technical assistance efforts.

Evidence: Revised performance management plans for senior executives (Form DL 1-2059, Rev. 10/2001) and for supervisors and managers (Form DL 1-382, Rev. 10/2001); ETA program reporting strategy (information available on request)

Does the program have oversight practices that provide sufficient knowledge of grantee activities?

Explanation: DOL has improved its oversight but continues to have insufficient knowledge of grantees' activities, due in part to WIA financial requirements. Even though states and localities may claim to have obligated most funding, the extent of such commitments is unknown, except case by case. As a result, it is difficult for DOL to say confidently the amount of funds available to provide services and whether funds have been spent as intended. DOL has taken several steps to improve oversight. In 2002, the Assistant Secretary directed minimum standards and set goals for grants monitoring. DOL achieved these goals in every region, and, to date, 44 of the 52 states and territories have been monitored on site. In 2003, DOL issued an accounting procedures manual for regions and an order outlining grants administration oversight roles and responsibilities. In addition, the Administration proposed in WIA reauthorization that financial reporting and any reallocations of unspent funding be based on when funds are spent, rather than when they are obligated or committed.

Evidence: GAO Reports GAO-02-1074 and GAO-03-239; DOL's OIG's Report No. 04-02-004-03-390, as cited in the evidence/data for question #3.3; DOL's OIG's Semiannual Report to the Congress: October 1, 2001-March 31, 2002; Employment and Training Order No. 1-03; and ETA Regional Accounting Policies & Procedures Manual. DOL's OIG's reports are accessible at www.oig.dol.gov/public/reports/main.htm.

Does the program collect grantee performance data on an annual basis and make it available to the public in a transparent and meaningful manner?

Explanation: States submit WIA performance information to DOL quarterly and annually. On a quarterly basis, DOL works with state grantees to review and analyze states' performance information and helps states to translate their performance into policy decision. On ETA's Web site, DOL provides an easy link to quarterly and annual performance results for the nation and individual states. This information includes summaries of WIA participant characteristics and services received and comparisons of states' performance. DOL will continue to enhance the ETA performance Web site over the next year. ETA will make it easier for the public to find performance information from the main Web site and to understand what ETA and its partners seek to achieve and whether those goals have been attained.

Evidence: State and national WIA annual performance results for PYs 2000 and 2001; definitions of the performance measures; and quarterly performance reports as of June 30, 2003. Performance information for the ETA programs, including this one, is accessible at www.doleta.gov/usworkforce/performance/. DOL's Annual Reports on Performance and Accountability are accessible at www.dol.gov/dol/aboutdol/main.htm

Has the program demonstrated adequate progress in achieving its long-term outcome goal(s)?

Explanation: This program has adopted four new long-term goals as part of the Job Training Common Measures initiative. DOL will establish numerical annual and long-term targets in 2004. Partial credit is based on the fact that DOL achieved its previous long-term goals.

Evidence: DOL's FYs 2003-2008 Strategic Plan; In PY 2001, which ended on June 30, 2002, DOL exceeded all three of its previous long-term goals from the FYs 1999-2004 Strategic Plan. Results reported in the Annual Report on Performance and Accountability for FY 2002 are: 79% entered employment; 87% retention; 101% earnings replacement. In PY 2002, which ended on June 30, 2003, DOL again exceeded previous long-term goals for two of three measures, failing to achieve the long-term goal for wage replacement. Strategic Plans and Annual Reports on Performance and Accountability are accessible at www.dol.gov/dol/aboutdol/main.htm.

Does the program (including program partners) achieve its annual performance goals?

Explanation: This program has adopted four new long-term goals as part of the Job Training Common Measures initiative. DOL will establish numerical annual and long-term targets in 2004. Partial credit is based on the fact that DOL achieved all of its three key annual goals in PYs, 1999, 2000, and 2001, even though it achieved only two of three for PY 2002. Although the federal goals were achieved, not all state and local partners achieved their goals.

Evidence: DOL's FYs 2003-2008 Strategic Plan; In PY 2001, which ended on June 30, 2002, DOL exceeded all three of its previous long-term goals from the FYs 1999-2004 Strategic Plan. Results reported in the Annual Report on Performance and Accountability for FY 2002 are: 79% entered employment; 87% retention; 101% earnings replacement. In PY 2002, which ended on June 30, 2003, DOL again exceeded previous long-term and annual goals for two of three measures, failing to achieve the annual or long-term goals for wage replacement. Strategic Plans and Annual Reports on Performance and Accountability are accessible at www.dol.gov/dol/aboutdol/main.htm.

Does the program demonstrate improved efficiencies and cost effectiveness in achieving program goals each year?

Explanation: The program cannot determine or exhibit that it is improving efficiencies and cost-effectiveness in states' and local implementation of this and related WIA programs through One-Stop Career Centers. However, DOL is participating in the development of one common measure of efficiency and cost-effectiveness.

Evidence: DOL has been unable to provide a reliable estimate of the unit cost of serving a dislocated worker program participant.

Does the performance of this program compare favorably to other programs with similar purpose and goals?

Explanation: This program and TAA both provide training to get the unemployed back to work and have adopted four new long-term and annual performance measures as part of the Job Training Common Measures initiative. Based on performance against previous measures and goals, WIA Dislocated Worker program outcomes for PY 2002 were better than those of TAA for FY 2003 in all three areas measured. Entered employment rate: 82% Dislocated Workers vs. 62% TAA; retention rate: 90% Dislocated Workers vs. 86% TAA; earnings replacement rate: 90% Dislocated Workers vs. 73% TAA. For the preceding year, the Dislocated Workers program exceeded TAA's performance on two of three measures (TAA's retention rate was higher.). Also, in FY 2001, the TAA cost per participant was substantially higher than that for dislocated workers: $11,600 for TAA and $1,800 for Dislocated Workers.

Evidence: DOL's Annual Reports on Performance and Accountability for FYs 2001 and 2002. Comparisons are difficult because services differ. Not all WIA exiters take training; some find jobs after reemployment services alone. TAA outcomes are based on training, where experienced workers may need to start over in new careers. As WIA individual outcomes data become available, DOL will be able to make these more refined comparisons using individual characteristics as well. Annual Reports on Performance and Accountability are accessible at www.dol.gov/dol/aboutdol/main.htm.

Do independent and quality evaluations of this program indicate that the program is effective and achieving results?

Explanation: No impact or outcome evaluations of this program have been completed. (Process evaluations of WIA implementation have been conducted.) However, DOL developed a revised evaluation plan that established a regular cycle of rigorous evaluation for all major job training and employment programs. A new multi-year impact evaluation of this program will begin after WIA reauthorization, as part of an evaluation of a new consolidated Adult grant program.

Evidence: Though it did not evaluate the program's impact or outcomes, GAO reported on it in February 2002 (GAO-02-274). GAO's report expressed concerns about the formula for allocations to states and discussed various state strategies for program design without making any recommendations.