ExpectMore.gov


Detailed Information on the
Dislocated Worker National Emergency Grants Assessment

Program Code 10009003
Program Title Dislocated Worker National Emergency Grants
Department Name Department of Labor
Agency/Bureau Name Employment and Training Administration
Program Type(s) Block/Formula Grant
Assessment Year 2007
Assessment Rating Adequate
Assessment Section Scores
Section Score
Program Purpose & Design 100%
Strategic Planning 50%
Program Management 89%
Program Results/Accountability 27%
Program Funding Level
(in millions)
FY2007 $113
FY2008 $159
FY2009 $283

Ongoing Program Improvement Plans

Year Began Improvement Plan Status Comments
2007

Developing a formalized prioritization system to process the most important grants more quickly.

Action taken, but not completed The Employment and Training Administration (ETA) has a standard process in place for reviewing and processing National Emergency Grant applications. ETA continues to explore possibilities to build upon this process and expand current standard operating procedures.
2007

Establishing ambitious baselines and targets for its performance measures.

Action taken, but not completed The Employment and Training Administration (ETA) is on track to developing a statement of work for a review of past program performance and to compare a break-out of National Emergency Grant performance outcomes to the overall dislocated worker formula program outcomes. The evaluation--to be completed by December 2009--will help ETA build a foundation for establishing more ambitious targets.
2007

Adopting efficiency measures that are linked to performance outcomes, account for all costs, and facilitate comparisons across Department of Labor training and employment programs.

Action taken, but not completed The Employment and Training Administration (ETA) is funding a contractor to study and define appropriate outcome-based efficiency measures for the job training programs by September 2008. ETA will develop, adopt and implement the new efficiency measures by June 2009.
2007

Initiate an evaluation of sufficient scope and quality to evaluate overall program effectiveness.

Action taken, but not completed Funding to support an independent evaluation of the National Emergency Grant program was not available. The Employment and Training Administration will continue to seek funding.

Completed Program Improvement Plans

Year Began Improvement Plan Status Comments

Program Performance Measures

Term Type  
Long-term/Annual Outcome

Measure: Entered Employment Rate


Explanation:National Emergency Grants (NEGs) are intended to temporarily expand the service capacity of Workforce Investment Act (WIA) training and employment programs at the state and local levels by providing time-limited funding assistance in response to large, unexpected economic shocks which cause significant job losses. Economic shocks include events such as business closures, mass layoffs, realignments and closures of military installations, as well as certain natural disasters. This is a federal job training program common measure, which enables the NEG program to describe in a similar manner the core purposes and results of the program compared to other education, employment and job training programs. For example, while the target population for NEG program is workers directly impacted by the economic shock, the ultimate outcomes for this program are the same as for all other employment and training programs. Common measures remove a barrier to service integration among programs by ensuring that programs no longer have different definitions and methodologies for measuring performance. This performance indicator tracks the success of the NEG program's purpose to provide assistance to help workers reenter employment and measures how many participants got a job according to the following formula: Of those who are not employed at the date of participation - The number of NEG participants who are employed in the first quarter after the exit quarter divided by the number of NEG participants who exited during the quarter. Results for PY 2006 are heavily weighted by services to evacuees impacted by Hurricane Katrina. For calendar year 2007, the program reported 55,544 participants with 23,241 exiting the program by December 31, 2007. Of those who exited, 68% entered employment or 15, 797. The PY 2007 target for this measure is 69%.

Year Target Actual
2002 78% 72.8%
2003 78% 76.7%
2004 82% 77.6%
2005 83% 71.0%
2006 84% 63%
2007 64% PY Data-Avail 11/08
2008 65%
2009 67%
2010 70%
2011 73%
2012 76%
2013 77%
Long-term/Annual Outcome

Measure: Employment Retention Rate


Explanation:National Emergency Grants (NEGs) are intended to temporarily expand the service capacity of Workforce Investment Act (WIA) training and employment programs at the state and local levels by providing time-limited funding assistance in response to large, unexpected economic shocks which cause significant job losses. Economic shocks include events such as business closures, mass layoffs, realignment and closure of military installations as well as natural disasters. This is a federal job training program common measure, which enables the NEG program to describe in a similar manner the core purposes and results of the program compared to other education, employment and job training programs. For example, while the target population for the NEG program is workers directly impacted by the economic shock, the ultimate outcomes for this program are the same as for all other employment and training programs. Common measures remove a barrier to service integration among programs by ensuring that programs no longer have different definitions and methodologies for measuring performance. This performance indicator tracks the success of the NEG programs purpose to provide assistance to help workers retain employment and measures how many participants retained their employment once placed in a job, according to the following formula: Of those who are employed in the first quarter after the exit quarter - The number of NEG participants who are employed in both the second and third quarters after the exit quarter divided by the number of NEG participants who exited during the quarter. This measure was implemented in PY 2005. Data prior to PY 2005 are from a previous, similar measure. By defining a positive outcome as employment in the first, second, and third quarters after the quarter of exit, the measure approximates retention for at least six months following participation in the program. . Experience shows actual performance results for this measure tend to fluctuate from year to year and a one percent target increase for PY 2007 is 81%. Based on data submitted in the PY 2006 WIASRD the total number of exiters used to measure employment retention was 70,302, with 56,242 staying employed in during the second and third quarters after exit. The program exited 66,162 participants between April 2006 and March 2007 and using a one percent increase methodology the program estimates 53,591 participants will retain employment this program year.

Year Target Actual
2002 88% 85.2%
2003 88% 88.9%
2004 91% 92.1%
2005 89% 89.0%
2006 90% 81.1%
2007 83% PY Data-Avail 11/08
2008 84%
2009 85%
2010 86%
2011 88%
2012 91%
2013 92%
Long-term/Annual Outcome

Measure: Average Earnings


Explanation:National Emergency Grants (NEGs) are intended to temporarily expand the service capacity of Workforce Investment Act (WIA) training and employment programs at the state and local levels by providing time-limited funding assistance in response to large, unexpected economic shocks which cause significant job losses. Economic shocks include events such as business closures, mass layoffs, realignment and closure of military installations as a result of Base Realignment and Closure (BRAC) which cause worker dislocation, as well as natural disasters declared by the Federal Emergency Management Agency (FEMA) as eligible for public assistance. This is a federal job training program common measure, which enables the NEG program to describe in a similar manner the core purposes and results of the program compared to other education, employment and job training programs. For example, while the target population for the NEG program is workers directly impacted by the economic shock, the ultimate outcomes for this program are the same as for all other employment and training programs. Common measures remove a barrier to service integration among programs by ensuring that programs no longer have different definitions and methodologies for measuring performance. In this case, the performance indicator measures participants' average six-month earnings once placed in a job (note: the average earnings for a year can be obtained by doubling the performance measure result), according to the following formula: Of those NEG program participants who are employed in the first, second, and third quarters after the exit quarter - Total earnings in the second quarter plus total earnings in the third quarter after the exit quarter divided by the number of NEG program participants who exited during the quarter. The common measure definition for average earnings was adopted in PY 2006, while prior year results were for a different earnings measure tracking wage replacement. For calendar year 2007 the program reported average earnings of $12,266 for the 23,241 participants who exited the program by December 31, 2007. This translates to $24,532 per year, $2,044 per month or $511 per week.

Year Target Actual
2002 98% 80.4%
2003 93% 86.5%
2004 91% 80.1%
2005 92% 93.3%
2006 Baseline $13,087
2007 $13,220 PY Data-Avail 11/08
2008 $13,350
2009 $13,490
2010 $13,620
2011 $13,760
2012 $13,900
2013 $14,248
Annual Efficiency

Measure: Cost per Participant


Explanation:The program calculates the average cost for each participant by dividing the total budget authority for the National Emergency Grant annual authorization by Congress by the total participants enrolled in National Emergency Grant programs reported during the program year.

Year Target Actual
2005 Baseline $2,879
2006 baseline $1,529
2007 $1,515 PY Data-Avail 11/08
2008 $1,498
2009 $1,484
2010 $1,469
Long-term/Annual Output

Measure: Grant Application Timeliness


Explanation:National Emergency Grants (NEGs) are intended to temporarily expand the service capacity of Workforce Investment Act (WIA) training and employment programs at the state and local levels by providing time-limited funding assistance in response to large, unexpected economic shocks which cause significant job losses. Economic shocks include events such as business closures, mass layoffs, realignment and closure of military installations as a result of Base Realignment and Closure (BRAC) which cause worker dislocation, as well as natural disasters declared by the Federal Emergency Management Agency (FEMA) as eligible for public assistance. This performance indicator tracks the NEG program's timeliness by measuring the average number of days to process an application from receipt to award. Because of the emergency nature of this program, it is important that grant applications are processed quickly. The majority of all grants are awarded within 30 working days. Moreover, the majority of disaster grants are awarded within 16 working days.

Year Target Actual
2007 30 22.37
2008 22
2009 21.5
2010 21
2011 20.5
2012 20
2013 19.5

Questions/Answers (Detailed Assessment)

Section 1 - Program Purpose & Design
Number Question Answer Score
1.1

Is the program purpose clear?

Explanation: National Emergency Grants (NEGs) are intended to temporarily expand the service capacity of Workforce Investment Act (WIA) training and employment programs at the state and local levels by providing time-limited funding assistance in response to large, unexpected economic shocks which cause significant job losses. Economic shocks include events such as business closures, mass layoffs, realignment and closure of military installations as a result of Base Realignment and Closure (BRAC) which cause worker dislocation, and natural disasters declared by the Federal Emergency Management Agency (FEMA) as eligible for public assistance.

Evidence: ?? Training and Employment Guidance Letter (TEGL) No. 16-03, "National Emergency Grant Policy Guidance," issued January 26, 2004; ?? Workforce Investment Act: National Emergency Grants - Application Procedures; Notice 69 Federal Register 23052-23080, April 27, 2004; and ?? Workforce Investment Act, Section 173, 29 U.S.C. 2918.

YES 20%
1.2

Does the program address a specific and existing problem, interest, or need?

Explanation: NEGs respond quickly to unforeseen emergencies and unexpected events that cause worker dislocation. NEG funds are designed to flexibly allow for a sudden need for employment-related assistance in order to return the affected individuals to the workforce as quickly as possible. In addition, NEGs address the needs of disaster victims by providing paid, temporary employment to assist in recovery of wages and simultaneously contribute to the recuperation of the affected area. Dislocated Worker formula funding is often not sufficient to respond to emergency events, and thus each NEG is awarded after careful analysis of the dislocated worker funding stream to ensure that the emergency funding is warranted. NEGs respond to specific problems and may be used to provide assistance to workers in the following situations: ?? A plant closure or single company layoff of 50 or more workers. ?? Multiple company or industry-wide layoffs that occur at approximately the same time, and where the dislocations from each company impact 50 or more workers. ?? Community impact where multiple small dislocations over a six month-period have a significant impact on the unemployment rate of the area. ?? Closures and realignments of military installations as a result of BRAC, where funds for transition services are available as well as for workforce transition planning. ?? Disaster Grants to create temporary employment to work on projects that: 1) provide food, clothing, shelter and other humanitarian assistance for disaster victims, and 2) help with the demolition, cleaning, repair, renovation and reconstruction. ?? Job loss due to foreign trade where workers are eligible for Trade Adjustment Assistance access NEG funds for wrap-around employment services, and to ensure that workers (assessed as requiring training to return to the workforce) will have access to such training as soon as possible after dislocation.

Evidence: ?? Training and Employment Guidance Letter (TEGL) No. 16-03, "National Emergency Grant Policy Guidance", issued January 26, 2004; ?? Training and Employment Guidance Letter (TEGL) No. 16-03, Change 2, "Use of National Emergency Grant (NEG) Funds under the Workforce Investment Act, as amended, to Support Activities Related to the 2005 Base Realignment and Closure (BRAC 2005) Commission", issued May 24, 2005; ?? Workforce Investment Act: National Emergency Grants - Application Procedures; Notice 69 Federal Register 23052-23080, April 27, 2004; and ?? Workforce Investment Act, Section 173, 29 U.S.C. 2918.

YES 20%
1.3

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: NEGs are designed to maximize coordination among related programs and make full use of WIA and Wagner-Peyser formula funds that are allocated for dislocated workers and state level Rapid Response funds to ensure a system-based approach whereby workers impacted by major economic events will be able to take advantage of the full array of employment-related assistance available from the workforce investment system. For example, Dislocated Worker allotments, part of WIA program funding, must be expended to a 70 percent level for states to receive a National Emergency Grant. These procedures are in place explicitly to avoid duplication of efforts and expenditures. In addition, NEGs may provide needs-related payment (income support) to unemployed dislocated workers to help them complete their training. There are specific eligibility requirements for needs-related payments to prevent duplication with other programs (TAA). Individuals must be unemployed and not qualify for or have ceased to qualify for Unemployment Insurance and Trade Readjustment Allowances in order to enable them to participate in training services. NEGs funds also complement assistance provided to workers under Trade Adjustment Assistance by allowing for dual enrollment. The primary purpose for dual enrollment is to provide trade-eligible dislocated workers with the services that are not available under TAA and where State formula Dislocated Worker program funds are not sufficient to provide services. NEG funds may also be requested to provide training when a state demonstrates that it has spent or will spend by the end of the Trade program's fiscal year its annual allotment. There are federal and non-federal agencies engaged in disaster relief and assistance but services do not overlap with the portion of NEGs issued in response to natural disasters. NEG funds may be used for paid, temporary disaster employment that enables dislocated workers to begin to rebuild their lives and provide temporary income as infrastructure and the economy is recovered. Workers perform demolition, cleaning, repair, renovation and reconstruction, as well as jobs that help to provide food, clothing, shelter, and related humanitarian services. Although FEMA provides funding for clean-up equipment and serious needs of individuals, such as medical expenses, NEGs are specifically for providing a workforce to carry-out cleaning and repair, and provide paid, temporary employment to dislocated workers. FEMA does not have a temporary jobs program. The Department of Defense (DOD) transitions communities and individuals affected by base closures and realignments, and ETA works closely with DOD in support of BRAC to coordinate services provided under NEGs. The purpose of the BRAC committee, in which ETA participates, is to coordinate services. Unlike DOD transition services, NEG funds are specifically to support the transition of civilians and contractors, not military personnel.

Evidence: ?? Training and Employment Guidance Letter (TEGL) No. 16-03, "National Emergency Grant Policy Guidance," issued January 26, 2004; ?? Training and Employment Guidance Letter (TEGL) No. 16-03, Change 2, "Use of National Emergency Grant (NEG) Funds under the Workforce Investment Act, as amended, to Support Activities Related to the 2005 Base Realignment and Closure (BRAC 2005) Commission," issued May 24, 2005; ?? Workforce Investment Act: National Emergency Grants - Application Procedures; Notice 69 Federal Register 23052-23080, April 27, 2004; ?? American Red Cross http://www.redcross.org/services/disaster/0,1082,0_319_,00.html; ?? Salvation Army http://www.salvationarmyusa.org/; and ?? Council on Foundations http://www.cof.org/.

YES 20%
1.4

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: National Emergency Grants are event-driven sources of workforce funding used to temporarily expand capacity of states to meet the needs of workers impacted by unexpected economic shocks. NEGs feature flexible design which allows for disaster responses that meet the needs of specific situations that are not anticipated as part of the normal course of workforce development operations, and targets resources efficiently to meet those specific needs. Performance results are positive and the employment rate of participants has gone up each year. Performance falls slightly below that of formula dislocated worker programs, due to the very specific populations NEG funding goes to support. Part of the flexible design of NEGs is to respond to discrete dislocations, so funds are targeted to help a particular population and the grant goes to assist that group. The agency's WIA re-authorization proposal does not recommend changes to the structure of NEGs; the proposal reforms other workforce programs to operate more like NEGs. Indeed, the program design most closely resembles the administration's proposal for block grant funding to states that ensure the maximum flexibility and also the most targeted use of funds. In the agency's FY 2008 budget proposal, the NEG program design remains unchanged. The agency recently announced a pilot in the use of NEG funds for Regional Innovation Grants. These NEGs will allow regions that are experiencing economic shock and downturns in critical industries to conduct strategic planning for their areas' economic recovery, which will enhance and complement the current approach to NEGs.

Evidence: ?? Training and Employment Guidance Letter (TEGL) No. 16-03, "National Emergency Grant Policy Guidance", issued January 26, 2004; ?? Workforce Investment Act: National Emergency Grants - Application Procedures; Notice 69 Federal Register 23052-23080, April 27, 2004; ?? Fiscal Year (FY) 2008 Budget Briefing Package http://www.doleta.gov/budget/08bud.cfm; and ?? Workforce Investment Act Proposals http://www.doleta.gov/reports/dpld_legislative.cfm.

YES 20%
1.5

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: According to GAO, between January 2004 and December 2005, more than 30,000 mass layoffs involving 50 or more workers occurred in the United States, causing more than 3.4 million workers to lose their jobs. National Emergency Grants are targeted to some of those affected workers by providing time-limited resources to expand the service capacity of the workforce system in order to provide much needed support for these workers such as temporary employment, training assistance, and other supportive services when other state and federal programs are insufficient to meet their needs. Quarterly program reports indicate that the number of workers served through the grants exceeded the planned number. For example, during recent program years the planned participation level was 12,854 workers, but the actual number of workers served (13,164) exceeded the planned/targeted number. The NEG awards are determined based on an application submitted which describes the event, the impact on the workforce, and the planned response. Trained NEG specialists look at historical information and other activities in the state to make recommendations on funding. For eligible dislocation events, specialists compare planned participant levels to Rapid Response survey data to establish baselines of 50 percent of affected workers. NEGs are often awarded with incremental funding so that progress is reviewed throughout the life of the project to ensure enrollments are accurate, particularly in the case of disaster grants. NEG applications are received on a rolling basis throughout the program year. As such, grants are generally awarded on a first-come, first served basis, though resources are awarded judiciously to ensure that funds are available throughout the program year. In addition, states can apply for supplemental funding under current awards should the needs change for such projects. Of 124 application requests received in 2004 and 2005, six applications were denied for non-compliance with application guidelines (TEGL 16-03). Applications that do not support eligible purposes and/or that do not meet the state formula Dislocated Worker program 70 percent expenditure threshold are not considered for funding as submitted; ETA cites the deficiencies in formal communication to the applicant. The number of applications turned down are low because ETA regional and national office work closely with the proposed applicant prior to submission to determine the viability of their request and will discourage submissions if the applicant can not meet the NEG requirements. In general, NEG funds may be used to pay an appropriate level of administrative costs based on the design and complexity of the program. The expenditures for administrative costs are generally limited to ten percent of the award amount because the focus is on providing services to workers not building the permanent capacity of the workforce system. However, in rare circumstances, NEG applicants may negotiate with the Department on a case by case basis for additional administrative funds where it can be clearly demonstrated that such adjustments will achieve a greater positive benefit for the workers and /or communities being assisted. For PY 2004 and PY 2005, only two grants awarded were over the 10 percent administrative limit (11 percent and 12 percent). There is no evidence that these grants supplant funding for other worker services.

Evidence: ?? Training and Employment Guidance Letter No. 16-03, Change 3, "Use of NEG Funds to Support Disaster Relief Employment and Training Assistance for Individuals Impacted by Hurricane Katrina" issued October 14, 2005; and ?? Workforce Investment Act: National Emergency Grants - Application Procedures; Notice 69 Federal Register 23052-23080, April 27, 2004. http://www.doleta.gov/layoff/pdf/FlexibilityAct.pdf.

YES 20%
Section 1 - Program Purpose & Design Score 100%
Section 2 - Strategic Planning
Number Question Answer Score
2.1

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: NEGs have been part of the Job Training Common Measures initiative since ETA implemented the measures in 2004. Accordingly, the program has adopted 3 specific long term measures that measure the impacts of the program and allows for some comparison across similar programs. These measures are: 1) entered employment, 2) employment retention, and 3) wage replacement rate/average earnings. NEGs utilize the Dislocated Worker performance goals for each state as part of the criteria in evaluating applications and utilize the average cost per participant in the state's dislocated worker program to make comparisons in NEG applications to evaluate the reasonableness of costs. In addition, ETA tracks the timeliness with which funds are provided to areas which have an emergency need by the average number of days. Each NEG application is tagged with the date of receipt in the NEG e-system to the date it is approved or turned down by the Secretary of Labor. Following the approval from the Secretary, the grant application continues to be tracked until the grant award is executed, which is when funding becomes available to the grantee. ETA will add the current timeliness measure as part of its annual measures.

Evidence: ?? NEG Application Procedures, TEGL 17-05, Common Measures Policy for ETA, Planning Forms and Quarterly Reports.

YES 12%
2.2

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: WIA requires states to negotiate with DOL to establish performance targets for each of the common measures. NEGs are a component of the Dislocated Worker program. Therefore, the NEG program has adopted the targets and timeframes for its long-term measures established by the Dislocated Worker program. Just one of the four measures has ambitious long-term targets in place. Two of the four long-term measures lack established baselines or long-term targets. The long-term goal for entered employment is the lone ambitious target. The current outcome for entered employment is 71%, the PY11 target is 85%. The long-term target for employment retention is not ambitious, as previous outcomes have already surpassed the PY11 goal. PY 2004 and 2005 outcomes were 92.1% and 89%. The long-term target is 91%. There is not long-term target for the earnings measure or the timeliness measure. ETA is currently developing a baseline and will base the long-term targets on the baseline.

Evidence: ?? Strategic Five Year Plan, TEGL 17-05, Common Measures Policy for ETA.

NO 0%
2.3

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: The Job Training Program Common Measures are the foundation of ETA's performance accountability system. Like the Dislocated Worker program and other DOL employment and training programs, NEG uses the same measures for evaluation of its annual performance as for achievement of long-term goals. The program has been tracking its timeliness regarding NEG application processing internally but will add a timeliness measure as part of its annual and long-term performance measures. Given the emergency nature of the grants provided, continual improvement to timeliness of application processing is an important measure of the program's effectiveness.

Evidence: ?? State Planning Guidance 13-06, State plans and DOL's annual budgets, performance plans, and performance reports, including for this program, are available at www.dol.gov/dol/aboutdol/main.htm#budget.

YES 12%
2.4

Does the program have baselines and ambitious targets for its annual measures?

Explanation: Targets are re-negotiated with state workforce agencies on an annual basis, using actual performance data from the previous years. The target for one annual measure is ambitious. Two of the measures lack formal baselines. Entered employment targets for PY 2006-08 are ambitious because they are still above performance levels attained in recent years. Employment Retention, though, represents a more modest improvement over PY 2005 results but the PY2006 and PY2007 targets are below previous performance outcomes. The Earnings measure has recently changed, from earnings replacement to average earnings and a baseline target has yet to be established. The baseline and targets will be in place in PY 2006. The timeliness measure is currently without a baseline or an annual target.

Evidence: ?? State Planning Guidance 13-06, State plans and DOL's annual budgets, performance plans, and performance reports, including for this program, are available at www.dol.gov/dol/aboutdol/main.htm#budget.

NO 0%
2.5

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: As discretionary grant awards by the Secretary, NEG projects, at both the grantee and projector operator level must be designed to achieve performance outcomes that support the performance goal commitments by the Secretary under the Government Performance and Results Act (GPRA). To support the attainment of those performance goals, TEGL 17-05 was released to formally communicate that states and workforce system grantees, including NEG grantees, must collect and report against program goals which are the common performance measures. Common measure targets are established for the WIA Dislocated Worker program, and NEG grants are evaluated according to these targets for award decisions, project performance goals, and outcomes. The targets are negotiated with state workforce agencies, which are the majority of NEG grantees, on an annual basis. NEG applicants submit a Project Synopsis Form that includes the planned performance for each project including entered employment rate, cost per participant, and planned wage replacement rate, which commits them to those goals for the life of the project unless a modification is requested and approved. Once awarded, grantees collect data on common measures from all local/subgrantee projects, certify the accuracy of the data, and submit quarterly progress reports and the state's Workforce Investment Act Standardized Record Data (WIASRD). Project operators are held accountable to the performance measures through sub-grants and contractual agreements which clearly lay out performance expectations. To address underperformance, DOL, through its monitoring efforts, negotiates corrective action plans with grantees and subgrantees.

Evidence: ?? TEGL 17-05 http://wdr.doleta.gov/directives/attach/TEGL17-05.pdf; ?? Workforce Investment Act: National Emergency Grants - Application Procedures; Notice 69 Federal Register 23052-23080, April 27, 2004; ?? NEG application guidelines; and ?? Texas Monitoring Report dated, September 8, 2006.

YES 12%
2.6

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: To date, there has been no impact or outcome evaluation to measure the overall effectiveness of the program. GAO, in two 2004 reports and one 2006 report, analyzed the timeliness of the NEG application process. In December 2006 DOL published a Mathematica report which detailed the implementation of National Emergency Grants in military communities (Base closing and realignment is a qualifying event for NEGs). The report did provide some analysis of program impacts, but was focused on one specific set of NEG grants and, as the Mathematica points out, the evaluation of outcomes did not use a formal experimental design.

Evidence: ?? GAO Report 04-222 can be found at: http://www.gao.gov/new.items/d04222.pdf; ?? GAO Report 04-496 can be found at: http://www.gao.gov/new.items/d0496.pdf; and ?? GAO Report 06-870 can be found at: http://www.gao.gov/new.items/d06870.pdf. - DOL/Mathematica Report on Military NEGs can be found at: http://wdr.doleta.gov/research/FullText_Documents/Military%20NEG%20Report%20Final%2Epdf

NO 0%
2.7

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: The budget for NEGs is not linked to performance outcomes. The program does not have an integrated budget and performance presentation that makes clear the impacts of funding, policy or legislative decisions on program performance. It is difficult to link the NEG budget to performance as the program and its outcomes are subject to the unpredictability of the nature of economic shock events and other emergency events. For instance, Congress provided additional funding and flexibility to the program in the case of Hurricane Katrina, The Community Based Job Training Grants have been funded from the Dislocated Worker National Reserve, reducing the funds available for NEGs. Subsequently, the NEG program has experienced a reduction in funding, overall. Under its FY 2008 performance budget request, the Department proposes that funds previously appropriated for the WIA Adult, WIA Dislocated Worker, and WIA Youth Programs and the Employment Service be allocated to states as a single funding stream. The proposal would provide 7.5 percent of the single stream amount as the reserve amount to be used for NEG activities.

Evidence: ?? Information on State negotiated levels of performance under WIA can be found at: http://www.doleta.gov/Performance/goals/st_neg_perf_level.cfm; ?? ETA's Strategic Plan for 1999-2004 can be found at: http://www.doleta.gov/Performance/goals/st_neg_perf_level.cfm; and ?? ETA's FY 2008 Budget Request can be found at: http://www.doleta.gov/budget/08bud.cfm.

NO 0%
2.8

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: ETA implemented in 2004, an electronic application system to improve the timeliness of responding to National Emergency Grant requests. This has resulted in significant improvements to meeting the internal goal of processing and awarding incoming NEG applications and modifications within 30 days. In addition, ETA implemented Common Measures in 2004, which improved the ability of the NEG program to measure and compare performance by requiring a common set of information about all individuals served under ETA grants. ETA is announcing Regional Innovation Grants (RIGs), which will offer a strategic planning component focused on regional economic development that will enhance and complement the current approach to NEGs. Regional Innovation Grants will allow grantees to develop a comprehensive, integrated, strategic regional plan that will facilitate a quality, timely, and effective system-wide response to unanticipated economic events. Studies have shown that areas which undertook effective timely planning successfully transitioned workers and workforces into newly focused economies, while others with less inclusive strategic planning languished to try to recover economically. Therefore, this enhancement may ultimately contribute to long-term performance of the NEG program.

Evidence: ?? GAO report -06-870: http://www.gao.gov/new.items/d06870.pdf; and ?? TEGL 17-05: http://wdr.doleta.gov/directives/attach/TEGL17-05.pdf.

YES 12%
Section 2 - Strategic Planning Score 50%
Section 3 - Program Management
Number Question Answer Score
3.1

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: Grantees report performance information (i.e., Common Measures) on a quarterly basis through the NEG e-system and WISARD. ETA staff monitors grantee performance and financial practices through the analyses of this quarterly data and through regular on-site and desk reviews. To assist staff with monitoring, ETA has developed a NEG monitoring guide that is about to be released as well as an on-line tool called "Grants E-Management System (GEMS)." GEMS helps staff to identify immediate and potential risks in grant activities and provides access to all program data through a related information tool called, "Enterprise Business Support System (EBSS)." ETA uses performance data to identify grantees that may need technical assistance as well as to determine if a grantee will receive an incremental award (if so requested).

Evidence: ?? Quarterly performance reports, NEG draft review guide, quarterly expenditure reports, and WIASRD; ?? Regional office on site review reports; and ?? Link to EBSS.

YES 11%
3.2

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: All ETA managers are held accountable through incorporation of relevant elements and performance standards in the appraisal process. These standards cascade down throughout the entire organization. For example, NEG specialists and managers have in their performance standards to prepare funding recommendations in an accurate and timely manner (within 15 days of receipt for modifications, 30 days for new applications) and to prepare well-formulated analysis of NEG applications. Analysis is defined in the performance standards as examining the applicant's prior performance and expenditures. NEG applications are evaluated for reasonableness of cost and planned performance outcomes. For new applications, incremental, and supplemental funding, NEG specialists analyze the per participant expenditure levels in the proposed project service area to see if it is consistent with the Dislocated Worker formula program; scrutinize the overall project performance goals for consistency with the Secretary's goals for the dislocated worker program, and determine whether there is an acceptable rate of usage of dislocated worker formula funding. Planned levels of performance must be consistent with the established NEG performance goals or the applicant must include specifics of employment-related information on the project's target group to justify a lower level of performance. When cost or outcomes are substandard, the managers and specialists work with the applicant to bring the project up to acceptable standards, or funding is denied. Additional information on subgrantees (project operators) is also requested during the application process, including information on number of participants, total budget, and project scope. For each NEG, a project operating plan is submitted by the grantee to ETA staff to reflect the approved project design and the funding parameters in the grant award. The project operating plan must be completed within 90 days following award and 60 days for disaster grants. The existence and completeness of the Operating Plan is a pre-condition for the release of additional funding increments as well as to begin implementation of the project. In addition to reviewing the Project Operating Plan, each project will be reviewed on-site at least once by ETA staff, between 6 months following the date of award and the project mid-point. The purpose of this review is to verify core compliance factors such as participant eligibility and adequate financial management, assess the effectiveness of participant service policies and processes in achieving project performance goals and analyzing the overall performance of the grantee. This review includes both the grantee and the project operators since they are also responsible for cost, schedule, and performance.

Evidence: ?? NEG Performance Standards; ?? Project Operating Plan; and ?? Monitoring Report. Examples: One example of supplemental funding being limited due to performance is a grant to the State of Massachusetts. The project was targeted to serve 770 participants. However, only 386 individuals were actually enrolled in the project at the time the request for supplemental funds was submitted. Therefore, when the request for $2,735,927 was submitted, the Department, after careful consideration for the community and the impacted workers and taking into account the performance issues as stated, decided to award a reduced funding incremental amount of $395,833 and would monitor closely the costs and performance associated with the project since there was a continued need. Furthermore, it was conveyed to the grantee to explore the availability of other resources to meet the continuing needs of the workers beyond this award. Another example was with the State of Ohio where the project was held in abeyance due to concerns raised by citizens and environmental organizations to the Department regarding clean-up activities that were having a detrimental effect on the environment being conducted by temporary job participants enrolled in the NEG project. The project was halted for 15 days pending termination unless the grantee was able to provide the Department with a list of information identifying the activities, locations and other relevant information to further justify the need for continued funding for the clean-up activities. Further, the grantee entered into negotiations with the Department and the U.S. Fish and Wildlife Service regarding what environmental practices would be allowed for the remainder of the grant. After negotiations, the grantee committed through a letter to implement those sound environmental practices. After six weeks, the grantee was able to resume activities for the remainder of the grant period.

YES 11%
3.3

Are funds (Federal and partners') obligated in a timely manner, spent for the intended purpose and accurately reported?

Explanation: Program funds are obligated and spent consistently for the intended purpose whether it is for dual-enrollments, regular, emergency, disaster, or HCTC grants. The majority of all awards are awarded within 30 working days. Moreover, the majority of disaster awards are awarded within 16 working days. Since 2004, many NEGs have been incrementally funded to minimize any end-of-project under expenditure as well as to be able to manage the program dollars as a whole, since management cannot predict when the grant funds will be needed and whether sizeable economic events will occur during the program year. The Department has a good record of quickly obligating funds once an execution of the NEGs grant agreement is made. When a Grant Officer signs a grant agreement, the grant is considered executed and the funds are legally obligated. This legal obligation means that the grantee has the legal authority to expend funds at the date of execution. Therefore, the execution of a grant award and the obligation of funds -when the grantee can incur costs, occur simultaneously. The Department's records for PY 04 and PY 05 indicate the average time taken by ETA's Office of Financial and Administrative Services to obligate the funds to the grantee is 6 to 7 working days. However, as with any grant, extenuating circumstances can dictate a longer processing time to obligate funds. For example, the California Katrina grant was a re-directing of funds, which required multiple approvals, moving of funds by deobligating from one account and obligating to another. Overall, every appropriate measure is taken to ensure funds are available for use by the affected area as soon as possible. Grantees expend funds for the purpose of providing employment and training related services which are verified through NEG regional office monitoring visits/reviews. ETA has developed procedures for reporting and evaluating actual expenditures and grantees submit quarterly financial status reports for obligations and expenditures. Of the amount of funds appropriated to the NEG program in PY 2004 and 2005, only 0.002 and 0.005 percent (respectively) remained unobligated. Likewise with the grantees funding award, if it is identified through an analysis check or monitoring visit that the awarded funds will not be expended fully, efforts are taken to negotiate a voluntarily return of funds prior to the expiration of the project to minimize the amount of funds unexpendedat the end of the grant. As a result of these efforts only 0.06 and 0.07 percent (respectfully) of grantee funds were unexpended for PY04/05.

Evidence: ?? NEG Award Letters, GAO Report GAO-O6-870; ?? Workforce Investment Act: National Emergency Grants - Application Procedures; Notice 69 Federal Register 23052-23080, April 27, 2004; ?? Monitoring Report; and ?? NEG quarterly financial summary report.

YES 11%
3.4

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: The program does track a timeliness measure. The process timeliness measure is used for the NEG program. The NEG program measures its administrative efficiency by average processing time from the date the application is received to the date of approval, and continues to track the average time from the date of approval to the issuance of the award letter when funds become available. Additionally, part of the NEG application evaluation is "reasonableness of costs," which compares the average cost per participant in a state's formula Dislocated Worker program with the proposed costs in a NEG application, excluding disaster projects and the NEG bridge program. ETA recognizes that a two-year project with a limited number of participants may incur a greater cost per participant than the formula program, whose costs may span more than one program year. ETA is reviewing options for a formal efficiency measure for National Emergency Grants (NEGs) and needs to determine how to rate the efficiency of: ?? "planning" grants, ?? re-employment/workforce development services grants which can vary in length from 12 to 36 months; and ?? disaster-related temporary job creation projects which can vary in length from 6 to 36 months. Recently, the program adopted a cost per participant efficiency measure and established a baseline, but has not established targets for this measure. The program will establish targets in the fourth quarter of FY 2007.

Evidence: ?? Process comparison spreadsheet; ?? GAO Report GAO-O6-870; and ?? Workforce Investment Act: National Emergency Grants - Application Procedures; Notice 69 Federal Register 23052-23080, April 27, 2004.

NO 0%
3.5

Does the program collaborate and coordinate effectively with related programs?

Explanation: NEGs are designed to make full use of WIA and Wagner-Peyser formula funds and state level Rapid Response funds. On a service delivery level, NEG projects maximize use of assistance and resources available through One-Stop partners, employers, and other federal (such as FEMA and DoD) agencies, state and local organizations. The realignment and closure of American bases (BRAC) demonstrate collaboration and coordination among ETA, Department of Defense (DoD), and state and local workforce agencies particularly utilizing the NEG BRAC planning grants. For example, one Florida region is doing a wide variety of planning and services to facilitate military growth including: a) a five WIB region Florida Defense Alliance has been created to coordinate activities; b) a joint land use study is being conducted by Eglin Air Force Base and the county; c) there is a strong veterans program in the One Stop that is linked closely to the Veteran's Affairs Vocational Rehabilitation services. This relationship has led to the creation of a Veterans Services network coordinating activities for veterans; d) Florida has dedicated general revenue dollars to assist military spouses; and e) the local university has created a Military Advisory Council to make sure that they are meeting the education needs of military members and their families. Regional Innovation Grants (RIGs) will shortly become available to encourage coordination and collaboration from key leaders in a region to assist in unanticipated economic events. RIGs will provide limited NEG funds to state workforce agencies and local Workforce Investment Boards (WIBs) to initiate comprehensive, sustainable, and integrated regional Planning using ETA's Workforce Innovation in Regional Economic Development (WIRED) conceptual framework and principles to facilitate a quality, timely, and effective system-wide response to unanticipated economic events. The response to Hurricanes Katrina and Rita in Mississippi and Louisiana in 2005 highlights the impact of effective coordination. NEGs provided the majority of DOL's assistance in the gulf coast by providing the full array of employment and training services, including the creation of temporary jobs to ensure that affected workers maximized their access to resources, ETA staff has worked with the states to ensure these funds are coordinated with other Department training efforts that include H-1B grants and the Pathways to Construction Employment Initiative. Coordination included co-enrollment in the training programs to allow workers to gain access to supportive services that are only available under NEGs.

Evidence: ?? www.doleta.gov/layoff/; ?? http://www.dol.gov/opa/media/press/eta/ETA20052160.htm; and ?? Guidance on "linkages," Federal Resources fact sheet.

YES 11%
3.6

Does the program use strong financial management practices?

Explanation: Auditors have not identified any material deficiencies in the program's financial management practices. The program collaborates with ETA budget office to reconcile its budget on a regular basis. Additionally, the program office uses the allocation and obligation information from DOLAR$ to reconcile its allocations on a monthly basis. The program has financial management review policies and procedures from pre-award through close-out to ensure funds awarded are used for the purpose of employment and training services. Grantees are required to submit quarterly financial status reports which are available to national and regional ETA staff for tracking, reviewing, and analyzing, and to provide technical assistance, if necessary. The grant agreement letter requires that grantees abide by OMB audit requirements.

Evidence: ?? The award chart; QPRs from e-system; TEGL 16-03 (4) (m); OMB Circular A-133, 29 CFR Part 96 & 99.

YES 11%
3.7

Has the program taken meaningful steps to address its management deficiencies?

Explanation: The program has taken meaningful steps by creating an automatic e-systems generated weekly "overdue" application and modifications reports to ensure the goal of processing awards within 30 working days is maintained and the goal of processing modification within 15 working days is maintained. The 2006 GAO report on timely processing of grant applications has been useful in identifying and subsequently addressing management deficiencies. Accordingly, two supervisory team leads were added to the Division in order to monitor the quality of work and expedite the processing time of applications. Additionally, DOL has tied its performance goals to performance ratings for managers. NEG time line process charts that outline expectations were created to improve processing deficiencies. NEG documents readiness checklist were created to improve reoccurring errors identified by management.

Evidence: ?? Weekly "overdue" reports; GAO-04-222; GAO-04-496; GAO-06-870; Performance Standards; NEG Readiness Checklist.

YES 11%
3.BF1

Does the program have oversight practices that provide sufficient knowledge of grantee activities?

Explanation: Each grant recipient is required to submit electronically to ETA a quarterly report form (ETA 9104) through the NEG system and in the WISARD on actual performance to date as well as a financial status report. ETA regional staff monitors grantee performance and financial practices through the analyses of this quarterly data as regular on-site and desk reviews. To assist staff with monitoring, NEG has developed a monitoring guide that will be effective in PY 2006. The guide will incorporate ETA's on-line tool called "Grants E-Management System (GEMS)." GEMS helps staff to identify immediate and potential risks in grant activities and provides access to all program data through a related information tool called, "Enterprise Business Support System (EBSS)." ETA uses performance and financial data to identify grantees that may need technical assistance. Additionally, the NEG program office at the national level holds a monthly call with regional NEG federal project officers in the regions to discuss current program and policy issues related to the program as well as to identify trends in technical assistance to grantees that may require broader training or review of certain policy issues. To ensure that NEG are effectively serving dislocated workers, regional office staff carry out a variety of monitoring activities throughout the lifecycle of the grant to track progress toward meeting its stated purpose and goals. There are general monitoring procedures for all grants but they are tailored for high-risk or complex grants. Regional staff quarterly assess the potential risk level of the grant and reviews the project operating plan, and analyze quarterly financial and progress reports to assess timeliness, accuracy, and effectiveness in providing services to dislocated workers. In addition to the Project Operating Plan that each ETA regional federal project officer reviews at the beginning of a NEG project, each grantee and its project operators will be reviewed on-site at least once by ETA staff - between the sixth month and the project midpoint. This comprehensive review typically last 2-4 days and the purpose of the review is to verify core compliance factors such as eligibility and adequate financial management, assess the effectiveness of participant service policies and processes in achieving project performance goals, and evaluate the need for funds to complete the project. Depending on the region, some high risk grantees are visited within 90 days after award and work closely with the grantee throughout the life of the award. For unusually large or visible grants, such as those that serve Hurricane Katrina evacuees, grantees are monitored more often and have to submit more extensive information more often than the quarterly reports.

Evidence: ?? NEG Monitoring Guide; ?? GEMS user Guide at http://www.etareports.doleta.gov/gems/help/table_of_contents.htm; ?? Monitoring reports; ?? Quarterly reports; and ?? Management report.

YES 11%
3.BF2

Does the program collect grantee performance data on an annual basis and make it available to the public in a transparent and meaningful manner?

Explanation: Grantees submit WIA performance information to DOL quarterly and annually. On a quarterly basis, DOL works with state grantees to review and analyze states' performance information and helps states translate their performance into policy decisions. On ETA's Web site, DOL provides an easy link to quarterly and annual performance results for the nation and individual states' performance. The information collected by the quarterly reports include a variety of performance data elements such as; participants enrolled in training; receiving supportive services; entering employment at exit, and program expenditures. The reports are then compared to the grantees planning form to track progress towards achieving proposed quarterly goals.

Evidence: ?? WIA annual performance report; ?? http://www.doleta.gov/neg/neg-data.cfm; ?? WIASRD report; and ?? The location of the common measures' report.

YES 11%
Section 3 - Program Management Score 89%
Section 4 - Program Results/Accountability
Number Question Answer Score
4.1

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: Entered employment rates have been in the mid-70 percent range for the last four years, reaching a peak of 78 percent in PY 2004 but falling again in PY 2005 to 71percent. This volatility reflects the nature of national emergency grants, which serves entirely different populations in a variety of circumstances from one year to the next. Retention - an indication of proper matching of workers with employers is far more stable and has trended upward from 85 percent in PY 2002 to 89 percent in PY 2005. Earnings replacement, like entered employment, has been volatile, but appears to be on an upward trend.

Evidence: ?? All data from WIASRD records and can be found on the web at http://doleta.gov/performance/results.

SMALL EXTENT 7%
4.2

Does the program (including program partners) achieve its annual performance goals?

Explanation: Recent performance is mixed. Entered employment rates in PY 03-05 were below targets and dropped significantly in the last period. Retention rose from 89 percent to 92 percent in PY 03-04 before returning to 89 percent in PY 05. Earnings replacement dipped sharply from 86.5 percent to 80.1 percent and then rose to a new high of 93.3 percent. There is evidence of improvements to the timeliness of processing grant applications, but the program does not have a formal baseline against to make comparisons.

Evidence: ?? Results and WIA Annual Reports are accessible at http://www.doleta.gov/performance/results/Reports.cfm?#wiastann. ?? Annual Performance and Accountability Reports are accessible at http://www.dol.gov/dol/aboutdol/main.htm.

SMALL EXTENT 7%
4.3

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: Since the program does not have a formal efficiency measure with targets (Question 3.4) it is unable to demonstrate improved efficiencies or cost-effective in achieving program goals. As mentioned in 3.4, NEG does have a timeliness measure for its grant application process - average number of days from receipt of an application to issuance of the award letter. DOL has focused on reducing the processing time for NEG awards and has reduced the average time from 50 days in 2002 to 30 days in Program Years 2004-05. According to GAO, the new electronic system has facilitated improvements in award processing time in three ways: 1. Because of the e-system's edit checks, applicants can no longer submit incomplete applications; 2. Because applications are electronic, submissions are nearly instantaneous and allow Labor and applicants to exchange information more efficiently; and 3. Under the new system, the applicants are only required to provide basic information which reduces the processing time. The NEG e-application system captures the proposed Project Operator's costs per participant from the previous Program Year's Dislocated Worker formula program. ETA uses this information as a "baseline" for the costs associated with the new grant application. When a Project Operator is proposing costs which exceed the previous year's costs, the applicant has to justify the higher level costs, e.g. the proposed target group has significantly greater barriers to re-employment that those workers served under the formula program. This allows ETA to manage the cost effectiveness of each discretionary award. ETA is reviewing options for a formal efficiency measure for National Emergency Grants (NEGs) and needs to determine how to rate the efficiency of: ?? "planning" grants, ?? re-employment/workforce development services grants which can vary in length from 12 to 36 months; and ?? disaster-related temporary job creation projects which can vary in length from 6 to 36 months.

Evidence: ?? ETA Common Measures Training and Employment Guidance Letter (TEGL) No.15-03 clarifies the efficiency measure and is available at www.wdr.doleta.gov/dorectives/TEGL15-03.pdf; ?? Information on program participants is found in the WIA Annual Report Data for PYs 2001 - 2004, which is accessible at http://www.doleta.gov/performance/results/wia_national_performance.cfm; and ?? www.gao.gov/cgi-bin/getrpt?GAO-06-870.

NO 0%
4.4

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: Although funding for NEG is linked to the WIA Dislocated Worker program, design and target population are most similar to the Trade Adjustment Assistance (TAA) program. Petitions for TAA are certified for specific worker groups; NEGs are awarded for discrete specific layoff events. In each case, the full array of re-employment services, including training, is available for both worker groups. The Dislocated Worker formula participants typically don't require as substantial a level of re-employment services as do those served by an NEG. No other Federal agency has a program that is a close fit with NEGs, and some do not have sufficient performance data for a comparison. For the most recent three comparable periods in (PY03/FY04 through PY05/FY06), NEGs compare favorably to TAA on all three of the common measures. For entered employment, NEG results were much higher in the first two years of the period and just one percent lower in the last year. Retention results are virtually identical in the first year, one point higher in the second and one point lower in the third. NEG wage replacement rates are consistently higher, with margins of 12 percentage points in the first year, four the second and 13 in the third. It should be noted, though, that the TAA program serves a traditionally older, longer tenured and more highly paid (prior to dislocation) population. The earnings replacement measure may not be an appropriate comparison. The new earnings measure (average earnings) will provide a better comparative tool, once baselines have been established. For the most recent three comparable periods (PY2003 through PY2005), NEG compares less favorably with the Dislocated Worker (DW) program, which is indicative of the differences in experiences of those impacted by emergency events and those served by dislocated worker formula funds. For entered employment, NEG results were lower in all periods, with a range of 4 to 13 percent lower. Retention results were more favorable, with a range of 1 percent lower in PY 2003, to two points higher in the other two periods. NEG wage replacement rates were consistently lower, ranging from 4 percent lower in PY2003, 12 percent lower in PY2004 and 10 percent lower in PY2005.

Evidence: ?? ETA Performance and Results information is available at http://www.doleta.gov/performance/results/Reports.cfm?#etaqr. See WIASRD Summary Reports and Trade Adjustment Assistance. Quarterly results also are available from the performance and results site.

LARGE EXTENT 13%
4.5

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: There have not been any evaluations of sufficient scope and quality to evaluate the program's effectiveness, especially its impact on participants' employment and earnings. Evaluations to date have focused on the administration,implementation processes, and timeliness of NEG applications and awards. The GAO reports analyzed the timeliness of the award and obligation of NEG funds. The initial reports, in 2004, found that the NEG application and award process needed improvement, especially in regards to timeliness. In reaction to the 2004 GAO reports, the program implemented a web based e-Application/modification process, which helped to significantly reduce the time the agency took to process NEG grant applications. The 2006 GAO report described improvements in the application and data collection processes and provided several follow-up recommendations, such as distributing more complete grant monitoring guidance and exploring cost effective ways of disseminating best practice grant management information to states and local areas. The program is reviewing these recommendations and will take action to continually improve such processes. The 2006 DOL/Mathematica report found increased employment rates among training participants, but noted that additional rigorous evaluation would be necessary to truly determine statistically significant impacts.

Evidence: ?? www.gao.gov/cgi-bin/getrpt?GAO-04-222; ?? www.gao.gov/cgi-bin/getrpt?GAO-04-496; and ?? www.gao.gov/cgi-bin/getrpt?GAO-06-870. http://wdr.doleta.gov/research/FullText_Documents/Military%20NEG%20Report%20Final%2Epdf

NO 0%
Section 4 - Program Results/Accountability Score 27%


Last updated: 09062008.2007SPR