Detailed Information on the
Distributed Energy Resources Assessment

Develop guidance that specifies a consistent framework for analyzing the costs and benefits of research and development investments, and use this information to guide budget decisions.

Develop a measure to assess performance of technical assistance activities that promote commercial deployment of distributed energy resource systems.

Focus resources on distributed energy systems integration and reduce resources for component technology development that is within industry's capability.

Measure: Cumulative number of technologies developed with 25 percent increase in energy efficiency (2000 baseline), with NOx emissions less than 0.15 lbs per MWh, and an equivalent or 10 percent reduction in cost to comparable technologies.

Explanation:The measure aims to increase energy efficiency of distributed generation and thermally activated technologies while reducing pollutant (nitrogen oxide) emissions and reducing cost, ensuring market acceptance. Deployment of the technologies can contribute to the Department's goals of increased energy efficiency and increased electricity reliability during periods of peak demand.

Measure: Cumulative number of integrated combined heat and power systems developed that will achieve 70 percent efficiency and customer payback in less than 4 years.

Explanation:The measure assesses the program's development highly efficient and cost effective CHP package systems, which can reduce baseload on the electric grid. The payback goal assumes commercial-scale production of CHP systems, which will reduce their capital costs.

Measure: Efficiency of energy conversion for prototype microturbines.

Explanation:This measures tracks the improvements in efficiency for microturbines while maintaining or reducing pollutant emissions.

Measure: Efficiency of energy conversion for reciprocating engines.

Explanation:This measures tracks the improvements in efficiency for reciprocating engines while maintaining or reducing pollutant emissions.

Measure: Amount of nitrogen oxide pollutant emitted per unit of power from small microturbines (less than 1 megawatt), in pounds per megawatt-hour (lbs/MWh).

Explanation:Environmental emissions are relevant because there is a tradeoff between efficiency and emissions.

Measure: Amount of nitrogen oxide pollutant emitted per unit of power from reciprocating engines (1-10 megawatts), in pounds per megawatt-hour (lbs/MWh).

Explanation:Environmental emissions are relevant because there is a tradeoff between efficiency and emissions.

Measure: Amount of nitrogen oxide pollutant emitted per unit of power from industrial gas turbines (1-10 megawatts), in pounds per megawatt-hour (lbs/MWh).

Explanation:Environmental emissions are relevant because there is a tradeoff between efficiency and emissions.

Measure: Efficiency of combined heat and power package systems.

Explanation:Tracks only development of the most efficient CHP package systems.

Measure: Thermally Activated Technologies.

Explanation:

Is the program purpose clear?

Explanation: The mission of the Distributed Energy Resources Program is to strengthen America's affordable energy infrastructure and provide distribution utilities and consumers with a greater array of energy efficient technology choices for generation and thermal energy. To accomplish the mission, the program funds research, development, demonstration, technology transfer, and education and outreach activities in partnership with industries, businesses, utilities, States, other Federal programs and agencies, universities, national laboratories, and other stakeholders.

Evidence: FY 2004 OMB Budget Request; numerous program publications (in print and on EREN website). Authorizing legislation includes: P.L. 93-577, "Federal Non-nuclear Energy Research and Development Act of 1974"; P.L. 94-163, "Energy Policy and Conservation Act" (EPCA) (1975); P.L. 94-91, "Department of Energy Organization Act" (1977) ; P.L. 94-385, "Energy Conservation and Production Act" (ECPA) (1976); P.L. 95-619, "National Energy Conservation Policy Act" (NECPA) (1978); P.L. 101-218, "Renewable Energy and Energy Efficiency Technology Competitiveness Act of 1989"; P.L. 102-486, "Energy Policy Act of 1992".

Does the program address a specific and existing problem, interest, or need?

Explanation: The program aims to help to upgrade America's aging electric power infrastructure, relieve congestion on transmission and distribution systems, reduce consumption and increase supplies and reliability during periods of peak demand, accelerate the introduction of advanced systems to improve the efficiency of market operations, support the transition from traditional monopoly regulation to more competitive markets, and reduce environmental emissions, including greenhouse gases. These efforts support the Administration's National Energy Policy as well as the Administration's climate change goals.

Evidence: The program focuses R&D on activities that it considers too technologically risky for the private sector to undertake alone. Risk levels vary on a project-by project basis.

Is the program designed so that it is not redundant or duplicative of any Federal, state, local or private effort?

Explanation: The program is the primary Federal program working to improve efficiency and reduce emissions from distributed generation technologies. In cases where potential overlap may exist with State activities, the program works closely with States. For example, the program's staff and California Energy Commission staff issued separate solicitations for work on microturbines, and staff from each program sat on the other's review panels to ensure that projects were synergistic. The program also works with industry to develop joint plans and establish mutual goals; other groups, such as the Electric Power Research Institute (EPRI), target areas not necessarily representing national public benefit or funded at a level necessary to mobilize national change.

Evidence: The program considers uncertain risk-to-return ratio and lack of industry capital to be market barriers to private sector investment in distributed energy technologies. The program considers its aggressive goals to improve efficiency, reduce emissions, and reduce cost simultaneously to be outside of industry's capability.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: The DER program is focused on increasing the efficiency and decreasing the emissions of distributed energy through research and development in order to achieve the outcomes of greater electricity reliability and reduced emissions of pollutant and greenhouse gases.

Evidence: A review RAND's web site (www.rand.org, energy and public policy research) has indicated no debate the relative costs and benefits of promulgating efficiency standards for generation technologies vs. conducting research and development.

Is the program effectively targeted, so program resources reach intended beneficiaries and/or otherwise address the program's purpose directly?

Explanation: In support of the Administration's R&D Investment Criteria initiative, the program was asked to prepare "bubble charts" that plot key program variables (e.g., expected public benefits, funding levels, years to commercialization). Bubble charts can serve as an informational tool to help determine, along with other considerations, whether the program appropriately targets its R&D funding. While the program has made progress estimating public benefits, the Department has not yet developed a methodology to estimate benefits consistently within and across programs. Therefore, the program could not prepare meaningful bubble charts.

Evidence: In general, the program appears to target its resources wisely, but a lack of ability to provide appropriate evidence mandates a "no" response. EERE continues to work internally and with other DOE program offices to improve consistency and accuracy in estimating benefits.

Does the program effectively articulate potential public benefits?

Explanation:  

Evidence:  

If an industry-related problem, can the program explain how the market fails to motivate private investment?

Explanation:  

Evidence:  

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: The program has two long-term goals that capture most of the activities supported in each of the two subprograms. One subprogram focuses on the development of 'next generation' distributed energy technologies (e.g., microturbines, reciprocating engines, industrial gas turbines, thermally activated cooling and humidity control devices, combined heat and power systems) that are cleaner and more reliable, fuel efficient, fuel flexible and affordable than existing equipment. The second subprogram concentrates on the development of technologies, tools, and techniques to enable prospective users of distributed energy systems - regardless of the type of technology - to evaluate benefits, install, operate, control, and maintain those systems in an optimized manner to meet the needs of their facilities and business operations, and those of the electric power and natural gas utilities to which the systems are interconnected.

Evidence: FY 2004 Budget.

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: The program has developed a Multi-Year Program Plan (July 2003) that sets out the objectives and milestones for each program activity over the 2003 - 2009 time frame and links them to the program's long-term goals. For example, one long-term measure of the program is to demonstrate three 70% efficient integrated combined heat and power (CHP) systems. (The current state of the art for these systems is approximately 60%.) The program is investing in 17 proof-of-concept demonstrations with the hopes of achieving at least 3 successes.

Evidence: Multi-Year Program Plan (FY2003-2009) for the Distributed Energy Resources Program (May 2003).

Does the program have a limited number of specific annual performance measures that demonstrate progress toward achieving the program's long-term measures?

Explanation: Annual measures are directly track to the program goals for increased efficiency and reduced pollution.

Evidence: Measures Tab.

Does the program have baselines and ambitious targets and timeframes for its annual measures?

Explanation: The targets for the annual measures are quantifiable and provide a specific value with which performance (e.g. efficiency) can be compared. Baselines are clearly defined. These targets are ambitious and approach the theoretical efficiency maxima.

Evidence: Measures Tab.

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, etc.) commit to and work toward the annual and/or long-term goals of the program?

Explanation: Technical milestones outlined in cooperative agreements with industry are intended to ensure monitoring of progress towards overall program goals. Competitive solicitations link activities to long-term goals of the program.

Evidence: Sample solicitation that documents performance goals as Objectives for solicitation. Sample contract document that includes program goals.

Are independent and quality evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: The program regularly solicits opinions from experts outside of the U.S. Department of Energy to guide decision making about program directions and priorities. To accomplish this, the program develops technology roadmaps and holds peer reviews. In addition, over the past several years, the DER program has engaged in discussions with hundreds of stakeholders - manufacturers, businesses, utilities, laboratories, universities, state agencies, and public interest groups - as part of an extensive series of visioning, roadmapping, and multi-year planning processes.

Evidence: Distributed Energy Resources Peer Review (January, 2002). Microturbine Peer Review (March 2002). Reciprocating Engine Peer Review (April 2002) Gas Turbine Peer Review (March 2002). Thermal Activation Technology Peer Review (May 2002). End-use Systems Peer Review (January 2002). Peer reviewers were independent evaluators that received no program funding. Peer review questions were standard questions documenting effectiveness and progress on research, gaps and concerns.

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: Program funding is explicitly tied to accomplishing the two long-term performance measures for the program. However, budget documents do not clearly indicate the full costs of achieving the program goals. Salaries, benefits, and other admininstrative expenses to support the program are included in a separate budgetary line item ("Policy and Management"). EERE does not report the allocation of Policy and Management funding to the various programs it supports.

Evidence: FY 2004 Budget.

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: The DER Strategic Peer Review in November, 2000, identified weak coordination with States as a strategic planning deficiency. The program implemented better coordination with the states by participating on CEC/NYSERDA solicitation reviews. The program has also prepared a draft Multi-Year Program Plan.

Evidence: DER Strategic Peer Review (November 2000), Multi-Year Program Plan (FY2003-2009) for the Distributed Energy Resources Program (May 2003).

If applicable, does the program assess and compare the potential benefits of efforts within the program to other efforts that have similar goals?

Explanation: Each year, the program estimates the public benefits of its activities in support of the Government Performance and Results Act (GPRA) and the Administration's R&D Investment Criteria initiative. However, the program has not yet developed a consistent and reliable methodology for comparing potential benefits within and across programs with similar goals.

Evidence: FY 2004 Budget

Does the program use a prioritization process to guide budget requests and funding decisions?

Explanation: At the beginning of each budget cycle, the program identifies a list of research priorities, based upon needs, issues, and trends -- not only for this budget, but for the next few years as well. These activities are ranked from lowest to highest. Without a formal Federal Advisory Committee, the program utilizes the peer review process to evaluate priorities for the program.

Evidence: The program prioritized its current activities as follows (highest to lowest priority): end-use systems integration, generation, combined heat and power, thermally activiated technologies. EERE Priority Ranking Tool, Zero Based Budget Exercise.

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: The EERE Strategic Management System -- which establishes at the beginning of each fiscal year an 18-month schedule for key planning, budget formulation, budget execution, and analysis / evaluation functions -- requires that each EERE program establish and track long-term and near-term program performance goals and measures. Program results as evaluated through the goals and measures are used annually and throughout the year to assess partners' performance, adjust funding, and re-align R&D portfolios.

Evidence: SMS Implementation Letter for FY 2002 - 2005 (October 2001). Sample Quarterly Progress Reports from Oak Ridge National Lab. Subprograms have quarterly reviews, with critical paths reviewed to ensure that program milestones are met. The program also reports on quarterly milestones in the Department's Joule database. However, in general, milestones in the Joule system are not necessarily meaningful or fully reflective of program progress. Thus, the Department's Joule system provides little value-added. The new I-MANAGE system, currently under development, will better integrate budget and performance.

Are Federal managers and program partners (grantees, subgrantees, contractors, cost-sharing partners, etc.) held accountable for cost, schedule and performance results?

Explanation: The Annual Performance Appraisals of all EERE Program Managers include criteria directly related to cost, schedule, and performance results. EERE reviews these criteria monthly in the EERE Monthly Management Reviews. Most EERE contracts include award fee and other performance criteria to hold those partners accountable.

Evidence: Performance Plan and Performance Appraisal Form for Performance Management System Employees; EERE Award Fee and Performance Based contracts; Uncosted Reports; Cooperative Agreements. The program indicates that a contracting officer the Chicago Operations office monitors spending, performance, cost and schedule, and that headquaters staff monitor laboratory funding, performance, cost, and schedule.

Are all funds (Federal and partners') obligated in a timely manner and spent for the intended purpose?

Explanation: Each year, the program develops an Annual Operating Plan, which is reviewed internally to ensure that new funding is planned to be obligated consistent with the appropriated purpose. EERE also develops a Spend Plan for all of its programs. The program uses data from Departmental procurement and financial systems -- and similar data from National Laboratory partners -- to assure that actual expenditures occur for intended purposes and on a schedule consistent with the Spend Plan. Unobligated balances brought forward to FY 2004 were $1.6 million, 2.6 percent of the program's FY 2003 appropriation of approximately $61 million.

Evidence: EERE Spend Plan; monthly obligation and cost reports from the Departmental financial systems. FY 2002 AOP and 2002 spend plan showing planned expenditures vs. actuals. FY 2004 apportionment.

Does the program have procedures (e.g., competitive sourcing/cost comparisons, IT improvements, approporaite incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: EERE's reorganization in 2002 clarified lines of responsibility and eliminated organizational "stovepipes" by consolidating planning, budgeting, and analysis into a single business administration office. The reorganization reduced management layers, although staff levels remained the same. EERE developed a new IT report to improve program managers' access to EERE cost, obligation, and procurement data. EERE plans to consolidate several legacy IT systems into a single program management system that is intended to track all required information on a project by project basis (cost share, type of contract according to A-11 definitions, etc.). EERE is also developing a measure to reduce uncosted balances, which means obligated funds will be put to use more quickly. These recent actions should achieve efficiencies and improve cost effectiveness, although it will be difficult in some cases to demonstrate definitively.

Evidence: EERE Reorganization "All Hands" presentation: www.eere.energy.gov/office_eere/pdfs/eere_reorg.pdf. EERE IT Business Case Number 019-20-01-12-01-1011-00-304-101. DER Program FY 2003 Financial Status Report (June 2003). The program notes that it uses electronic formats (e.g. CD) and the web as the primary source for information transfer to stakeholders, saving on document production costs.

Does the program collaborate and coordinate effectively with related programs?

Explanation: The program partners with EPA through an interagency agreement on combined heat and power (CHP) partnerships. The program also partners with Defense (Office of Naval Research) on materials R&D. Program staff peer review contract proposals received by the New York State Energy Research and Development Agency (NYSERDA) and the California Energy Commission (CEC). CEC's program plan for reciprocating engines shows the linkages to the DOE program.

Evidence: CEC web site: www.energy.ca.gov/distgen/equipment/reciprocating_engines/future.html; EPA web site: www.epa.gov/chp/index.htm. Sample "Funds Out Interagency Agreement" with Office of Naval Research (May, 2003).

Does the program use strong financial management practices?

Explanation: Each year, EERE develops and maintains a Spend Plan and a Measures spreadsheet that links the Spend Plan to annual and long-term goals and measures for each EERE program. The program reviews quarterly costing reports and weekly project status reports. There is no evidence of erroneous payments or statutory violations.

Evidence: Sample Quarterly Costing Report, EERE FY 2003 Spend Plan, measures spreadsheet

Has the program taken meaningful steps to address its management deficiencies?

Explanation: The National Association of Public Administrators (NAPA) found dozens of management deficiencies in the program's bureau (the Office of Energy Efficiency and Renewable Energy, or EERE) in a review published in 2000. EERE provided evidence that it addressed some of management deficiencies identified by NAPA, and has prepared a Management Action Plan that will address many of the remaining findings. While a few NAPA recommendations have not been addressed (e.g., that EERE conduct periodic audits to assure that cost-sharing partners actually provide funding they agree to), in general, EERE has taken meaningful steps to address most deficiencies.

Evidence: A Review of the Management in the Office of Energy Efficiency and Renewable Energy (NAPA, 2000). Letter Report from Assistant Secretary Garman to Chairman of the House Subcommittee on Interior and Related Agencies on implementation of NAPA recommendations (July 11, 2001). EERE Management Action Plan (August 2003)

Does the program allocate funds through a competitive, merit-based process, or, if not, does it justify funding methods and document how quality is maintained?

Explanation: All non-laboratory efforts are conducted under a competitive merit based solicitation. For the reciprocating engine program the laboratory and university funds were competed for phase I of the program. All funded activities are screened for merit on a scientific and technical basis and peer reviewed. The reciprocating engine program also conducted a capability review of the national laboratories.

Evidence: Chicago Solicitation for Microturbines and Reciprocating Engines. Review of the National Laboratories by ARES. Table showing funding allocations as per OMB Circular A-11 definitions for "Conduct of Research and Development."

Does competition encourage the participation of new/first-time performers through a fair and open application process?

Explanation:  

Evidence:  

Does the program adequately define appropriate termination points and other decision points?

Explanation:  

Evidence:  

If the program includes technology development or construction or operation of a facility, does the program clearly define deliverables and required capability/performance characteristics and appropriate, credible cost and schedule goals?

Explanation:  

Evidence:  

Has the program demonstrated adequate progress in achieving its long-term outcome performance goals?

Explanation: The program's 2002 peer review indicates that the program has generally met its short-term milestones that contribute the long-term performance goals. But the performance measures and targets were refined frequently each year before the FY 2004 Budget, making "large extent" a more reasonable response than "yes" to this question. Historically, the National Academy of Sciences notes that the program successfully completed its last long-term performance goal: develop and demonstrate an advanced industrial gas turbine (5MW) that achieves 40 percent efficiency and low NOx emissions (less than 10 lbs/MWh) by FY 2000. This program was completed in FY2000 and the program expects a commercial offering of this product in 2004.

Evidence: DER 2002 Peer Review. Energy Research at DOE: Was It Worth It? (NAS, 2000)

Does the program (including program partners) achieve its annual performance goals?

Explanation: The program's 2002 peer review indicates that the program has generally met its short-term milestones that contribute the long-term performance goals. But the performance measures and targets were refined frequently each year before the FY 2004 Budget, making "large extent" a more reasonable response than "yes" to this question.

Evidence: DER 2002 Peer Review.

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program performance goals each year?

Explanation: The program identified several activities that would seem to promote efficiency and cost-effectiveness, including: adjustable cost sharing to provide opportunities for reassessment of performance, government role and cost share with partners; integrated planning and identification of most cost effective investments/roles in R&D consortia; shifting work previously done by labs that the private sector; and developing electronic collection, storage, management and reporting systems that eliminate historic but unneeded reporting, and integrate performance, planning, fiscal and management data. The program could not provide evidence that these activities have improved efficiency and cost effectiveness.

Evidence:  

Does the performance of this program compare favorably to other programs, including government, private, etc., that have similar purpose and goals?

Explanation: No studies available comparing DER program performance with other R&D activities designed to improve energy efficiency in the Nation.

Evidence:  

Do independent and quality evaluations of this program indicate that the program is effective and achieving results?

Explanation: The DER 2002 Peer Review provided mixed, but largely positive, comments. The National Academy of Sciences reviewed one technology (Advanced Turbine Systems) developed by the program and concluded: "All in all the ATS program is a good example of a successful industry-government RD&D Program. The focus on design and build of actual equipment with a parallel supporting technology and with well-defined measurable performance goals and intermedialte milestones lead to this success." The program indicates that numerous techologies associated with the program (e.g. commerical absorption chiller; cooled silicon nitride turbine vanes; coatings extend turbine performance) have been honored by R&D Magazine as among the 100 most technologically significant products for the year. The awards indicate external recognition for program accomplishments, but it's difficult to assess their significance. It may be useful to benchmark awards/patents per dollar invested against similar applied R&D programs.

Evidence: DER 2002 Peer Review. Energy Research at DOE: Was It Worth It? (NAS, 2000)

If the program includes construction of a facility, were program goals achieved within budgeted costs and established schedules?

Explanation:  

Evidence: