Detailed Information on the
Highway Research and Development/Intelligent Transportation Systems Assessment

Reviewing the project selection process and determining whether projects funded are consistent with the priorities of the new 2007-2011 USDOT Strategic Plan

Making program improvements consistent with the recommendations of recently completed lab assessments

Measure: Annual highway fatality rate as based on the number of fatalities per 100 million vehicle miles traveled.

Explanation:Safety: The DOT and FHWA goal is to reduce highway fatality to not more than 1.0 per 100 million vehicle-miles traveled by 2008. FHWA's RD&T program has released numerous products in support of this goal. These include crash avoidance systems, improved visibility of pavements and signs, traffic signal improvements and bicycle and pedestrian safety design improvements.

Measure: Number of deliverable research products, innovations, and technologies that support the accomplishment of DOT and FHWA safety goals.

Explanation:Safety: RD&T's Safety goal is to deliver 30 safety technologies by 2008. Current deliverables include studies focused on roadway departure crashes, intersection fatalities, and pedestrian fatalities.

Measure: Number of deliverable research products, innovations, and technologies that support the accomplishment of DOT and FHWA safety goals.

Explanation:Safety: RD&T's Safety goal is to deliver 30 safety technologies by 2008. Current deliverables include studies focused on roadway departure crashes, intersection fatalities, and pedestrian fatalities.

Measure: Percentage of travel on the National Highway System (NHS) meeting pavement performance standards for "good" rated ride

Explanation:Mobility: DOT's and FHWA's goal is to increase the percentage of vehicle miles traveled on pavement with acceptable ride quality to 94.9 percent by 2008. FHWA's RD&T program has helped achieve this goal by developing long-term pavement performance guides, evaluating the use of fiber reinforced polymers in concrete, and developing procedures for estimating alkali-silica reactivity.

Measure: Number of deliverable research products, innovations, and technologies that support the accomplishment of DOT and FHWA infrastructure improvement goals.

Explanation:Mobility: RD&T's goal is to development 30 new pavement and bridge longevity technologies by 2008. Current projects include studies on weigh-in-motion pavement smoothness specifications, accelerated performance testing for superpave, and low-temperature binder characterization.

Measure: Number of deliverable research products, innovations, and technologies that support the accomplishment of DOT and FHWA infrastructure improvement goals.

Explanation:Mobility: RD&T's goal is to development 30 new pavement and bridge longevity technologies by 2008. Current projects include studies on weigh-in-motion pavement smoothness specifications, accelerated performance testing for superpave, and low-temperature binder characterization.

Measure: The percentage of urban area road travel occurring in congested conditions.

Explanation:Mobility: DOT's goal is to decrease the growth in percent of urban area road travel occurring in congested conditions by 0.2 percent annually. FHWA's research program has released numerous products that have directly contributed to the success of the DOT congestion goal.

Measure: The percentage of urban area road travel occurring in congested conditions.

Explanation:Mobility: DOT's goal is to decrease the growth in percent of urban area road travel occurring in congested conditions by 0.2 percent annually. FHWA's research program has released numerous products that have directly contributed to the success of the DOT congestion goal.

Measure: Number of deliverable research products, innovations, and technologies that support the accomplishment of DOT and FHWA congestion mitigation goals.

Explanation:Mobility: RD&T's goal is to develop at least 20 new congestion mitigation technologies by 2008. Current projects include transportation management center studies, traffic control system studies, and advanced incident management detection systems.

Measure: Number of deliverable research products, innovations, and technologies that support the accomplishment of DOT and FHWA congestion mitigation goals.

Explanation:Mobility: RD&T's goal is to develop at least 20 new congestion mitigation technologies by 2008. Current projects include transportation management center studies, traffic control system studies, and advanced incident management detection systems.

Measure: Thee ratio of wetland acres replaced versus the number of wetland acres impacted by a given transportation project.

Explanation:Human and Natural Environment: DOT has a goal of replacing at least one and a half times the amount of wetlands that are impacted by highway projects. On a program-wide basis, one and a half acres of wetlands will be replaced for every acre affected by Federal-aid Highway projects (where impacts are unavoidable). The RD&T program contributes to this goal by developing techniques to evaluate the impacts of toxic spills and techniques to manage highway runoff.

Measure: Number of deliverable research products, innovations and technology that support the accomplishment of DOT and FHWA environmental goals.

Explanation:Natural and Human Environment: RD&T's goal is to develop at least 10 new environmental preservation, mitigation activities and pollution reduction technologies by 2008. Current projects include studies on diffussion of airborne highway polutants and improvements in traffic modeling that reduce congestion and reduce emissions.

Measure: Number of deliverable research products, innovations and technology that support the accomplishment of DOT and FHWA environmental goals.

Explanation:Natural and Human Environment: RD&T's goal is to develop at least 10 new environmental preservation, mitigation activities and pollution reduction technologies by 2008. Current projects include studies on diffussion of airborne highway polutants and improvements in traffic modeling that reduce congestion and reduce emissions.

Measure: Number of deliverable research products, innovations and technology that support the accomplishment of DOT and FHWA security goals.

Explanation:Security: RD&T's goal is to develop 10 new national security and system preservation technologies by 2008. Current projects include studies using Global Positioning Satellite observation systems to track vehicle movement across the country and use of security systems at bridges and tunnels.

Measure: Number of deliverable research products, innovations and technology that support the accomplishment of DOT and FHWA security goals.

Explanation:Security: RD&T's goal is to develop 10 new national security and system preservation technologies by 2008. Current projects include studies using Global Positioning Satellite observation systems to track vehicle movement across the country and use of security systems at bridges and tunnels.

Measure: The annual percent of research project deliverables that are completed on time.

Explanation:Organizational Excellence: RD&T's goal is to deliver their research projects on time at least 90% of the time. Timeliness is measured based on established completion dates set during the corporate planning process and the actual completion of the project research and the delivery of the final report.

Measure: The annual percent of all research projects that are completed within budget.

Explanation:Organizational Excellence: RD&T's goal is to delivery research services within budget at least 90% of the time. Costs are measured based on established budgets set during the corporate planning process. These are compared to the actual expenses of the research project upon the completion of the project.

Measure: The level of customer satisfaction for deploying technology and innovation as measured by the FHWA State and Local Partners Satisfaction Survey.

Explanation:Organizational Excellence: RD&T's target is to continually improve customer satisfaction and to achieve a satisfaction level of at least 80%. The survey is issued bi-annually to one quarter of FHWA's transportation partners - State DOT's and Local MPO's.

Is the program purpose clear?

Explanation: The Research, Development, and Technology Program's (RD&T) mission is to: Lead in developing a nationally coordinated R&T program; champion the advancement of highway technological innovation in support of FHWA strategic goals and performance objectives (primarily improving highway safety); advance knowledge through research, development, testing, and evaluation services; and provide support and assistance throughout FHWA in matters relating to RD&T. The Federal Highway Administration (FHWA) also conducts research through the Intelligent Transportation Systems (ITS) program, which is a cross-cutting program within DOT. ITS research focuses on development and operational tests of intelligent vehicles and intelligent infrastructure systems, such as adaptive signal controls and ramp metering.

Evidence: The RD&T authorization is available at Title 23 of the U.S. Code, chapter 5, Section 502 is available at www4.law.cornell.edu/uscode/23/502.html. The specific responsibilities of the FHWA RD&T are found in FHWA ORDER M 1100.1A, Chg. 28, Chapter 13, dated September 4, 2002. The program's mission statement is found in The FHWA FY-2004-5 RD&T Performance Plan is posted at: www.tfhrc.gov/about/03085/index.htm.

Does the program address a specific and existing problem, interest or need?

Explanation: Traffic congestion on the nation's highways is worsening, and potential to be involved a roadway accident remains a serious risk. Program offices within FHWA (e.g., safety, environment, planning) as well as the Office of Research and Technology direct the research program work on the most critical issues affecting highway users.

Evidence: Highway statistics are found in the FHWA Strategic Plan: at:www.fhwa.dot.gov/policy/fhplan.html. Also see the 2002 FHWA Conditions and Performance report: www.fhwa.dot.gov/policy/2002cpr/.

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: FHWA's work most closely coincides with the research programs of state governments and universities (a portion of the annual Federal highway funding provided to states is reserved for research). The program uses several mechanisms to avoid duplicative efforts. The National Academy of Science's Transportation Research Board (TRB) and FHWA have developed a Transportation Research Information System (TRIS) and a Research In Progress Database. DOT and non-federal researchers considering projects must consult these databases to identify similar efforts. FHWA also participates in the National Cooperative Highway Research Program (NCHRP) to help ensure that it does not duplicate state work. Further, FHWA coordinates within and outside of the Department (for example, the President's National Science and Technology Council Committee on Technology). Local governments do not conduct national highway research, and private research is mostly product related.

Evidence: The Transportation Research Information Services is on-line at www.ntl.bts.gov/tris and is described at www.trb.org/trb/tris-nst/web/tris-online. NCHRP is a voluntary program where states, working with AASHTO, pool funds to conduct applied research; see www4.trb.org/trb/crp.nsf/reference/appendices/NCHRP+Overview. Also see the National Science and Technology Council's National Transportation Science and Technology Strategy at www.volpe.dot.gov/infosrc/strtplns/nstc/strategy99/ntsts99.pdf; the Technology Plan www.volpe.dot.gov/infosrc/strtplns/nstc/nttplan/index.html#toc; and Research Plan www.volpe.dot.gov/infosrc/strtplns/nstc/srplan00/index.html.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: In response to GAO and TRB recommendations, RD&T has improved its processes for setting its program agenda and evaluating its research. Specifically, FHWA has developed a Research & Technology (R&T) Corporate Master Plan that outlines a strategy for investing in and conducting research on behalf of FHWA stakeholders. Stakeholders are now invited to participate in agenda-setting and planning, research implementation, and evaluation of projects. FHWA also implemented an assessment program of RD&T's various labs, which is conducted by independent, expert reviewers. Further, the Corporate Master Plan incorporates findings by the Volpe Center (an independent evaluation organization within DOT) of best practices of other Federal research agencies (e.g., NIH, NIST and NSF) for improving stakeholder involvement, program reviews, and project evaluations.

Evidence: The GAO report 'Highway Research: Systematic Selection and Evaluation Processes Needed for Research Program (GAO-02-573)' is available at www.gao.gov/. In it, GAO notes that two characteristics leading research programs are, 1) developing research agendas through the involvement of external stkeholders, and 2) evaluating research using expert review of the quality of research outcomes. The U.S. DOT's FY-2004 Performance Plan is available on-line at www.dot.gov/PerfPlan2004/index.html. The FHWA's Corporate Master Plan for Research is posted at :www.fhwa.dot.gov/legsregs/directives/policy/cmp/03077.htm. For DOT's lab assesment schedule, see www.tfhrc.gov/services/labroadmap04.htm

Is the program effectively targeted, so that resources will reach intended beneficiaries and/or otherwise address the program's purpose directly?

Explanation: In 2004, 47 percent of RD&T program's budget was earmarked by Congress for specific projects. Earmarking limits the program's ability to conduct research on the most relevant subjects and in the most effective manner. Additionally, within the RD&T budget, Congress allocates funds to different program areas (e.g., safety, environment, infrastructure), which limits FHWA's flexibility in managing its resources. Otherwise, FHWA has made improvements in involving program stakeholders (primarily state and local highway departments) in designing its research agenda. DOT communicates its findings through websites, technical journals, technical reports, and a catalog of products and services. FHWA customer surveys indicate high levels of satisfaction with research products and deployment services.

Evidence: See the FHWA FY-2002-3 RD&T Performance Plan www.tfhrc.gov/about/perfplan/home.htm; the FHWA's Corporate Master Plan for RD&T www.fhwa.dot.gov/legsregs/directives/policy/cmp/03077.htm; the RD&T Catalog of Products tfhrc.gov/about/perfplan/products.htm; and the RD&T Catalog of Services at tfhrc.gov/about/perfplan/services.htm

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: RD&T goals include both the FHWA strategic goals and product development goals specific to research (DOT has not officially tracked and reported product development goals until now). For example, RD&T contributes to the achievement of the FHWA strategic goal of reducing fatalities per 100 million vehicle-miles of travel is facilitated. To support this goal, RD&T plans to develop at least 30 new safety-related innovations or projects by 2008. This output measure reflects the program's productivity in delivering technologies that help improve highway safety. RD&T also has productivity goals for FHWA's other strategic goals (mobility, security, environment, global connectivity, and organizational excellence). FHWA management actively links strategic goals and product development goals through multi-year program plans, referred to as "road maps."

Evidence: See the FHWA Strategic Plan www.fhwa.dot.gov/policy/fhplan.html; The U.S. DOT's FY-2004 Performance Plan: www.dot.gov/PerfPlan2004/index.html; FHWA's Performance Plan website www.fhwa.dot.gov///reports/2003plan/fy03fhwaplan.pdf. Note the DOT / FHWA / RD&T Program Goal matrix. For a general discussion of the development of RD&T's performance measures see the NCHRP reporht "Performance Measures for Research, Development, and Technology Programs" at gulliver.trb.org/publications/nchrp/nchrp_syn_300.pdf (page 26).

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: RD&T strives to develop new products, technologies, and innovations that are deployed by FHWA. The program manages the development of these products using multi-year program plans, or "road maps," for 63 different research areas. These plans include ambitious milestones, schedules, and annual performance targets for the delivery of research products. Examples of recently completed products include a traffic noise model, ground penetrating radar, and 511 traveler information technology. Based on the road maps, RD&T has established targets for the number of project deliverables. The number of deliverables is also set based on DOT's ability to meet strategic goals and on the financial resources available. RD&T also measures customer satisfaction to gauge its effectiveness in delivering its products.

Evidence: See the 2003-2008 U.S. DOT Strategic Plan: www.dot.gov/stratplan2008/strategic_plan.htm; and the FHWA's RD&T Corporate Master Plan www.fhwa.dot.gov/legsregs/directives/policy/cmp/03077.htm. Note U.S. DOT / FHWA / RD&T Program Goal matrix.

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: RD&T aims to meet yearly targets of the FHWA strategic goals and RD&T productivity goals. For each FHWA strategic goal, a given target number of deliverable products and technologies has been established. These products are described in the road maps created for each research program.

Evidence: The U.S. DOT's FY-2004 Performance Plan is available at: www.dot.gov/PerfPlan2004/index.html. The FHWA FY-2002-3 RD&T Performance Plan: www.tfhrc.gov/about/perfplan/home/htm The FHWA FY-2004-5 RD&T Performance Plan is a available at: www.tfhrc.gov/about/03085/index.htm.

Does the program have baselines and ambitious targets for its annual measures?

Explanation: FHWA has baselines and targets for its annual research measures, which are maintained in the research road maps and reported in the annual RD&T performance reports. The roadmaps establish ambitious programs regarding the type of research (like greater emphasis on advanced research), timelines, and deliverables. Targets are set for the number of new products, technologies, and innovations that are deployed by FHWA. A baseline of deliverables has been established within each FHWA strategic goal.

Evidence: The FHWA's Corporate Master Plan for RD&T: www.fhwa.dot.gov/legsregs/directives/policy/cmp/03077.htm The U.S. DOT's FY-2004 Performance Plan: www.dot.gov/PerfPlan2004/index.html The FHWA FY-2004 Performance Plan and FY-2004-5 RD&T Performance Plan: www.tfhrc.gov/about/03085/index.htm U.S. DOT / FHWA / RD&T Program Goal matrix.

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: All partners assist in achieving annual and long term goals. Approximately 70 percent of FHWA's research work is conducted by contractors, with the rest done by permanent Federal staff. FHWA contracts describe the relationship between a research project and the agency's long-term strategic goals. When contracts are awarded, the FHWA contracting officer meets with the contractor to review the terms of the contract and performance expectations.

Evidence: See the Federal Highway Administration Executive Acquisition Handbook. Also see the Federal Highway Administration Contracting Officer's Technical Representative Reference Guide. The SAFETEA proposal is available on-line at: www.fhwa.dot.gov/reauthorization/safetea_bill.pdf

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: FHWA routinely assesses its "in house" laboratories at the Turner-Fairbank Highway Research Center. Six labs are reviewed each year, and each lab is reviewed once every four years. The reviews are performed by outside research experts who do not participate in FHWA highway research contracts. They systematically review the processes, techniques, standards, staffing, project selection, and quality control practices of each laboratory. For work conducted externally by contractors or universities, FWHA hires third-party contractors to help oversee, evaluate, and manage the work. Further, FHWA invites the Transportation Research Board's Research and Technology Coordinating Committee to provide periodic (three times per year) advice on the overall structure and quality of the FHWA RD&T Program. Moreover, FHWA has hired had independent contractors to validate research designs and document program benefits.

Evidence: Information the lab assesments is available at http://www.tfhrc.gov/services/labassessmentprocess.htm. The lab assessment schedule is available at http://www.tfhrc.gov/services/labroadmap04.htm. Information on the Turner Fairbank's R&D facility is available at http://www.tfhrc.gov/. The most recent Transportation Research Board Research and Technology Coordinating Committee Operating Plan (June 2003) is available on-line as Appendix B of the September letter report http://gulliver.trb.org/publications/reports/rtcc_sept_2003.pdf. The TRB's Research and Technology Coordinating Committee letter reports are posted on TRB's website at http://www4.trb.org/trb/homepage.nsf/web/rtcc?OpenDocument.

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: Agency and program performance objectives are thoroughly incorporated into the RD&T budget development process. The RD&T budget is founded on multi-year "road maps," which include resource information and are reviewed annually by RD&T management with stakeholder input. Stakeholder input on user needs is balanced by expert technical advice regarding scientific merit and likelihood of success. RD&T selects projects that are most likely to contribute to program goals. For presentation, the RD&T budget request is integrated in the FHWA's budget request to Congress, and research is recognized as a "tool" for achieving the agency's strategic goals. However, the budget requests do not make clear how changes to the research budget would affect the achievement of strategic and research productivity goals.

Evidence: See the U.S. DOT's FY-2004 Performance Plan: www.dot.gov/PerfPlan2004/index.html; Excerpts from FHWA's FY-2004 and 2005 Budget Requests to the Congress. Note work sheets for the FY-2005 Budget Request: Programs and Priority Areas contributing to accomplishment of Departmental Goals. The FHWA's Corporate Master Plan for RD&T and Innovation is available at www.fhwa.dot.gov/legsregs/directives/policy/cmp/03077.htm.

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: Responding to recommendations by GAO and TRB, RD&T has developed a Corporate Master Plan for RD&T and Innovation to improve management of the program. RD&T has implemented a number of the action items from the plan including instituting a lab review system, using research "road maps" that include detail schedule and budget information, and increasing outreach to research stakeholders in the project selection and priority setting process. For example, the road map for Highway Needs and Investment Analysis describes stakeholder outreach efforts to industry, academia and State DOTs. The Office of RD&T is a winner of the FHWA Quality Breakthrough Award, recognizing significant progress in developing clear plans, building sound processes, and achieving measurable results. TRB's Research and Technology Coordinating Committee has commended the FHWA for its progress on the implementation of the Corporate Master Plan.

Evidence: See the Transportation Research Board Special Report 261, "The Federal Role in Highway Research and Technology," available at gulliver.trb.org/publications/sr/sr261.pdf. For latest TRB letter report on the RD&T program see trb.org/publications/reports/rtcc_sept_2003.pdf. Also See the FHWA Corporate Master Plan for Research and Deployment of Technology and Innovation, posted at :www.fhwa.dot.gov/legsregs/directives/policy/cmp/03077.htm

If applicable, does the program assess and compare the potential benefits of efforts within the program to other efforts that have similar goals?

Explanation: FHWA and state researchers are required to consult the TRB Transportation Research Information System and Research in Progress databases to identify similar projects. If a subject has been studied previously, the new study is not approved, unless it will advance earlier work. The FHWA's RD&T Leadership Team also reviews the multi-year road maps to avoid duplication with other DOT work. Within FHWA program offices, senior management annually reviews research proposals (road maps), prioritizing projects based on their merits and their potential to improve agency performance. Further, DOT's R&T Coordinating Council (RTCC) compares the potential benefits of the research with efforts of other modal administrations. FHWA has also contracted a review of approaches to research and technology development, evaluation, and deployment used by other agencies to find best practices.

Evidence: FY-2004-5 RD&T Performance Plan: www.tfhrc.gov/about/03085/index.htm; FY-2003 RD&T Performance Report: www.tfhrc.gov/about/04083/index.htm; The Volpe Center (Annalynn Lacombe) review of R&D Planning Models for FHWA. A summary of the material appears as Appendix B of the Corporate Master Plan www.fhwa.dot.gov/legsregs/directives/policy/cmp/03077.htm; Assessing Research and Development at Federal Agencies Through Peer Review; SAIC; October 2001

Does the program use a prioritization process to guide budget requests and funding decisions?

Explanation: The FHWA Administrator has designated three program goals ' referred to as the 'Vital Few' ' as emphasis areas for the agency. When developing budget requests, research programs that support these goals are given higher priority. When funds are appropriated, each FHWA program office is responsible for managing a share of the FHWA research budget. To allocate funding to the various program offices, the R&T leadership team uses the Vital Few and information about the agency's progress towards meeting its long-term goals. The individual research program offices complete the prioritization level down to individual projects based on their individual research road maps contribution to specific office goals and program demands.

Evidence: The "Vital Few" priorities (safety, congestioin mitigation, and environmental stewardship and streamlining) are discussed in the FHWA FY-2003 Performance Plan available at www.fhwa.dot.gov///reports/2003plan/fy03fhwaplan.pdf. The minutes of the FHWA Leadership Team on December 13, 2001 describe how the Vital Few goals were selected. Note excerpts from the FHWA 2005 Budget Request submitted to Congress. See FHWA's Corporate Master Plan for Research and Deployment of Technology and Innovation www.fhwa.dot.gov/legsregs/directives/policy/cmp/03077.htm.

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: FHWA uses performance data from the National Highway Traffic Safety Administration's Fatality Accident Reporting System, the Highway Safety Information System, and the FHWA Highway Performance Monitoring System. DOT collects Intelligent Transportation Systems data from at least ten metropolitan sites. RD&T also collects customer satisfaction data annually. It employs the Baldridge management system to assess program performance for a wide range of indicators and uses performance data to direct its research agenda. In administering contracts, RD&T collects time and cost data and tracks project deliverables such as the release of reports and the deployment of new technologies. These deliverables are the basis of RD&T's performance measurement system.

Evidence: Note sample outputs from the RD&T Program tracking system. The program's organizational performance management framework (Baldridge) is found at www.tfhrc.gov/about/perfplan/perfmgmtfrmwrk.htm. Note the FHWA Satisfaction Survey: State/Other Local Partners National Report, Wave 1&3 Results, Pacific Consulting Group. Approximately 13 states are surveyed in each wave every 6 months (the goal is to complete all four waves in 2 years). The State and local partner survey results help FWHA Division offices improve local performance while national data identifies areas where program offices can address systemic concerns.

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: Through annual personnel evaluations, FHWA managers are held accountable for the success and performance of research and development activities. For program partners, RD&T has established a computerized tracking system to track contracts and provide routine notices of deadlines to DOT contracting officers. This system has resulted in a significant decrease in cost and time extensions. Most contracts include standardized language requiring quarterly and annual progress reports from the contractor. FHWA contracting officers evaluate the performance of contractors in terms of quality, time, and money (i.e., did they produce a quality product on time and within budget), which is considered when future contracts are awarded.

Evidence: Note the FHWA Executive Acquisition Handbook and the FHWA Contracting Officer's Technical Representative (COTR) Reference Guide. See the Office of Federal Procurement Policy's Guide to Best Practices for Collecting and Using Current and Past Performance Information at www.acqnet.gov/Library/OFPP/BestPractices/pastpeformguide.htm. Also see the Senior Procurement Executive's News for June 21, 2000: www.dot.gov/ost/m60/062100.htm. See the lab assessment schedule at www.tfhrc.gov/services/labroadmap04.htm

Are funds (Federal and partners') obligated in a timely manner and spent for the intended purpose?

Explanation: The RD&T allocates funds to its labs in accordance with an annual spending plan that is built on program road maps. No incidents of fraud, waste, or abuse are currently under investigation by the U.S DOT's Inspector General's office. The Office works closely with the FHWA Budget Division to ensure budget submissions are supportive of administration, departmental, and agency goals and utilized in accordance with established laws, regulations, and policies.

Evidence: Samples of the multi-year program plans implementing the Corporate Master Plan; Outputs from the RD&T Program tracking system; Sample Spending Plan for Highway Operations research in FY-2003.

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: Each research office uses the RD&T project tracking system to monitor whether projects are 1) on time and, 2) within budget. RD&T's computerized tracking system actively monitor project costs and provides routine notices of deadlines to FWHA contracting officers. The system has resulted in a significant decrease in cost and time extensions. Additionally, FHWA is implementing a managerial cost accounting system and supporting processes within the agency to align overhead costs and specific project activities to national goals.

Evidence: See the Federal Highway Administration Executive Acquisition Handbook. Also see the Federal Highway Administration Contracting Officer's Technical Representative (COTR) Reference Guide.

Does the program collaborate and coordinate effectively with related programs?

Explanation: For example, under the Transportation Pooled Fund Program, FHWA and State DOTs combine resources to pursue R&T projects of mutual interest. There are currently 119 active FHWA-led pooled fund studies. Successful pooled fund studies have documented the impact of congestion on High Occupancy Vehicles (HOV) Systems and developed a promotional campaign to improve safety in work zones. Other studies revised the methodology for analyzing left turn operations currently included in the Highway Capacity Manual, analyzed pavement deterioration, and revised the guidelines for "Superpave" mixtures. RD&T Performance Reports document additional examples of succesful collaberations with other agencies, including the development of High Performance Steel, developed with the U.S. Navy and the American Iron and Steel Institute. Further, DOT participates in the NCHRP, which is a voluntary program where states, working with AASHTO, pool funds to conduct applied research.

Evidence: For information on pooled fund projects see http://www.pooledfund.org/. For information on the NCHRP, see Federal-Aid Policy Guide, Title 23 - Code of Federal Regulations: http://www4.trb.org/trb/crp.nsf/reference/appendices/NCHRP+Overview. The Transportation Research Information Service is available at: www.ntl.bts.gov/tris and www.trb.org/trb/tris-nst/web/tris-online. Also see the National Partnership Report: 'Making the Case for Greater Investment' http://gulliver.trb.org/publications/rtforum/HwyRandT.pdf & http:// www.pooledfund.org.

Does the program use strong financial management practices?

Explanation: The RD&T program has own Accounting and Budgeting System to track all financial transactions. Individual research offices can access to the Accounting and Budgeting System to check on research, development, and technology records. In addition, the program uses its project tracking system to monitor whether projects are on time and within budget. These systems complement FHWA's higher-level financial management system (Delphi), which does not track contract-level data. The FHWA is also developing a managerial cost accounting system and is working to further align overhead costs and specific project activities to national goals.

Evidence: Note outputs from the RD&T Program tracking system, and sample spending plan for Highway Operations research in FY-2003.

Has the program taken meaningful steps to address its management deficiencies?

Explanation: RD&T's executive Leadership Team, sponsored by the FHWA Administrator and composed of FHWA Associate Administrators, directs the implementation of programmatic and management improvements. The Leadership Team has written an 2004 Workforce Plan update and takes part in other initiatives relating to the President's Management Agenda (i.e., financial management, human capital, competitive sourcing, budget and performance integration, electronic government). Additionally, regular lab assessments address human capital issues within the labs such as preserving institutional memory, utilizing contractor support, and retaining and rewarding a high-quality workforce.

Evidence: For information on the R&T leadership team see www.fhwa.dot.gov/rnt4u/charter.htm. For information on FWHA's lab review process see www.tfhrc.gov/services/labassessmentprocess.htm

For R&D programs other than competitive grants programs, does the program allocate funds and use management processes that maintain program quality?

Explanation: To maintain program quality, RD&T uses merit reviews to analyze research proposals and to evaluate program performance. RD&T vets research proposals through a formal peer review process, where independent reviewers from private industry, academia, and state DOTs consider the merits of a project before it is reviewed by RD&T senior management through program road maps. When evaluating its in-house work, FHWA employs lab assessments conducted by outside experts who provide independent feedback on improving the quality and performance of laboratory research and services, and whether the research program is appropriately structured. For off-site contracted work, FHWA hires third-party contractors to review applications, guide the design, and evaluate the quality of final products. Additionally, FHWA has outside firms perform broad-based evaluations of program benefits.

Evidence: For information on review of research proposals see Appendix C of the Corporate Master Plan posted at www.fhwa.dot.gov/legsregs/directives/policy/cmp/03077.htm. Peer review input is included in program road maps; Also seet the Laboratory Assessment Process Handbook for Expert/Peer Reviews at the Turner-Fairbank Highway Research Center, Version 1.1, December 2003. The lab assessment schedule is available at www.tfhrc.gov/services/labroadmap04.htm and general information the lab assesments is available at www.tfhrc.gov/services/labassessmentprocess.htm. For information about FHWA contracting practices, see the FHWA's Executive Acquisition Handbook and FHWA's Contracting Officer's Technical Representative (COTR) Reference Guide.

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: RD&T is on track to meet all of its long-term performance goals. For example, the fatality rate per 100 million vehicle miles traveled has decreased from 3.3 in 1980 to 1.47 in 2003, with a long-term target of 1.0 in 2008. In support of this strategic goal, RD&T plans to produce 30 new highway safety technologies by 2008. Third party reviews have affirmed the importance of research in enabling the agency to achieve its larger goals. AASHTO's Strategic Highway Safety Plan acknowledges the contribution of research to improved highway safety statistics. Likewise, the Texas Transportation Institute found that R&T solutions ' including ramp meters and signal coordination ' reduced annual per capita commuter delays by 26 hours in 2001 to 24 hours in 2003.

Evidence: The FHWA RD&T FY-2003 RD&T Performance Report is available at: www.tfhrc.gov/about/04083/index.htm; The American Association of State Highway and Transportation Officials' Strategic Highway Safety Plan: safety.transportation.org includes a sub-page specifically devoted to FHWA's safety research; Also see the 2003 Texas Transportation Institute Urban mobility.tamu.edu/ums/news_release

Does the program (including program partners) achieve its annual performance goals?

Explanation: RD&T has achieved its annual product development goals, based on the data available since 2002. However, FHWA has not met all of the annual targets for its strategic goals. Note that the "real world" effects of RD&T technologies are often not realized for several years because of the time required to deploy technologies. An example is use of rumble strips, which are proven to reduce of risk of drivers leaving the roadway. Originally, they were installed on interstates, but are now being installed on rural roads where most crashes occur. Increased use of this technology should yield improved safety statistics in future years.

Evidence: The FHWA RD&T FY-2003 Performance Report is available at:www.tfhrc.gov/about/04083/index.htm

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: RD&T has met one of its two efficiency goals -- percent of projects completed on budget. Going forward, the program has implemented a computerized contract tracking system, which should lead to fewer cost overruns and requests for contract time extensions. This new system should help RD&T meet its timeliness measure in 2004. In terms of effectiveness, customer satisfaction surveys indicate that state and local partners rate the program at 71 percent, which slightly exceeds the 2004 target.

Evidence: The FHWA RD&T FY-2003 Performance Report is available at:www.tfhrc.gov/about/04083/index.htm. Also note that the latest FHWA Satisfaction Survey found 68% of State and local pargners rate FHWA 4 or 5 on the 5-point scale.

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: RD&T has taken several steps to adopt best practices that ensure relavent and high-quality research. Key process improvements include managing through multi-year program plans and using independent expert lab assessments. These lab assessment are modeled after the evaluation procecess of other federal agencies, including NSF and DOE. In addition, FHWA has implemented best practices identified by the Volpe Center, such as improving stakeholders involvement, employing merit reviews, and conducting on-going R&D evaluations. FHWA conducts regular surveys of it customers and receives strong feedback. FHWA's RD&T Performance Management Framework is featured on OPM's website as an example for other Federal agencies, and was reviewed favorably in NCHRP's Synthesis Report #300. GAO has also cited the program's planning efforts as a model for other agencies.

Evidence: From the latest FHWA Satisfaction Survey, 68% of State and local partners rate FHWA 4 or 5 on the 5-point scale. Note SAIC's report "Assessing Research and Development at Federal Agencies Through Peer Review," October 2001. Appendix B of the Corporate Master Plan summarizes the Volpe Center review of R&D Planning Models, available on-line at www.fhwa.dot.gov/legsregs/directives/policy/cmp/03077.htm. OPM's description of FHWA's RD&T Performance Management Framework is atapps.opm.gov/perform/clearing/clearing.cfm?id=124. Also see GAO report 01-822 "Combating Terrorism" for discussion the RD&T's planning process. The NCHRP study on RD&T program and performance management is at gulliver.trb.org/publications/nchrp/nchrp_syn_300.pdf

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: Independent contractors (Battelle, Cambridge Systematics, and the University of North Carolina, for example) have conductd studies documenting research benefits in safety, operations, congestion, work zones, and infrastructure. Collectively the studies worked with universities, trade associations, consultants, county governments, and over 30 state DOTs. Batelle concluded that, "programs and projects achieved their research objectives and have developed products that are widely used by states, local agencies and other institutions," and that, "estimated cost savings to the public is more than ten times the annual research funding." A 2003 study by the Texas Transportation Institute found that RD&T solutions have significantly decreased congestion. A TRB Research Committee stated that 'highway research has yielded many advances and innovations that have contributed to improvements in all aspects of highway development."

Evidence: See the Batelle study, "Recommend Measures of the Benefits of Infrastructure RD&T" (April 2003). Also note the UNC Highway Safety Research Center report, "Evaluation of Highway Safety Information System-III," (August 2001), which concluded the system was extensively used, met the needs of safety researchers, and had proven to be an effective, accepted tool for synthesizing and analyzing safety data. Cambridge Systematics found the Quickzone software tool for highway work zones is easy to learn and use, relevant to user needs, and typically results in an overall increase in agency efficiency. Also note results of the FHWA Satisfaction Survey: State/Other Local Partners National Report, Wave 1 & 3 Results, by Pacific Consulting Group. Also see the 2003 Texas Transportation Institute Urban Mobility Study at http://mobility.tamu.edu/ums/report/ for information on the usefullness of R&T projects.