Detailed Information on the
Science and Technology: Standards Development for Homeland Security Technology Assessment

Conduct assessment of program and project milestones.

As the program was scored adequate, the FY 2006 Budget decreased funding by $3 million.

The Administration will await the results of the program evaluation and analysis process that the Directorate is developing. That process will evaluate the progress that each Portfolio makes toward achieving their respective goals and remedying any deficiencies. Once that process is complete, it is expected that this Portfolio will achieve an increased PART score once it is reassessed.

Initiate a Department of Homeland Security Standards Council. The council will gather requirements from all of Department of Homeland Security and review standards submitted for adoption and recommended by standards working groups. The council has representatives from all Department of Homeland Security elements and makes recommendations to the Standards Executive.

Develop a five-year research and development plan.

Develop an FY 2007 Execution Plan.

Conduct assessment of program and project milestones.

Develop 5 Year Research and Development Plan.

Measure: Percentage of Testing & Evaluation and Standards Division milestones that are met, as established in the fiscal year's budget execution plan.

Explanation:The program has established a set of milestones that are necessary for achieving the goals and objectives of the program. These milestones are presented in the program's portion of the Science and Technology Directorate's fiscal year budget execution plan, which details the allocation of dollars and projected accomplishments for the year.

Measure: Establish and accredit a network of private/public lasbs to perform testing, evaluation and certification of WMD emergency response technologies.

Explanation:the primary goal of the standards portfolio is to develop and adopt homeland security standards.

Measure: Number of Department of Homeland Security official technical standards introduced.

Explanation:

Measure: Percent of standards introduced that are adopted by Department of Homeland Security and partner agencies. - These measures are from FYHSP.

Explanation:

Measure: Percentage of Testing & Evaluation and Standards research program costs allocated for administration. (New measure added, August 2007)

Explanation:This measure identifies the percentage of research and development funding that is being used for strictly for administration of the program which includes travel, SETA costs, IPAs, etc. This measure is an excellent indicator of indirect versus direct costs.

Is the program purpose clear?

Explanation: The Standards program mission is to develop and coordinate the adoption of National standards and appropriate evaluation methods to meet Homeland Security mission needs

Evidence: The Homeland Security Act of 2002; Homeland Security Presidential Directive 8, Sections 14 and 15

Does the program address a specific and existing problem, interest or need?

Explanation: There is an urgent need to develop and implement standards, and test and evaluation protocols for technology used to support the homeland security mission(i.e., technology used for detecting, mitigating, and recovering from terrorist attacks in support of, or used by state and local stakeholders).

Evidence: The Homeland Security Act of 2002; Homeland Security Presidential Directive 8, Sections 14 and 15

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: One of the mission objectives of this program is to coordinate standards development with other Federal agencies, US Standards committees, and international partners.

Evidence: DHS Management Directive for Standards Process (defining interagency and intragency standards coordination) in Clearance Stage, American National Standards Institute Homeland Security Standards Panel (ANSI HSSP); American Society for Testing and Materials (ASTM) Homeland Security Applications Committee activities, Interagency Task Force on Bacillus Anthracis

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: Strong emphasis is placed on the integration of activities at the Federal, State, and Local level. The standards portfolio expands its impact by leveraging on resources of other US government entitities including the National Instititute of Standards and Technology (NIST), Department of Defense (DoD), Food and Drug Administration (FDA), Environmental Protection Agency (EPA), and the Department of Energy (DOE). In addition the standards portfolio leverages on the resources of the private sector by using the voluntary consensus standards process mandated by OMB. The program is designed to coordinate activities at all levels (federal, state, local) and to leverage on existing available resources

Evidence: OMB Circular 119, National Technology Transfer Act of 1995, Three DHS management directives.

Is the program effectively targeted, so that resources will reach intended beneficiaries and/or otherwise address the program's purpose directly?

Explanation: The Standards program as structured ensures the active engagement of federal, state, and local first responders. Engagement is critical in developing effective standards for equipment and sytems that detect, protect, prevent, respond, and aid in recovery, and attribution. This program provides consistent and verifiable measures of effectiveness of homeland security-related technologies, operators, and systems in terms of basic functionality, interoperability, efficiency, and sustainability. The development of guidelines is performed in conjunction with both users and developers.

Evidence: Three DHS management directives govern the process and establish homeland security standards working groups in key homeland security subject areas. ANSI Standards and Test and Evaluation Protocols for Radiological/Nuclear Detection ; American Association of Analytical Chemists (AOAC) Performance and Official Methods for Detection of Bacillus Anthracis using hand-held immunoassays, Adoption of National Institute of Occupational Safety and Health (NIOSH) and National Fire Protection Association (NFPA) respiratory protection standards, Adoption of International Committee for Information Technology Standards (INCITS) Facial Photograph standards for DHS US VISIT Program.

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: Establish an integrated infrastructure for determining and developing standards, and test and evaluation protocols for technology used for detecting, mitigating, and recovering from terrorist attacks and also to support other Departmental components' technologies. Provide consistent and verifiable measures of effectiveness of homeland security-related technologies, operators, and systems in terms of basic functionality, interoperability, efficiency, and sustainability. Facilitate the development of guidelines in conjunction with both users and developers.

Evidence: Current program area descriptions (PAD) for standards development in a number of areas including biological countermeasures, chemical countermeasures, high explosives, radiological/nuclear countermeasures, cyber security, emergency preparedness and response, borders and transportation security, personal protective equipment, critical infrastructure protection, and training, strategic planning templates, DHS Future years Homeland Security Performance Measures, Ongoing work within ANSI HSSP and ASTM Homeland Security Applications Committee

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: The standards portfolio has defined targets and timeframes for long term measures to include establishing technical standards and test/evaluation protocols for all current and future CBRNE detection and decontamination technologies and analysis tools. In addition, the standards program will adopt or develop standards to meet the current and future needs of operational directorates within DHS including biometric standards, critical infrastructure protection standards, training standards, interoperability standards, and personal protective equipment standards

Evidence: Current program area descriptions (PAD) for standards development in a number of areas including biological countermeasures, chemical countermeasures, high explosives, radiological/nuclear countermeasures, cyber security, emergency preparedness and response, borders and transportation security, personal protective equipment, critical infrastructure protection, and training, strategic planning templates, DHS Future years Homeland Security Performance Measures, Ongoing work within ANSI HSSP and ASTM Homeland Security Applications Committee

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: The standards portfolio has specific annual performance measures outlined in the PADs and Execution Plans in a number of homeland security areas. Technical standards and test/evaluation protocols will be established for WMD decontamination technologies and analysis tools. For FY05, "Consumer's report" on radiation and bioagent detection devices for federal, state, and local users will be published. For FY2006, establish and accredit a network of private/public labs to perform testing, evaluation, and certification of WMD emergency response technologies to allow effective procurement and deployment of technologies that will substantially reduce risk and enhance resiliancy of the federal, state, and local response capability.

Evidence: Unconstrained and constrained Science and Technology strategic planning templates, PADs, DHS FYHSP

Does the program have baselines and ambitious targets for its annual measures?

Explanation: The Standards program is in the process of developing annual performance measures as part of the current Science and Technology Strategic Planning Activities. In addition, the standards portfolio has outlined through its execution plans ambitious targets (such as developing and adopting standards in eleven critical homeland security areas.

Evidence: Science and Technology strategic planning templates. Exeuction Plans, PADs, Future Year HSP performance measures

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: The Standards program has an intra-agency integrated product team (IPT) which develops long term goals for the program. In addition, the standards program has established a number of interagency working groups and advisory groups that provide input into long and short term planning. Grantees, contractors, and cost-sharing partners are aware of and agree to the standards long term goals as specified in the execution plan, statement of work, quarterly reports, etc.

Evidence: S&T Stragetic Planning Templates, Execution Plans, Statement of Work documents, Management Directives, Quarterly reports. Examples of partners committed to and working towards our annual and long term goals include the Task Force for Bacillus Anthracis (representatives from EPA, FDA, DoD, OSTP, HHS, DHS, etc.) which is supporting the method validation of hand-held immunoassays for the detection of anthrax. Other examples include the Consequence management subcommittee, chartered by the Subcommittee on Standards, to develop standardized approach to consequence management after a Radiological Dispersal Device or Improvised Nuclear Device incident.

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: As part of the standards vetting process, a Homeland Security Standards Advisory Council looks at the scope and quality of the standards portfolio and advises whether the program is meeting the evolving mission requirements of the S&T Directorate and the department. In addition, the program will undergo an annual independent program review as part of the S&T directorate's STAR Program Review Process

Evidence: Homeland Security Standards Advisory Council as defined in the management directives, STAR Program Review Process

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: The Standards budget is developed through a strategic planning process that requires program mission to be linked to specific program objectives which are tied to program goals and objectives. In addition, the Future Years Homeland Security Performance Measures are tied to budget requests

Evidence: Standard's Strategic Planning Templates, DHS Planning, Programming & Budgeting System process

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: The Standards program is currently participating in the Science and Technology strategic planning activities.

Evidence:  

If applicable, does the program assess and compare the potential benefits of efforts within the program to other efforts that have similar goals?

Explanation: The standards portfolio has an established process for coordinating with similar standards programs in both the government and private sector as defined in the management directive. In a similar manner the program is continually comparing and contrasting other similar efforts and expanding its impact by leveraging on resources of other US government entitities including the National Instititute of Standards and Technology (NIST), Department of Defense (DoD), Food and Drug Administration (FDA), Environmental Protection Agency (EPA), and the Department of Energy (DOE). In addition the standards portfolio leverages on the resources of the private sector by using the voluntary consensus standards process mandated by OMB. The program is designed to coordinate activities at all levels (federal, state, local) and to leverage on existing available resources.

Evidence: All federal agencies have a standards executive and the standards portfolio director has been designated as the standards executive for DHS. The standards executive works closely with the standards executives from all other federal agencies to coordinate, compare, and leverage existing efforts. In addition the standards portfolio has set up working groups with key players from other similar programs contributing to input. Examples of other similar programs include the EPA's Environmental Technology Verification Program, Department of Justice, Office of Law Enforcement Standards. Evidence includes management directives, OMB Circular 119.

Does the program use a prioritization process to guide budget requests and funding decisions?

Explanation: As part of the standards vetting process, working groups of subject matter experts that include broad agency representation develop prioritized lists of standards requirements with appropriate budget estimates. A standards steering committee that includes representation from the portfolio managers in S&T combines the lists and determines priorities for funding.

Evidence: Management Directives, Strategic Planning Templates, Program Decision Memorandum, DHS Plannning, Programming & Budgeting System process

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: Intramural and extramural procurement requests require collection of performance data , additionaly, DHS collects performance information in their Future Year Homeland Security Program database. The S&T program includes and provides performance information on the Standards program in every update.

Evidence: Intramural and extramural procurement requests, quarterly reports, monthly reports to the Office of Research and Development DHS Future Years Homeland Security Program database (FYHSP).

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: All portfolio managers are required to be accountable for cost, schedule, and performance results.

Evidence: Quarterly reports, STAR Program Review Process

Are funds (Federal and partners') obligated in a timely manner and spent for the intended purpose?

Explanation: The funds for the standards portfolio are directed to National and Federal laboraboraty partners and to private sector standards development organizations. These funds have enabled fast track development of standards for radiation and biological agent detectors as well as personal protective equipment standards.

Evidence: Obligation rates, quarterly reports, Adopted Standards

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation:  

Evidence:  

Does the program collaborate and coordinate effectively with related programs?

Explanation: The department is using the American Voluntary Consensus Standards (VCS) process mandated by OMB to develop standards in concert with other State and Federal agencies and the private sector. This move to leverage the resources of the private sector is more efficient and cost effective than developing standards using only Federal employees. In addition, the standards portfolio has a clear process for collaborating and coordinating with other related programs in both the federal and private sectors as stated in the management directives

Evidence: Management Directives, American National Standards Institute Homeland Security Standards Panel, Task Force on Baccillus Anthracis, SoS Subcommittee on CMS - for RDD/INDD,

Does the program use strong financial management practices?

Explanation: The Science and Technology Directorate is a new directorate within DHS. Processes for developing financial management activities are currently being developed.

Evidence:  

Has the program taken meaningful steps to address its management deficiencies?

Explanation: The Science and Technology Directorate is a new directorate within DHS. Processes for developing financial management activities are currently being developed.

Evidence:  

For R&D programs other than competitive grants programs, does the program allocate funds and use management processes that maintain program quality?

Explanation: Implementing project management process based upon the Project Management Book of Knowledge (PMBOK).

Evidence: Guidelines to Develop FY04 S&T ORD Execution Plans dated 10 Oct 03

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: Standards and test programs have been developed and deployed to the public.

Evidence: 8 Respiratory Protection Standards have been adopted (5 NIOSH, 3 NFPA), 4 Radiation Detection Device Standards Developed and Adopted, 4 Radiation Detection Test and Evaluation Protocols approved; Equipment meeting the standards are currently being procured and put into use. Developing and revising protective clothing standards to address CBRN threats.

Does the program (including program partners) achieve its annual performance goals?

Explanation: Annual Performance Goals for standards are defined in the PADS , the strategic planning templates, and in the future years homeland security performance measures. They include establishing the DHS standards prioritization, adoption and development process, and adopting and developing key standards in eleven subject areas including CBRNE countermeasures and operational directorates needs.

Evidence: FYHSP, Copies of management directives, execution plans, quarterly reports, copies of adopted standards and standards under development

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: The Science and Technology Directorate is a new directorate within DHS.

Evidence:  

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: The Science and Technology Directorate is a new directorate within DHS.

Evidence:  

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: An independent evaluations of the standards program has not been accomplished to date. The Homeland Security Standards Advisory Council will report on the FY04 program. In addition, the STAR Review Process will take place in August 2004

Evidence: