ExpectMore.gov


Detailed Information on the
Arson and Explosives Program Assessment

Program Code 10002202
Program Title Arson and Explosives Program
Department Name Department of Justice
Agency/Bureau Name Bureau of Alcohol, Tobacco, Firearms, and Explosives
Program Type(s) Direct Federal Program
Assessment Year 2008
Assessment Rating Adequate
Assessment Section Scores
Section Score
Program Purpose & Design 80%
Strategic Planning 75%
Program Management 43%
Program Results/Accountability 40%
Program Funding Level
(in millions)
FY2007 $262
FY2008 $274
FY2009 $263

Ongoing Program Improvement Plans

Year Began Improvement Plan Status Comments
2008

ATF will work with the FBI to implement the provisions of the July 2008 Memorandum of Understanding between the two agencies.

Action taken, but not completed
2008

ATF will ensure that all financial management related material weaknesses identified are corrected in an expeditious and comprehensive manner.

Action taken, but not completed
2008

ATF will conduct independent program evaluations to determine whether the program is effective and achieving results.

No action taken
2008

ATF will establish a performance measure based on in-depth evaluation of the application of select training it provides.

No action taken
2008

Consolidate the Bomb Arson Tracking System and other internal arson and explosives information systems.

Action taken, but not completed

Completed Program Improvement Plans

Year Began Improvement Plan Status Comments

Program Performance Measures

Term Type  
Long-term/Annual Outcome

Measure: Percentage of explosives licensees/permittees who improve compliance with Federal explosives regulations upon re-inspection


Explanation:This measure reflects the long-term impact of ATF's inspection program. When ATF finds violations during an inspection, the violations are corrected before the inspection is closed. When the inspector determines that a recall inspection is advisable, another complete inspection is conducted at a later time, with the high-level goal being that the licensee/permittee has decreased the number of violations, and has thereby increased compliance with the Federal explosives regulations.

Year Target Actual
2008 Baseline
2009
2010
2011
2012
2013
Long-term/Annual Outcome

Measure: Percentage of arson/explosives cases with defendants referred for prosecution.


Explanation:The majority of ATF's resources in arson and explosives are expended to prevent explosives from getting into the hands of those who would use them for criminal purposes, and to investigate and refer for prosecution those who have used explosives for such purposes. This measure reflects the outcome of ATF's investigative efforts in arson and explosives criminal cases, from the initial on-scene assessment and determination that a crime has been committed, the application of technical and forensic resources in the course of the investigation, the identification of suspects, the development/gathering of evidence sufficient to support criminal charges, and finally, the presentation of a criminal case suitable for prosecution to the prosecuting attorney.

Year Target Actual
2005 N/A 20
2006 N/A 21
2007 N/A 37
2008 23
2009 24
2010 25
2011 26
2012 27
2013 28
Annual Outcome

Measure: Percentage of explosives licensees/permittes inspected in compliance with the Safe Explosives Act.


Explanation:This measure provides incremental targets for ATF's goal to inspect 100 percent of explosives licensees/permittees over a 3-year period in accordance with the Safe Explosives Act. The Safe Explosives Act requires that ATF conduct on-site inspections of all explosives licensees/permittees at least once every 3 years. These inspections are conducted upon license application or renewal. The first year that the SEA was implemented a large number of new licensees/permittees entered the system. This large number of licensees/permittees must then be inspected every third year when their licenses/permits become eligible for renewal. This has resulted in an on-going inspection schedule that requires ATF to inspect 45 to 55 percent of an ever-fluctuating population of 11,000 to 12,000 licensees/permittees one year out of every three. The remaining percentage of the population is inspected over the other two years of the 3-year cycle.

Year Target Actual
2007 N/A 28
2008 53
2009 100
2010 28
2011 53
2012 100
2013 28
Annual Efficiency

Measure: Number of forensic arson and explosives cases completed per chemist FTE.


Explanation:The timely forensic analysis of evidence is a critical part of any arson or explosives investigation. ATF's laboratories strive to create efficiencies that will result in the more efficient processing of such evidence. This measure reflects targets for chemist FTEs to increase the number of cases processed each year, without sacrificing quality.

Year Target Actual
2008 Baseline
2009
2010
2011
2012
2013
Annual Output

Measure: Percentage of forensic explosives cases closed within 30 days


Explanation:

Year Target Actual
2005 35 35
2006 35 35
2007 35 32
2008 36
2009 37
2010 39
2011 41
2012 43
Annual Output

Measure: Percentage of forensic arson cases completed within 30 days.


Explanation:

Year Target Actual
2005 45 45
2006 50 50
2007 55 62
2008 57
2009 58
2010 60
2011 62
2012 64
2013

Questions/Answers (Detailed Assessment)

Section 1 - Program Purpose & Design
Number Question Answer Score
1.1

Is the program purpose clear?

Explanation: ATF's strategic plan states that the purpose of ATF's arson and explosives programs is to "resolve and prevent explosives and fire related crimes" which is also true for a myriad of other law enforcement entities. ATF's purpose overlaps and in many instances, duplicates the purposes of other Federal agencies and state and local law enforcement agencies. ATF conducts criminal investigations, regulates the explosives industry, provides training, and assists state, local, tribal and other Federal agencies in fire and explosives-related matters.

Evidence: ATF's arson and explosives programs have their foundation in the Federal explosives laws, including the National Firearms Act of 1934, the organized Crime Control Act of 1970, the Arms Export Control Act of 1976, the Anti-Arson Act of 1982, the Antiterrorism and Effective Death Penalty Act of 1996, the U.S. Patriot Act of 2001, and the Safe Explosives Act of 2002.

YES 20%
1.2

Does the program address a specific and existing problem, interest, or need?

Explanation: Crimes involving explosives or fire continue to occur on a regular basis throughout the United States. State and local fire and police departments routinely handle such incidents. The statutory authorities delegated to ATF confer responsibility for regulating explosives and firearms.

Evidence: According to the Uniform Crime Report, arson crimes increased by 2.1 percent in 2006 (the most recent statistics available), with a rate of 26.8 offenses for every 100,000 persons. According to the U.S. Bomb Data Center, in 2006 there were 3,797 explosives incidents in this country, which included 476 bombings, as well as explosives thefts, incendiary bombings, attempted bombings, and other incidents.

YES 20%
1.3

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: Within the Department of Justice, ATF and the FBI provide explosive training, forensic analysis, forensic auditing, and intelligence services. In 2004, the Attorney General issued a policy memorandum to help clarify the roles of ATF and the FBI with respect to other potentially redundant areas of interest. The memo (1) provided ATF jurisdiction over criminal explosive incidents and the FBI jurisdiction over terrorist related explosives incidents, (2) ordered that the FBI and ATF bomb data centers consolidate under the ATF; and, (3) designated ATF as the agency to train and certify all explosives detection canines used by Department agencies. However, many of the recommendations included in the AG's memo have not been implemented. ATF and FBI continue to compete for jurisdiction on investigating explosive incidents. Both ATF and FBI have separate explosives training facilities and bomb data centers (ATF Bomb Arson Tracking System and the FBI Bomb Data Center). A 2004 OIG report found confusion and duplication in reporting of explosives incidents by state and local law enforcement agencies.

Evidence: 2004 Attorney General policy memorandum, OIG report on ATF and FBI explosives intelligence databases released October 2004, http://www.usdoj.gov/oig/reports/ATF/a0501/final.pdf, FBI Lab Explosives Unit http://www.fbi.gov/hq/lab/html/eu1.htm, ATF Forensic Laboratories http://www.atf.gov/labs/services.htm, FBI Hazardous Device School, FBI Terrorist Explosives Device Analytical Center, ATF National Center for Explosives Research, FBI Bomb Data Center, ATF US Bomb Data Center, Department of Homeland Security Trip Wire, DHS Office of Bomb Prevention.

NO 0%
1.4

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: Both the ATF and the FBI respond to explosives incidents. While under a 2004 Attorney General memorandum the FBI has responsibility for terrorist incidents and the ATF for other incidents involving explosives, the FBI continues to lead the investigation of many explosives crimes. Additionally, both the ATF and the FBI provide explosives training to State and local police at the National Center for Explosives Training and Research and the Hazardous Device School. These ongoing challenges have been identified by the Inspector General as well as in Homeland Security Presidential Directive HSPD-19. The Department of Justice should take steps to clarify the roles and responsibilities of both the FBI and the ATF.

Evidence: Homeland Security Presidential Directive HSPD-19 ("Combating Terrorist Use of Explosives in the United States") /homeland/hspd19/ OIG report on ATF and FBI explosives intelligence databases released October 2004, http://www.usdoj.gov/oig/reports/ATF/a0501/final.pdf, ATF Bomb Arson Tracking System (BATS), ATF Arson and Explosives Incident System (AEXIS), ATF DFuze, and ATF US Bomb Data Center.

YES 20%
1.5

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: ATF's explosives and arson programs are targeted to reach intended beneficiaries and, in most cases, are executed through direct contact. For example, State and local agencies requiring the assistance of ATF's National Response Team (NRT) call ATF directly. NRT members work side-by-side with local responders. ATF's regulation of the explosives industry is directly targeted at explosives licensees/permittees. ATF's training programs are flexible and are designed to target the needs of specific groups (federal partners, state and local law enforcement, fire investigators, etc.) Through the U.S. Bomb Data Center (USBDC), registered users of ATF's Bomb and Arson Tracking System (BATS) are able to directly contribute to and access intelligence information to support arson and explosives investigations.

Evidence: Requests for NRT assistance are weighed on a case-by-case basis and must meet a high threshold of need (e.g., complexity, community impact, loss of lives). Other arson and explosives needs may be met by regional task forces or may be approached on a national level (ex. Church Arson Task Force). ATF conducts inspections of licensees/permittees based on the requirements of the Federal explosives laws and regulations. ATF's training programs address specific needs, such as the arson and explosives detection canine training and certification. The U.S. Bomb Data Center serves as the national repository for arson and explosives information and has amassed approximately 180,000 searchable records on incidents in the United States. Today over 2,680 registered users can access explosive and arson investigation information through the Bomb and Arson Tracking System (BATS).

YES 20%
Section 1 - Program Purpose & Design Score 80%
Section 2 - Strategic Planning
Number Question Answer Score
2.1

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: ATF's long term measures for arson and explosives are (1) percentage of explosives licensees/permittees who improve compliance with Federal explosives regulations upon re-inspection, and (2) the percentage of arson and explosives cases with defendants referred for prosecution. These measures reflect ATF's long-term commitment to preventing and resolving fire- and explosives-related incidents.

Evidence: The performance measure for the percentage of explosives licensees/permittees who improve compliance with Federal explosives regulation upon re-inspection, reflects the long-term impact of ATF's inspection program. When ATF finds violations during an inspection, the violations are corrected before the inspection is closed. When the inspector determines that a recall inspection is advisable, another complete inspection is conducted at a later time, with the high-level goal being that the licensee/permittee has decreased the number of violations, and has thereby increased compliance with the Federal explosives regulations. The majority of ATF's arson and explosives resources are expended to prevent explosives from getting into the hands of those who would use them for criminal purposes, and to investigate and refer for prosecution those who have used explosives for such purposes. The percentage of arson and explosives cases with defendants referred for prosecution reflects the outcome of ATF's investigative efforts in arson and explosives criminal cases, from the initial on-scene assessment and determination that a crime has been committed, the application of technical and forensic resources in the course of the investigation, the identification of suspects, the development/gathering of evidence sufficient to support criminal charges, and finally, the presentation of a criminal case suitable for prosecution to the prosecuting attorney.

YES 12%
2.2

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: ATF's target for the percentage of arson/explosives investigations with defendants referred for prosecution is set at 23 percent for FY-2008 and increases to 28 percent by FY-2013. Although this does not appear to be ambitious, the target for FY-08 of 23 percent represents a rise in the percentage of referrals over the rates of two of the three years of baseline data gathered for this measure, and this annual increase is targeted with no projected increase in investigative resources. (The large increase in the referral rate from 21 percent in FY-06 to a rate of 37 percent for 2007 is an anomaly.) The percentage of explosives licensees/permittees who improve compliance with Federal explosives regulations upon re-inspection is a new measure and has no established targets.

Evidence: According to the Uniform Crime Report for 2006, the National averaged arrest rate for fire investigations is 16 percent. (No data is available for explosives referrals.)

YES 12%
2.3

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: ATF has four annual measures that addresses its long term goals. These are (1) percentage of explosives licensees/permittees inspected in compliance with the Safe Explosives Act, (2) the number of forensic arson and explosives cases completed per chemist FTE, (3) percentage of forensic explosives cases closed within 30 days, and (4) percentage of forensic arson cases closed within 30 days.

Evidence: The percentage of explosives licensees/permittees inspected in compliance with the Safe Explosives Act is an annual measure that provides incremental targets to inspect 100 percent of explosives licensees/permittees over a 3-year period in accordance with the Safe Explosives Act. Annual inspections of explosives licensees/permittees enable ATF to find and correct violations and increase compliance with the Federal explosives regulations. The evaluation of forensic evidence is a critical aspect of identifying those cases that would be deemed suitable for referral for prosecution. The timely forensic analysis of evidence is a critical part of any arson or explosives investigation. Creating efficiencies in ATF's laboratories should result in the more efficient processing of such evidence. This measure reflects targets for chemist FTEs to increase the number of cases processed each year, without sacrificing quality.

YES 12%
2.4

Does the program have baselines and ambitious targets for its annual measures?

Explanation: The percentage of explosives licensees/permittes inspected in compliance with the Safe Explosives Act, provides targets for ATF's goal to inspect 100 percent of explosives licensees/permittees over a 3-year period in accordance with the Safe Explosives Act. The target rates for the completion of forensic fire and explosives cases within 30 days provide incremental increases. The number of forensic arson and explosives cases completed per chemist FTE is a new measure and therefore has not established baselines and targets.

Evidence: The Safe Explosives Act requires that ATF conduct on-site inspections of all explosives licensees/permittees at least once every 3 years. These inspections are conducted upon license application or renewal. The first year that the SEA was implemented a large number of new licensees/permittees entered the system. This large number of licensees/permittees must then be inspected every third year when their licenses/permits become eligible for renewal. This has resulted in an on-going inspection schedule that requires ATF to inspect 45 to 55 percent of an ever-fluctuating population of 11,000 to 12,000 licensees/permittees one year out of every three. The remaining percentage of the population is inspected over the other two years of the 3-year cycle.

YES 12%
2.5

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: Federal, state, and local regulatory agencies and trade associations work with ATF to ensure compliance with explosives laws and regulations, however program partners do not measure or report on their performance as it relates to the achievement of ATF's stated annual and long term goals. ATF conducts its fire and explosives investigations in conjunction with other Federal, state and local agencies and these program partners are not required nor do they commit to ATF's goals. However, most of ATF's partner agencies commit resources toward joint operations and investigations, including personnel and equipment.

Evidence: ATF has entered into Memoranda of Understanding (MOUs) with many agencies, though not all program partners measure and report on their performance. The training of accelerant detection canines represents an example of a reciprocal arrangement of ATF training canines for other agencies that, in consideration of the training they have received, agree to make the canine teams available to support ATF during NRT activations, at National Security Special Events, and at other significant investigations involving violations of the Federal arson and explosives laws. Similarly, State and local agencies may commit resources to ATF in support of law enforcement operations targeting a specific problem in a particular area. These operations may involve long-term arson task forces to address ongoing crime, or task forces dedicated to identifying and apprehending a serial arsonist or bomber.

NO 0%
2.6

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: ATF relies on several discrete evaluation methods that do not provide a comprehensive evaluation of its program. These include OIG/GAO audits of specific program areas, periodic operational reviews by ATF's Office of Professional Responsibility and Security Operations (OPRSO), and surveys of the recipients of ATF's arson and explosives services. Note: the PART guidance requires that evaluations meet all the criteria of the proceeding to receive a "yes". OIG/GAO audits are not routinely scheduled and lack sufficient scope because they only evaluate certain aspects of a program. In 2004, ATF contracted with Sirota Consulting, an independent firm, to devise, issue, collect, and analyze surveys of the recipients of ATF's arson and explosives services. The surveys were posed as "Customer Satisfaction" instruments, which is not an evaluation on the effectiveness of the program.

Evidence: GAO report on thefts of explosives from state and local government storage facilities released October 2005 http://www.gao.gov/new.items/d06182t.pdf, OIG report on ATF's Arson and Explosives Databases released October 2004 http://www.usdoj.gov/oig/reports/ATF/a0501/final.pdf and ATF's implementation of the Safe Explosives Act released March 2005, http://www.usdoj.gov/oig/reports/ATF/e0505/final.pdf. Customer satisfaction survey conducted by Sirota Consulting on behalf of ATF.

NO 0%
2.7

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: The ATF 2009 Congressional budget request for arson and explosives is directly tied to its ability to accomplish its arson and explosives mission, as defined in its strategic plan. Requests for funding increases are linked to accomplishment of annual and long term performance goals. Additionally the budget submission includes a narrative section unit entitled "Performance Plan and Report for Outcomes" which provides additional discussion of performance reported in the performance tables section. Direct and indirect costs allocated to the different initiatives are not presented in a complete and transparent manner within the arson and explosives program budget.

Evidence: ATF's 2009 Congressional budget submission includes explanations of ATF's programs, initiatives, performance goals, and funding request. http://www.usdoj.gov/jmd/2009justification/pdf/fy09-atf.pdf

YES 12%
2.8

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: ATF's current strategic plan (FY 05 through 09) was approved in 2004. ATF is currently in the process of updating its strategic plan. ATF has developed strategic priority action committees (SPACs), each focused on an ATF strategic goal. PART guidance states that the program should be working to adopt a limited number of specific, ambitious long term and annual performance goals. ATF's arson and explosives priorities include five strategic goals, however performance measures have been developed for only two strategic goals. In addition the majority of the performance measures identified are annual output measures.

Evidence: ATF's draft strategic plan and documents relating to the SPACs are provided as evidence.

YES 12%
Section 2 - Strategic Planning Score 75%
Section 3 - Program Management
Number Question Answer Score
3.1

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: At this time, collected performance data is not subjected to a systematic review process with quality controls to confirm the validity of the data being reported by external program partners or beneficiaries. (Note: PART guidance states that credible performance information is information that is collected through a systematic process with quality controls to confirm the validity of the data.) ATF is planning to implement a recently purchased software survey program that will allow the programs and training offices to solicit information to assess its many and varied programs. Additionally, ATF's Office of Inspection conducts operational reviews of each office once every 4 years, during which information about the quality of ATF's services is collected from external customers. In 2004, ATF contracted with Sirota Consulting, an independent firm, to devise, issue, collect, and analyze surveys of recipients of ATF's arson and explosives services. ATF has used the surveys to implement program improvements, however the surveys are not regularly scheduled to collect timely performance information. (Note: PART guidance states that timely performance information is information that reflects current peformance and is current enough to be useful in program mangagement.)

Evidence: The survey results provided by Sirota Consulting in 2004 identified several areas of customer dissatisfaction in the explosives licensing/permitting process. In response, ATF has temporarily created an insert for the application that clarifies instructions. The application has been revised and is going through the approval process. The 2004 Sirota Consulting report also recommended that ATF develop a program to increase the visibility of the U.S. Bomb Data Center among its customers. Since then, ATF has trained thousands of potential users on the Bomb Arson Tracking System (BATS) and its capabilities. The results of interviews conducted by ATF's Office of Inspection are used to help ATF determine if it is meeting the needs of its external partners.

NO 0%
3.2

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: At this time there is no uniform method for holding program partners accountable based on performance data and contribution to program's goals. ( Note: PART guidance states that the program establish specific performance standards and accountability for program partners when they contribute to the achievement of program goals.) ATF's senior executives are, by contract, accountable for the performance of those programs under their purview. Senior Executive Service Performance Work Plans (PWPs) are linked to the Department of Justice's and ATF's strategic plans, as well as other guiding instruments such as the President's Management Agenda, to ensure that individual performance is tied to agency performance.

Evidence: Performance Work Plans and Enterprise Architecture Board documents are submitted as evidence.

NO 0%
3.3

Are funds (Federal and partners') obligated in a timely manner, spent for the intended purpose and accurately reported?

Explanation: Funds are allocated to program divisions and are obligated for their intended purpose in a timely manner. Supervisory review procedures are in place to ensure that funds are expended appropriately. Obligations, commitments, and receivers are reviewed quarterly for accuracy and completeness. Financial transactions are managed through a monthly accountability process. ATF continues to carry a repeat material weakness for recording accounts payable as found by independent auditors.

Evidence: ATF's financial management system allows for tracking of spending by project code, and regular reviews of expenditures are conducted. Funds are obligated on an annual basis except for no-year carry over accounts. From a total of all sources of $1.041B, $10.2M was non-obligated; of that non-obligated balance, $8.5M was in no-year carry over accounts to FY 07. Therefore, the amount actually non-obligated was $1.7M. ATF accomplished an execution rate of 100 percent in FY 07. OIG commentary and summary on ATF Financial statements for FY 2007, http://www.usdoj.gov/oig/reports/ATF/a0812.htm KPMG Independent Auditors Report on Internal Control http://0225.0145.01.040/ag/annualreports/pr2007/sect3/p9-26.pdf (p.11)

YES 14%
3.4

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: The ATF U.S. Bomb Data Center consists of multiple, custom databases that contain arson and explosives information that are maintained separately. Efficiencies can be gained by consolidating systems resulting in cost savings (easing State and local law enforcement agencies access to ATF bomb information and by having only one system to maintain and support, eliminating manual entry). While ATF understands these efficiencies and is currently taking steps towards consolidation, the Bomb Arson Tracking System and other critical systems have yet been combined. ATF has examined and implemented the feasibility of achieving efficiencies through contracted services. ATF has also identified a new annual efficiency measure for the number of forensic arson and explosives cases completed per chemist FTE.

Evidence: ATF has determined that combining the databases and converting from custom to Commercial-off-the-Shelf (COTS) software will result in significant efficiencies. The project was approved by ATF's Enterprise Architecture Board, put out for bid, and a contract was awarded. ATF has weekly meetings with the contractor to ensure the contract remains on schedule. ATF has implemented contracted services at the Federal Explosives Licensing Center and at the Canine Training Center.

YES 14%
3.5

Does the program collaborate and coordinate effectively with related programs?

Explanation: ATF regularly collaborates with other law enforcement and regulatory agencies to accomplish its mission; yet effective coordination and deconfliction has proven more difficult to achieve. Although ATF participates in cooperative agreements with other agencies, task forces, professional organizations, and has extensive interaction within the federal law enforcement community; agreements, meetings, discussions groups, and task forces do not - by themselves, yield effective coordination and deconfliction of mission and purpose.

Evidence: To help better define federal roles and responsibilities in the area of explosives, ATF has been working with the FBI, the Department of Homeland Security, the Department of Defense, and the Homeland Security Council regarding the implementation of Homeland Security Presidential Directive (HSPD) 19, entitled, "Combating Terrorist Use of Explosives in the United States." ATF also partners with the FBI in the Terrorist Explosive Device Analysis Center and at the FBI's National Center for the Analysis of Violent Crime. ATF has participated in intergovernmental working and policy groups regarding the potential regulation of ammonium nitrate although the effort has not yet produced an agreed upon delineation of responsibility and authority. ATF is also supporting the Department of Defense in Iraq. Through an agreement with the Department of State, ATF deploys its International Response Team (an outgrowth of the arson and explosives National Response Team) in support of Diplomatic Security Service investigative responsibilities and foreign government requests. ATF also works with the Consumer Products Safety Commission to coordinate enforcement efforts related to pyrotechnics. At this time, however, ATF has not produced or disseminated joint announcements, planning documents, performance goals, or information systems with its federal or state and local law enforcement partners.

NO 0%
3.6

Does the program use strong financial management practices?

Explanation: ATF FY 2007 financial statements received an unqualified opinion with one material weakness for the recording of accounts payable and one significant deficiency for information technology. Due to the continued existence of errors and inaccuracies with the accrual process used to record accounts payable ATF has been carrying the material weakness on the recording of accounts payable since 2005. (Note: PART guidance states that adequate procedures should exist for reporting actual expenditures, comparing them against their intended use and taking timely and appropriate action to correct single audit findings.) The accounts payable material weakness finding relates to ATF's internal controls over the recording of the receipt of goods and services. KPMG Independent auditors noted in their report on internal control that " despite modifications made by ATF to its existing policies and procedures and training provided to purchasing agents involved in the process, component auditors identified errors in the receiver process controls as well as errors in the recording of transactions related to undelivered orders...Such errors impacted the amounts reported for net cost of operations and obligations incurred." KPMG auditor's also reported that "ATF's financial management systems did not substantially comply with applicable federal financial management systems requirements and federal accounting standards." Department of Justice Office of Inspector General did not issue an opinion on ATF's financial statements but acknowledge that KPMG complied with all the auditing standards.

Evidence: The FY 2007 Independent Auditors' Report on Compliance and Other Matters. http://0225.0145.01.040/ag/annualreports/pr2007/sect3/p27-28.pdf

NO 0%
3.7

Has the program taken meaningful steps to address its management deficiencies?

Explanation: ATF conducts comprehensive internal reviews, and has implemented OIG recommendations. As required by OMB Circular A-123, ATF's Office of Inspection reviews all directorates (divisions, districts, administrative components, and other offices) over a 4-year cycle. The review covers all aspects of management, including funds management, procedures, personnel management, and asset utilization. Deficiencies noted during the review are corrected by management and addressed in a follow-up, corrective action memorandum. ATF also utilizes a log to track and follow up on all external audit findings and recommendations. At this time, ATF has not addressed/corrected errors and weakness identified in the KPMG Independent Auditors' Report on Compliance and Other Matters ( see explanation in question 3.6.)

Evidence: In addition to the OMB Circular A-123 assessment, ATF also requires all of its directorates to do their own self assessments through a specific management control tool; the results of these self assessments are evaluated as part of the certification required under A-123. The Office of Enforcement Programs and Services (EPS) does a thorough job of reviewing its internal controls on an annual basis in accordance with OMB, DOJ and ATF guidelines. For the FY 2007 Mid-year Internal Control Assessment Review, EPS was in compliance in most areas. Where necessary, EPS executed corrective action plans to address deficiencies. Following the OIG audit recommendations ATF implemented process changes including contract reviews by Acquisitions and the Office of Chief Counsel for contracts with vendors who will be providing ATF services for events. ATF has also taken steps to improve its internal controls pertaining to travel and other administrative areas.

YES 14%
Section 3 - Program Management Score 43%
Section 4 - Program Results/Accountability
Number Question Answer Score
4.1

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: The long term measure for the percentage of licensees/permitees who imporve compliance with Federal explosives regulations upon re-inspection has recently been developed and lacks baseline and performance data to indicate past performance. The second long term measure, the percentage of arson/explosives cases with defendants referred, identifies an incremental increase of 1 percent for 2006 and an increase of 16 percent for 2007 (reported this as an anomaly.) Therefore historical data necessary to determine whether the program is making progress is severely limited.

Evidence: The percentage of defendants referred for 2007, at 37 percent, is an anomaly. The data query is based on cases that were closed during the fiscal year, as these are fully adjudicated cases. In 2007, the field divisions were instructed to examine their open cases in the data base and close all those that were fully adjudicated or no longer viable for various reasons. This resulted in a large number of cases that had been completed in previous years being closed, and therefore, an indication of a higher percentage of investigations referred for prosecution.

SMALL EXTENT 7%
4.2

Does the program (including program partners) achieve its annual performance goals?

Explanation: The number of forensic arson and explosives cases completed per chemist FTE is a new measure and has no established baseline. The percentage of explosives licensees/permittees inspected in compliance with the Safe Explosives Act has also recently been developed and therefore lacks performance data to indicate baseline and performance. In 2007, ATF failed to meet the goal for forensic explosives cases closed within 30 days due to laboratory renovations. However, ATF has achieved its performance goal for forensic arson cases closed within 30 days.

Evidence: Performance measure table. FY-07, the Atlanta laboratory was undergoing extensive renovations that caused an interruption of work. During that year, the Atlanta laboratory showed a 30% decrease in both arson and explosives cases completed. To address this, ATF transferred "expedite" arson cases to the Washington laboratory for examination, and a forensic chemist was detailed from Atlanta to Washington for one month to complete Atlanta arson cases. As a result, ATF was able to achieve its performance target in FY07 for completing arson cases. However, in addition to the impact of the Atlanta laboratory renovation, ATF experienced the loss of 5 explosives chemists across three laboratories. This loss, combined with the renovations, resulted in an inability to meet the 35 percent target in FY07

SMALL EXTENT 7%
4.3

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: ATF is in the process of implementing several efficiencies that should generate savings: the ongoing consolidation of databases under the U.S. Bomb Data Center; and, the decision to require law enforcement agencies to pay their own expenses for canine training. ATF has adopted process improvements evidenced through the issuance of explosives industry rulings; and through the streamlining of processes at the Federal Explosives Licensing Center (FELC).

Evidence: The consolidation of information systems within the USBDC should when implemented, provide an efficiency "ripple effect." The consolidation of multiple information systems into a single-source information system should result in cost savings by no longer having database administrators and support personnel for multiple systems; additionally, users no longer have to search multiple information systems to acquire information. Over the past two fiscal years, the demand for canine training by State and local agencies increased four-fold overburdening ATF's ability to fully fund the training. Consequently, beginning in FY-08, ATF is requiring that law enforcement agencies pay their own expenses for training.

LARGE EXTENT 13%
4.4

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: ATF's arson and explosives program may compare favorably with state and local law enforcement agencies with similar purpose and goals. However, a comprehensive evaluation of the ATF arson and explosives program with federal law enforcement agencies with similar purpose has not been done, only certain aspects of ATF's explosives program has been evaluated. ATF asserts that when the assistance of ATF's National Response Team is requested, it is because the requesting departments do not have the experience and resources needed to resolve the case. In terms of its regulatory mission, many states request ATF advice to implement stricter regulations for explosives.

Evidence: Although the FBI and DHS have resources dedicated to explosives issues, neither agency have undergone a PART in this area, however the 2004 OIG evaluation on the ATF and FBI arson and explosives intelligence databases reported that both ATF and FBI having similar responsibilities of compiling explosives data were not found to be effective and efficient.

LARGE EXTENT 13%
4.5

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: The evaluations ATF has undergone do not indicate that the overall program is effective and achieving results. Customer satisfaction surveys administered, analyzed, and furnished to ATF by Sirota Consulting, an independent firm, do not meet the PART guidance for evaluations of sufficient scope and quality to indicate program effectiveness. GAO and OIG reports on arson and explosives programs are consistent with PART guidance for sufficient quality but have limited scope, evaluating only certain aspects of ATF's arson and explosives programs.

Evidence: OIG report on ATF and FBI Investigations' Arson and Explosives Intelligence Databases found "the similar responsibilities of the ATF and the FBI in compiling data have resulted in duplication of effort, confusion and duplicate reporting of incidents by state and local agencies." OIG report on ATF Implementation of the Safe Explosives Act found that " ATF information systems are ineffective for managing the explosives licensing functions mandated by the Safe Explosives Act.

NO 0%
Section 4 - Program Results/Accountability Score 40%


Last updated: 09062008.2008SPR