ExpectMore.gov


Detailed Information on the
Counterdrug Technology Transfer Program Assessment

Program Code 10001153
Program Title Counterdrug Technology Transfer Program
Department Name Office of Natl Drug Control
Agency/Bureau Name Office of National Drug Control Policy
Program Type(s) Competitive Grant Program
Assessment Year 2003
Assessment Rating Results Not Demonstrated
Assessment Section Scores
Section Score
Program Purpose & Design 80%
Strategic Planning 38%
Program Management 60%
Program Results/Accountability 11%
Program Funding Level
(in millions)
FY2007 $16
FY2008 $10
FY2009 $0

Ongoing Program Improvement Plans

Year Began Improvement Plan Status Comments
2006

Working with the Congress to redirect funding to other higher priorities.

Not enacted

Completed Program Improvement Plans

Year Began Improvement Plan Status Comments
2004

Developing a performance measurement system, including acceptable goals and timeframes.

Completed
2004

Revising the selection process and procedures for determining technology assistance awards so that they are need and merit-based.

Completed
2004

Complete, by September 2004, an assessment of the CTAC TTP performance and management processes.

Completed
2004

Improve the quality of program data collected and the use of such data to manage the program.

Completed

Program Performance Measures

Term Type  
Long-term Outcome

Measure: Percent of recipient agencies that report improved efficiency relative to officer safety, investigative capability, and investigative effectiveness.


Explanation:This measure serves as a proxy for objective evidence of increases in the investigative capabilities of state and local law enforcement officers. TBR = To Be Reported.

Year Target Actual
2004 75% 80%
2005 75% 89%
2006 75% 82%
2007 75% TBR Dec 2008
2008 85%
Annual Efficiency

Measure: Percent of total TTP costs dedicated to administrative expenses.


Explanation:This measure serves as a proxy for the level of program bureaucracy, which CTAC endeavors to keep to a minimum. This measure will monitor the efficiency of program management so that the majority of program dollars are allocated to equipment and training rather than to extensive administrative overhead. TBR = To Be Reported.

Year Target Actual
2004 10% 6%
2005 10% 9%
2006 10% 12.5%
2007 10% TBR May 2008
2008 10%
Annual Outcome

Measure: Percentage of recipient agencies that report TTP equipment has provided a technological solution to an investigative requirement.


Explanation:Recipients are asked this directly via a follow-up survey. TBR = To Be Reported.

Year Target Actual
2004 95% 92%
2005 95% 97%
2006 95% 96%
2007 95% TBR Dec 2008
2008 95%
Annual Outcome

Measure: Percent of recipient agencies that report adequate training on TTP equipment (New measure, added February 2008)


Explanation:Measure assesses quality of training provided to expand the investigative capabilities of state and local law enforcement by deploying proven and appropriate technologies.

Year Target Actual
2004 95% 97%
2005 95% 96%
2006 95% 93%
2007 95% TBR Dec 2008
2008 95%
2009

Questions/Answers (Detailed Assessment)

Section 1 - Program Purpose & Design
Number Question Answer Score
1.1

Is the program purpose clear?

Explanation: The general purpose of ONDCP's CTAC Technology Transfer Program is to provide technologies directly to state and local law enforcement agencies (LEAs). However, the lack of authorizing language clearly describing the purpose of the program resulted in varied definitions of the program purpose. ONDCP has developed a mission statement for the Technology Transfer Program that establishes the purpose of the program as "transferring technologies to state and local law enforcement agencies that may otherwise be unable to benefit from the developments due to limited budgets or a lack of technological expertise to expand the investigative capabilities of state and local law enforcement.

Evidence: ONDCP FY 2004 Congressional Budget Submission/Performance Plan; CTAC Research and Development Blueprint Update, 2003; and various annual appropriations acts.

YES 20%
1.2

Does the program address a specific and existing problem, interest, or need?

Explanation: Salaries and general overhead constitute the largest share of State and local law enforcement agency (LEA) budgets and leave relatively little for the purchase of drug-crime fighting technologies. In addition, local political considerations often make it difficult for local law enforcement officials to purchase needed technology rather than hiring additional officers. CTAC funds the development, testing, and distribution of effective investigative technology to help supplement LEA budgets.

Evidence: Historically, surveys and censuses of local law enforcement agency budgets by the Bureau of Justice Statistics (BJS) have found that approximately 85 percent of the typical agency's budget is allocated to salaries and other general overhead expenses, leaving little funding available to procure technologies to expand investigative capabilities. According to BJS staff, recent surveys haven't been asking for that data because there was relatively little variation in the responses received.

YES 20%
1.3

Is the program designed so that it is not redundant or duplicative of any Federal, state, local or private effort?

Explanation: There are other Federal programs that either directly transfer technology to State and local law enforcement agencies or provide funds to purchase equipment, but those programs are sufficiently distinct from the TTP that there is no substantial overlap. For example, the $190M Law Enforcement Technology Program, part of the Justice Department's Community Oriented Policing Services (COPS) program, provides funding for law enforcement technology. However, these grants are typically used for administrative equipment rather than investigative equipment and are intended to move officers from paperwork to spending more time on the street. For example, an August 2000 NIJ study indicated that 79 percent of COPS technology grant recipient agencies used funds for the purchase of mobile computers. In addition, unlike many State and local assistance grant programs, the appropriation for the TTP has not been earmarked by the Congress for specific grantees.

Evidence: "National Evaluation of the COPS Program Title I of the 1994 Crime Act," National Institute of Justice. Discussions with ONDCP staff.

YES 20%
1.4

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: There is no clear evidence that another approach or mechanism would be a more efficient/effective mechanism to transfer investigative technology to state and local law enforcement agencies.

Evidence:  

YES 20%
1.5

Is the program effectively targeted, so program resources reach intended beneficiaries and/or otherwise address the program's purpose directly?

Explanation: The lack of authorizing language has caused ambiguity concerning purpose and intended beneficiaries of the program. As a result, ONDCP has operated the program essentially on a first-come, first-served basis. ONDCP has begun to developing procedures to target its resources more effectively, including devising a means to improve the ability to distinguish the relative merits of the requests received.

Evidence: ONDCP FY 2004 Congressional Budget Submission/Performance Plan and CTAC Research and Development Blueprint Update, 2003.

NO 0%
Section 1 - Program Purpose & Design Score 80%
Section 2 - Strategic Planning
Number Question Answer Score
2.1

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: ONDCP has established a new long-term performance measure for the technology transfer program - - the percentage of recipient agencies that report improvement relative to officer safety, investigative capability, and investigative effectiveness from use of CTAC sponsored equipment and training.

Evidence: ONDCP FY 2004 Congressional Budget Submission/Performance Plan; CTAC Research and Development Blueprint Update, 2003; and discussions with ONDCP staff.

YES 12%
2.2

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: This is a new target, and ONDCP has not established a baseline due to lack of relevant information. Baselines will be established following a review of data collected from TTP recipients during FYs 2003 and 2004.

Evidence: ONDCP FY 2004 Congressional Budget Submission/Performance Plan; CTAC Research and Development Blueprint Update, 2003; and discussions with ONDCP staff.

NO 0%
2.3

Does the program have a limited number of specific annual performance measures that demonstrate progress toward achieving the program's long-term measures?

Explanation: ONDCP has established new annual performance measures for the technology transfer program. These measures include: Maintain administrative expenses at less than 10 percent of total program funds expended; provide 95% of TTP recipients with equipment they report has provided a technological solution to an investigative requirement; and provide 95% of TTP recipients with training in use of the TTP equipment they report was adequate based on experience using the equipment in the field.

Evidence: ONDCP FY 2004 Congressional Budget Submission/Performance Plan; CTAC Research and Development Blueprint Update, 2003; and discussions with ONDCP staff.

YES 12%
2.4

Does the program have baselines and ambitious targets and timeframes for its annual measures?

Explanation: This is a new target, and ONDCP has not established a baseline due to lack of relevant information. Baselines will be established following a review of data collected from TTP recipients during FYs 2003 and 2004.

Evidence: ONDCP FY 2004 Congressional Budget Submission (includes Performance Plan); CTAC Research and Development Blueprint Update, 2003 (ONDCP) ; discussions with ONDCP staff.

NO 0%
2.5

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, etc.) commit to and work toward the annual and/or long-term goals of the program?

Explanation: ONDCP has just recently developed adequate long-term and annual measures for the TTP and there has not been sufficient time for CTAC partners to review and commit to the goals. Previously, ONDCP did not have adequate measures for the CTAC program. Consequently, the program must receive a "no" answer for this question.

Evidence: See above.

NO 0%
2.6

Are independent and quality evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: There has not been an independent evaluation of CTAC's TTP.

Evidence: Discussions with ONDCP staff.

NO 0%
2.7

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: ONDCP has not provided budget requests that make clear the impact of funding, policy, or legislative decisions on expected performance and explain why the requested performance/resource mix is appropriate. This is largely due to the absence of adequate program performance measures in past years.

Evidence: ONDCP FY 2004 Congressional Budget Submission (includes Performance Plan)

NO 0%
2.8

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: ONDCP staff have begun to define a limited number of specific, ambitious long-term performance goals and a limited number of annual performance goals that demonstrate progress toward achieving the long-term goals. ONDCP staff have also committed to improving the information that the TTP collects and using that information to review program performance.

Evidence: Discussion with ONDCP staff.

YES 12%
Section 2 - Strategic Planning Score 38%
Section 3 - Program Management
Number Question Answer Score
3.1

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: Until the Summer of 2003, ONDCP did not have meaningful, ambitious performance targets for the CTAC TTP and has relied on survey responses from TTP recipients as a gauge of the program's performance. The measures generally reflect only 'customer satisfaction,' are limited in both number and scope, and rely exclusively on unverified self-reported responses from TTP recipients. There is no indication these data have been used to improve program performance. ONDCP has agreed to improve the management measures and to collect and analyze them on a regular basis in the future.

Evidence: ONDCP FY 2004 Congressional Budget Submission (includes Performance Plan); CTAC Research and Development Blueprint Update, 2003 (ONDCP) ; discussions with ONDCP staff.

NO 0%
3.2

Are Federal managers and program partners (grantees, subgrantees, contractors, cost-sharing partners, etc.) held accountable for cost, schedule and performance results?

Explanation: ONDCP procedures governing the distribution of CTAC funds includes a broad description of services to be provided by the entities that serve as CTAC agents, including: special instructions placing restrictions on funds to be spent for travel and administrative support; details on reporting requirements; a termination clause; and a requirement that the agent adhere to DOD regulations for program and financial management. However, performance standards for ONDCP managers who are responsible for achieving key TTP program results have not been established.

Evidence: ONDCP FY 2004 Congressional Budget Submission (includes Performance Plan); CTAC Research and Development Blueprint Update, 2003 (ONDCP) ; discussions with ONDCP staff.

NO 0%
3.3

Are all funds (Federal and partners') obligated in a timely manner and spent for the intended purpose?

Explanation: CTAC uses interagency agreements to transfer appropriated funds to its technical and contracting agents. These agreements are prepared in advance of apportionment so that funding may be transferred as soon as it becomes available. There have been no negative findings from audits or other financial reviews.

Evidence: Treasury reports on obligations.

YES 10%
3.4

Does the program have procedures (e.g., competitive sourcing/cost comparisons, IT improvements, approporaite incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: ONDCP established efficiency measures and targets for the TTP in Summer of 2003. That measure requires ONDCP to keep administrative costs to less than 10% of program expenditures.

Evidence: ONDCP FY 2004 Congressional Budget Submission (includes Performance Plan)

YES 10%
3.5

Does the program collaborate and coordinate effectively with related programs?

Explanation: CTAC staff participate in organizations established by LEAs such as the TPC (Technology Policy Council) chaired by the National Institute of Justice and technology committees of the IACP (International Association of Chiefs of Police) and NSA (National Sheriff's Association). CTAC also attends Advanced Planning Briefings to Industry such as those held by TSWG (Technical Support Working Group - DOD). These meetings enable ONDCP to identify technologies desired by law enforcement.

Evidence: ONDCP FY 2004 Congressional Budget Submission (includes Performance Plan); CTAC Research and Development Blueprint Update, 2003 (ONDCP) ; discussions with ONDCP staff.

YES 10%
3.6

Does the program use strong financial management practices?

Explanation: CTAC's Technical and Contracting agents, as members of the Department of Defense use DOD financial management practices. No material internal control weaknesses, reports of erroneous payments, or the failure of financial management systems to meet statutory requirements have been identified.

Evidence: Army Audit Agency (AAA) audits, per ONDCP Financial Management Staff.

YES 10%
3.7

Has the program taken meaningful steps to address its management deficiencies?

Explanation: ONDCP staff have begun to define a limited number of specific, annual performance goals and measures. ONDCP staff have also committed to reviewing and revising where necessary the information that the TTP collects to determine program performance.

Evidence: Discussions with ONDCP staff.

YES 10%
3.CO1

Are grants awarded based on a clear competitive process that includes a qualified assessment of merit?

Explanation: ONDCP operates an aggressive outreach program for the TTP and applications to the program are reviewed on a first-come, first-served basis. ONDCP requires that each request be reviewed by one of ten active-duty law enforcement officers. These reviewers provide their expert judgment as to whether: the technologies requested will improve the operational capabilities of the requesting department or organization; the organization has the requisite infrastructure to integrate the technology into its daily operations; and the equipment is too complex for the organization. However, because the requests for assistance have exceeded the available funding, many LEAs cannot be given the equipment requested. ONDCP is working to establish adequate criteria to weigh the relative merit of applications.

Evidence: ONDCP FY 2004 Congressional Budget Submission (includes Performance Plan); CTAC Research and Development Blueprint Update, 2003 (ONDCP) ; discussions with ONDCP staff.

NO 0%
3.CO2

Does the program have oversight practices that provide sufficient knowledge of grantee activities?

Explanation: ONDCP contracts with five former-senior law-enforcement officials to follow-up with recipient agencies. ONDCP also requires recipients to complete an "evaluation" 90-, 180-, and 270-days after receiving the technology. The 90-, 180-, and 270-day evaluation forms request specific objective and quantifiable data regarding results achieved with use of TTP equipment. Agencies also provide information on the number of cases in which TTP equipment was employed and details of specific operational experience with the technology. ONDCP is also developing more relevant post-award data collection to improve its knowledge of grantee activities.

Evidence: ONDCP FY 2004 Congressional Budget Submission (includes Performance Plan); CTAC Research and Development Blueprint Update, 2003 (ONDCP) ; discussions with ONDCP staff.

YES 10%
3.CO3

Does the program collect grantee performance data on an annual basis and make it available to the public in a transparent and meaningful manner?

Explanation: A series of evaluation forms are collected from recipient agencies but they are not regularly analyzed and are not made available to public in an accessible manner, such as via a web site or widely available program reports. The lack of public access to such data, and other CTAC information, has been a consistent problem with the CTAC programs. ONDCP has committed to improving all forms of CTAC communication with the public.

Evidence: ONDCP FY 2004 Congressional Budget Submission (includes Performance Plan); CTAC Research and Development Blueprint Update, 2003 (ONDCP) ; discussions with ONDCP staff.

NO 0%
Section 3 - Program Management Score 60%
Section 4 - Program Results/Accountability
Number Question Answer Score
4.1

Has the program demonstrated adequate progress in achieving its long-term outcome performance goals?

Explanation: ONDCP has established a new long-term performance measure for the technology transfer program. However, the measure was just recently established and there are performance data available.

Evidence: See question 2.1

NO 0%
4.2

Does the program (including program partners) achieve its annual performance goals?

Explanation: ONDCP has established a new annual performance measure for the technology transfer program. However, the measure was just recently established and there are performance data available.

Evidence: See question 2.3

SMALL EXTENT 11%
4.3

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program performance goals each year?

Explanation: The CTAC TTP program has an established history of using approximately 90% of funding for equipment and training and only small amounts, generally about 10%, being required for administrative costs. Expecting further improvements beyond this level may be unrealistic.

Evidence: National Drug Control Strategy Counterdrug R&D Blueprint Update, February 2003, (p. 10; "Effectiveness and Interest in the Program")

NA 0%
4.4

Does the performance of this program compare favorably to other programs, including government, private, etc., that have similar purpose and goals?

Explanation: As stated in response to question 1.3, other Federal programs that either directly transfer technology to State and local law enforcement agencies are sufficiently different from the TTP program that no explicit comparison can be made. In addition, unlike many State and local assistance grant programs, the appropriation for the TTP has not been earmarked by the Congress for specific grantees.

Evidence: See questions 2.1 and 2.3

NA 0%
4.5

Do independent and quality evaluations of this program indicate that the program is effective and achieving results?

Explanation: There has been no independent evaluation of CTAC's TTP.

Evidence:  

NO 0%
Section 4 - Program Results/Accountability Score 11%


Last updated: 09062008.2003SPR