ExpectMore.gov


Detailed Information on the
Substance Abuse Drug Courts Assessment

Program Code 10009088
Program Title Substance Abuse Drug Courts
Department Name Dept of Health & Human Service
Agency/Bureau Name Substance Abuse and Mental Health Services Administration
Program Type(s) Competitive Grant Program
Assessment Year 2008
Assessment Rating Moderately Effective
Assessment Section Scores
Section Score
Program Purpose & Design 80%
Strategic Planning 88%
Program Management 60%
Program Results/Accountability 73%
Program Funding Level
(in millions)
FY2007 $10
FY2008 $10
FY2009 $40

Ongoing Program Improvement Plans

Year Began Improvement Plan Status Comments
2008

Take steps to improve linkage between DOJ and SAMHSA such as funding joint grant programs where appropriate.

Action taken, but not completed
2008

Begin collecting data on DOJ/SAMHSA common measure.

Action taken, but not completed
2008

Finalize Contract award for cross-site evaluation of the Drug Court Program

Action taken, but not completed

Completed Program Improvement Plans

Year Began Improvement Plan Status Comments

Program Performance Measures

Term Type  
Long-term/Annual Output

Measure: Juvenile Drug Courts: Increase number of clients served


Explanation:Number of clients that participate in an intake interview. Clients refers only to clients of juvenile drug courts funded by SAMHSA.

Year Target Actual
2006 Baseline 477
2007 821 856
2008 929 Oct-08
2009 449 Oct-09
2010 2,904 Oct-10
2011 3,056 Oct-11
2012 2,619 Oct-12
2013 2,794 Oct-13
Long-term/Annual Outcome

Measure: Juvenile Drug Courts: Percentage of clients that complete treatment


Explanation:The percent of clients who have successfully completed the SAMHSA treatment program. Clients refers only to clients of juvenile drug courts funded by SAMHSA.

Year Target Actual
2006 Baseline 68%
2007 69% 73%
2008 74% Oct-08
2009 75% Oct-09
2010 76% Oct-10
2011 77% Oct-11
2012 78% Oct12
2013 79% Oct-13
Annual Outcome

Measure: Juvenile Drug Courts: Increase percentage of clients receiving services who: a) Were currently employed or engaged in productive activities


Explanation:Proportion of clients (that participate in a 6 month-post-admission interview) that report being employed or being enrolled in school or job training. Clients refers only to clients of juvenile drug courts funded by SAMHSA.

Year Target Actual
2006 Baseline 86%
2007 87% 86%
2008 87% Oct-08
2009 88% Oct-09
2010 89% Oct-10
Annual Outcome

Measure: Juvenile Drug Courts: Increase percentage of clients receiving services who: b) Had a permanent place to live in the community


Explanation:Proportion of clients (that participate in a 6 month-post-admission interview) that report being housed (most of the time in the last 30 days) in an apartment/house/room that they own/rent. Living with someone else counts as housed if clients are under 18. Clients refers only to clients of juvenile drug courts funded by SAMHSA.

Year Target Actual
2006 Baseline 77%
2007 78% 80%
2008 81% Oct-08
2009 82% Oct-09
2010 83% Oct-10
Annual Outcome

Measure: Juvenile Drug Courts: Increase percentage of clients receiving services who: c) Had no involvement with the criminal justice system


Explanation:Proportion of clients (that participate in a 6 month-post-admission interview) that report not having (during the last 30 days): been arrested. Clients refers only to clients of juvenile drug courts funded by SAMHSA.

Year Target Actual
2006 Baseline 90.3%
2007 91% 91%
2008 92% Oct-08
2009 93% Oct-09
2010 94% Oct-10
Annual Outcome

Measure: Juvenile Drug Courts: Increase percentage of clients receiving services who: d) Experienced no/reduced alcohol or illegal drug related health, behavioral or social, consequences


Explanation:Proportion of clients (that participate in a 6 month-post-admission interview) that report: things not being stressful due to their use of alcohol or other drugs, not giving up or reducing activities because of their use or alcohol or other drugs and not having emotional problems because of their use of alcohol or other drugs. Clients refers only to clients of juvenile drug courts funded by SAMHSA.

Year Target Actual
2006 Baseline 89%
2007 90% 91.2%
2008 92% Oct-08
2009 93% Oct-09
2010 94% Oct-10
Annual Outcome

Measure: Juvenile Drug Courts: Increase percentage of clients receiving services who: e) Had no past month substance use


Explanation:Proportion of clients (that participate in a 6 month-post-admission interview) that report no use (during the last 30 days) of illegal drugs and alcohol. Clients refers only to clients of juvenile drug courts funded by SAMHSA.

Year Target Actual
2006 Baseline 68%
2007 69% 71%
2008 72% Oct-08
2009 73% Oct-09
2010 74% Oct-10
Annual Outcome

Measure: Juvenile Drug Courts: Percent of drug court participants who exhibit a reduction in substance use while in the drug court program. Measured in conjunction with DOJ.


Explanation:Proportion of clients that report reduced use of illegal drugs and alcohol.

Year Target Actual
2008 Baseline TBD
2009 TBD TBD
2010 TBD TBD
Annual Efficiency

Measure: Juvenile Drug Courts: Reduce cost-per-client served


Explanation:Total juvenile drug court program dollars for a given fiscal year divided by the number of clients that participate in an intake interview during the next fiscal year. Clients refers only to clients of juvenile drug courts funded by SAMHSA.

Year Target Actual
2006 Baseline $8,742
2007 $6,742 $6,463
2008 $5,905 Oct-08
2009 $5,610 Oct-09
2010 $5,330 Oct-10
Long-term/Annual Output

Measure: Adult Drug Courts: Increase number of clients served


Explanation:Number of clients that participate in an intake interview. Clients refers only to clients of adult drug courts funded by SAMHSA.

Year Target Actual
2005 Baseline 796
2006 345 357
2007 N/A N/A
2008 N/A N/A
2009 965 Oct-09
2010 2,904 Oct-10
2011 3,056 Oct-11
2012 2,619 Oct-12
2013 2,794 Oct-13
Long-term/Annual Outcome

Measure: Adult Drug Courts: Percentage of clients that complete treatment


Explanation:The percent of clients who have successfully completed the SAMHSA treatment program. Clients refers only to clients of adult drug courts funded by SAMHSA.

Year Target Actual
2005 Baseline 61%
2006 62% 66%
2007 N/A N/A
2008 N/A N/A
2009 67% Oct-09
2010 68% Oct-10
2011 69% Oct-11
2012 70% Oct-12
2013 71% Oct-13
Annual Outcome

Measure: Adult Drug Courts: Increase percentage of clients receiving services who: a) Were currently employed or engaged in productive activities


Explanation:Proportion of clients (that participate in a 6 month-post-admission interview) that report being employed or being enrolled in school or job training. Clients refers only to clients of adult drug courts funded by SAMHSA.

Year Target Actual
2005 Baseline 70%
2006 72% 86%
2007 N/A N/A
2008 N/A N/A
2009 88% Oct-09
2010 89% Oct-10
Annual Outcome

Measure: Adult Drug Courts: Increase percentage of clients receiving services who: b) Had a permanent place to live in the community


Explanation:Proportion of clients (that participate in a 6 month-post-admission interview) that report being housed (most of the time in the last 30 days) in an apartment/house/room that they own/rent. Living with someone else counts as housed if clients are under 18. Clients refers only to clients of adult drug courts funded by SAMHSA.

Year Target Actual
2005 Baseline 69.9%
2006 71% 77%
2007 N/A N/A
2008 N/A N/A
2009 82% Oct-09
2010 83% Oct-10
Annual Outcome

Measure: Adult Drug Courts: Increase percentage of clients receiving services who: c) Had no involvement with the criminal justice system


Explanation:Proportion of clients (that participate in a 6 month-post-admission interview) that report not having (during the last 30 days): been arrested. Clients refers only to clients of adult drug courts funded by SAMHSA.

Year Target Actual
2005 Baseline 89%
2006 90% 90.3%
2007 N/A N/A
2008 N/A N/A
2009 93% Oct-09
2010 94% Oct-10
Annual Outcome

Measure: Adult Drug Courts: Increase percentage of clients receiving services who: d) Experienced no/reduced alcohol or illegal drug related health, behavioral or social, consequences


Explanation:Proportion of clients (that participate in a 6 month-post-admission interview) that report: things not being stressful due to their use of alcohol or other drugs, not giving up or reducing activities because of their use or alcohol or other drugs and not having emotional problems because of their use of alcohol or other drugs. Clients refers only to clients of juvenile drug courts funded by SAMHSA.

Year Target Actual
2005 Baseline 86.6%
2006 88% 89%
2007 N/A N/A
2008 N/A N/A
2009 93% Oct-09
2010 94% Oct-10
Annual Outcome

Measure: Adult Drug Courts: Increase percentage of clients receiving services who: e) Had no past month substance use


Explanation:Proportion of clients (that participate in a 6 month-post-admission interview) that report no use (during the last 30 days) of illegal drugs and alcohol. Clients refers only to clients of adult drug courts funded by SAMHSA.

Year Target Actual
2005 Baseline 67%
2006 68% 68%
2007 N/A N/A
2008 N/A N/A
2009 73% Oct-09
2010 74% Oct-10
Annual Outcome

Measure: Adult Drug Courts: Percent of drug court participants who exhibit a reduction in substance use while in the drug court program. Measured in conjunction with DOJ.


Explanation:Proportion of clients that report reduced use of illegal drugs and alcohol.

Year Target Actual
2008 Baseline TBD
2009 TBD TBD
2010 TBD TBD
Annual Efficiency

Measure: Adult Drug Courts: Reduce cost-per-client served


Explanation:Total adult drug court program dollars for a given fiscal year divided by the number of clients that participate in an intake interview during the next fiscal year. Clients refers only to clients of adult drug courts funded by SAMHSA.

Year Target Actual
2006 Baseline $7,711
2007 N/A N/A
2008 $5,905 N/A
2009 $5,610 Oct-09
2010 $5,330 Oct-10

Questions/Answers (Detailed Assessment)

Section 1 - Program Purpose & Design
Number Question Answer Score
1.1

Is the program purpose clear?

Explanation: The purpose of the SAMHSA Juvenile and Adult Drug Courts Treatment Program is to provide a comprehensive array of substance abuse treatment and recovery support services for non-violent offenders in established drug court programs. The SAMHSA Juvenile Drug Court Program targets delinquent adolescents with substance use disorders and the Adult Drug Court focuses on adults charged in criminal courts who might otherwise be sentenced to jail. Drug Courts are designed to combine the sanctioning power of courts with effective treatment services to break the cycle of child abuse/neglect or criminal behavior, alcohol and/or drug use, and incarceration or other penalties. The key goals of the SAMHSA Drug Court Program are to promote abstinence and rehabilitation of drug use by engaging and retaining drug-involved offenders in treatment and related services in a collaborative process with the State Substance Abuse Director and Department of Juvenile and Adult Criminal Services. To meet these goals, drug courts use a coordinated multi-system approach that combines judicially supervised treatment, mandatory periodic drug testing, community supervision, and appropriate sanctions and other rehabilitation services with the Department of Justice and State Substance Abuse Director. SAMHSA funds are primarily used to provide treatment services such as detoxification and residential inpatient and outpatient treatment; but may also be used to provide support services to assist in recovery. These services are designed to maximize opportunities for drug court clients to create a lifetime of recovery and wellness for self, family, and community including: life skills management, relapse prevention, substance abuse education, parent education, and vocational training. Funds for this Program are provided to courts that are fully operational for treatment services only; they are not used to help establish new drug courts or to support daily operations of the court.

Evidence: The SAMHSA Drug Court Program is authorized under Title 5, PART A, Section 509 of the Public Health Services Act. The SAMHSA FY 2002 http://www.samhsa.gov/grants/content/2002/ti02008_drugcourts.html and FY 2008 http://www.samhsa.gov/Grants/2008/ti_08_007.aspx grant announcements clearly state the Program's purpose is "to expand and/or enhance substance abuse treatment services to problem solving courts which use the treatment drug court model in order to provide alcohol and drug treatment, recovery support services supporting substance abuse treatment, screening assessment, case management, and program coordination to adult defendants/offenders." The purpose of the program is also articulated in the Congressional Justification documents for each affected fiscal year. The FY 2009 SAMHSA Congressional Justification states that the courts are designed to combine the sanctioning power of courts with effective treatment services to break the cycle of child abuse/neglect or criminal behavior, alcohol and/or drug use, and incarceration or other penalties. http://www.samhsa.gov/Budget/FY2009/SAMHSA_CJ2009.pdf

YES 20%
1.2

Does the program address a specific and existing problem, interest, or need?

Explanation: In 2006, the National Survey on Drug Use and Health (NSDUH) found that 23.6 million Americans needed treatment for an alcohol or illicit drug use problem. However, only 10 percent of the individuals in need of treatment received it at a specialty facility in the past year. SAMHSA's Treatment Episode Data Set (TEDS) also found in 2006 that the criminal justice system was the principal source of referral for 37 percent of all substance abuse treatment admissions. In the absence of appropriate substance abuse treatment, non-violent offenders often enter into a recurring cycle of abuse and commit non-violent crimes. The availability of substance abuse treatment within the prison system is limited, with only a small percentage of people with alcohol and illicit drug problems being treated in specialty facilities designed to specifically meet their needs. Drug Courts are being created at a high rate to address these issues while also creating a challenge to support sufficient substance abuse treatment options for people referred by the court. SAMHSA funding is used to address these gaps in the continuum of treatment. Treatment Drug Courts are helping to fill this need by expanding and/or enhancing substance abuse treatment services in "problem solving" courts which use the treatment drug court model in order to provide alcohol and drug treatment, and recovery support services supporting substance abuse treatment, screening, assessment, and case management. Studies have shown that the treatment drug court model has proven to keep clients in treatment longer. Unless supervised by a judge, roughly 60 to 80 percent of offenders drop out of treatment prematurely and few successfully graduate.

Evidence: A 2000 National Center on Addiction and Substance Abuse (CASA) study found that four in five people arrested for juvenile crimes had problems with alcohol and other drugs (http://www.casacolumbia.org/Absolutenm/articlefiles/JJreport.pdf). For juvenile offenders, the CASA study also found that only 1.3% to 3.6% of the estimated 1.9 million juvenile's arrested received any form of addiction treatment. In 2006, the FBI estimated that over 14 million were arrested nationwide; two-thirds of those tested were found positive for illicit drugs at jail intake. It is estimated that there are between four and five million adult addicts in the justice system. There are over seven million Americans under justice supervision (probation, parole, jail, or prison) including approximately five million being supervised in the community. The DOJ Drug Use Forecasting (DUF) surveys and the subsequent Arrestee Drug Abuse Monitoring (ADAM) surveys have consistently demonstrated a high rate of drug abuse among arrestees with result findings indicating that 60% percent or higher tested positive for drugs. The same trends are also seen for those under 18 years of age. A National Center on Addiction and Substance Abuse (CASA) study found that four in five people arrested for juvenile crimes had problems with alcohol and other drugs. Further, the study of the juvenile system found that addiction treatment for the 1.9 million arrested juvenile offenders is scarce; only 68,600 received any form of addiction treatment. The number of drug courts has grown exponentially; according to the National Drug Court Institute between 2003 and 2004 the number of adult courts rose to 811, or 22%, and the number of juvenile courts rose to 357 a 33% increase. According to DOJ, a key component of a drug court includes providing access to a continuum of alcohol, drug, and other related treatment and rehabilitation services is a key component of drug courts (http://www.ojp.usdoj.gov/BJA/grant/DrugCourts/DefiningDC.pdf). The GAO found that in 2005 there were over 1,200 drug court programs in addition to about 500 being planned. United States Department of Justice, Federal Bureau of Investigation. (September 2007). Crime in the United States, 2006. Retrieved (February 6, 2007), from http://www.fbi.gov/ucr/cius2006/ United States Department of Justice, Bureau of Justice Statistics Correctional Surveys. (2006). The Annual Probation Survey, National Prisoner Statistics, Survey of Jails, and The Annual Parole Survey. Retrieved (February 6, 2008), from http://www.ojp.usdoj.gov/bjs/abstract/ppus06.htm and http://www.ojp.usdoj.gov/bjs/abstract/p06.htm The National Center on Addiction and Substance Abuse at Columbia University. (October 2004). Criminal Neglect: Substance Abuse, Juvenile Justice and The Children Left Behind. Retrieved (February 7, 2008), from http://www.casacolumbia.org/Absolutenm/articlefiles/JJreport.pdf The National Center on Addiction and Substance Abuse at Columbia University. (October 2004). Criminal Neglect: Substance Abuse, Juvenile Justice and The Children Left Behind. Retrieved (February 7, 2008), from http://www.casacolumbia.org/Absolutenm/articlefiles/JJreport.pdf

YES 20%
1.3

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: Although other State and local funds maybe used to support Drug Treatment Courts such as the Substance Abuse Treatment and Prevention Block Grant, the SAMHSA Drug Treatment Drug Court Program provides an additional service delivery mechanism for these services while also expanding capacity. The SAMHSA Treatment Drug Court program provides funding specifically for the expansion and/or enhancement of substance abuse treatment and recovery support services for drug court clients. SAMHSA awards grants directly to drug court clients through partnerships with established drug court judges or administrative court officers. Previous grant awards provided funding to both the courts and/or directly to service providers. The SAMHSA Treatment Drug Court grant program works in tandem with the Department of Justice's (DOJ) Drug Court grant program. The DOJ Program primarily provides funding for the infrastructure and services necessary to integrate treatment with assessment, and to allow for proper case processing and accountability in drug courts. SAMHSA's Program enhances treatment services and recovery support services necessary for the Drug Court Program. Both the DOJ and SAMHSA require grantees to address compliance with the established standards for model drug courts, such as the 10 key components of drug courts and coordinate to ensure that an individual drug court does not receive grant funds from both agencies for overlapping services. SAMHSA is also a co-sponsor, with the Bureau of Justice Assistance at DOJ, of the Federal Consortium Addressing Substance-Abusing Offenders (FCASOA) that increases collaboration and helps eliminate duplication in crime and substance abuse programs.

Evidence: To ensure that SAMHSA Drug Court funds are not being used for duplicative or overlapping purposes, SAMHSA's Treatment Drug Court RFA includes language designed to prevent duplication in the funding of SAMHSA and DOJ grants to drug courts through coordination of the separate funding sources. For example, the 2008 RFA included the following language: "SAMHSA will coordinate with the Department of Justice's Office of Justice Programs, Bureau of Justice Assistance (BJA) on FY 2008 Treatment Drug Court awards to ensure that an individual Treatment Drug Court does not receive both a new BJA implementation grant and a new SAMHSA Treatment Drug Court expansion and/or enhancement grant." In addition, SAMHSA and DOJ share copies of applications of potential awardees and work together during the review period for drug court grants to ensure there is no duplication. The CSAT Chief of the Targeted Population Branch also served as a reviewer on the Office of Juvenile Justice and Delinquency Prevention (DOJ) peer review of the FY 2007 Juvenile Drug Court grant program applications. A review of previous Adult and Juvenile Treatment Drug Courts grantees revealed only one grantee agency received both a BJA and SAMHSA/CSAT treatment drug court grant. The FASCO partnership also prevents redundancy because one of its primary functions is to exchange programmatic and policy issues/ideas regarding federal funding for the "drugs and crime" arena in order to better coordinate federal activities. SAMHSA also review all grant applications, budgets, and bi-annual reports to ensure that funds are not being used to supplant activities. While there are potential funding sources from State and local governments for treatment services, including the SAMHSA block grant funds, criminal justice and drug court clients are not priority populations for SAPT funding and only six states indicate using this funding for drug court clients.

YES 20%
1.4

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: Although the Program may have design flaws, these differences have not prevented the Drug Court Treatment Program from reaching its goals and objectives. For example, performance data has shown that the Program has achieved positive outcomes on its annual measures evidencing that the design of the Program has not prohibited its effectiveness. SAMHSA services provided may be under-researched and may not have a strong evidence base. However, SAMHSA has allowed applicants to use less rigorous data to ensure that applicants are not precluded from providing services to certain populations, i.e., the Native American population. To ensure clients are provided appropriate and culturally competent care, the Program requires a screening and assessment component. The awarding of past grants to service providers, as opposed to drug courts themselves also posed possible design flaws as it was difficult to ensure that funds were spent solely on drug court clients, and increased the possibility of funding service providers that may not be needed and/or utilized by the drug court. However, SAMHSA has now improved the design of the Program and has restricted eligibility only to Operational Drug Courts. Another design aspect that SAMHSA is working to improve is the coordination between DOJ and SAMHSA in administering the Program. An interagency agreement between OJJDP and SAMHSA has been put in place to jointly award juvenile Treatment Drug Court grants. Other positive aspects of program design include: limiting funding for drug courts that have been operational for a least one year and that have a designated judge (this helps to ensure that funds are spent only in courts that are capable of appropriately processing offenders), and requiring grantees to report on performance. Because of the cost involved in conducting longer term follow up, SAMHSA has chosen to collect data at six months post admission and at discharge and not beyond. The Program does not measure data at completion of the program or six months after completion. The lack of follow-up after a client ends treatment was marked as a deficiency by current research.

Evidence: Belenko (1998, 1999, 2001) stated that methodological deficiencies, such as the absence of short follow-up periods, 18 months or less, were a common deficiency. The Government Accountability Office analyzed 23 different drug court evaluations (http://www.gao.gov/new.items/d05219.pdf). The study found lower rates of re-arrest and reconviction for drug court participants; however, there was no conclusive evidence that specific drug court components, such as the behavior of the judge or the amount of treatment received, affected participant's recidivism while in drug court programs. There was also limited outcome data available to measure the effectiveness of drug court programs in reducing participant's substance use relapse. The GAO(2005), however, was unable to find conclusive evidence that specific drug court components, such as the behavior of the judge or the amount of treatment received, affected participant's recidivism while in the program. Although the drug testing data "generally showed significant reductions in use during participation in the program, self-reported results generally showed no significant reductions in use" (GAO, 2005, p. 6). This raises serious concerns about whether substance abuse treatment providers are administering evidence based treatment (EBT) as the drug court planning initiative and model recommends.

YES 20%
1.5

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: Program resources have not consistently been used in a manner that directly supports the program's purpose. SAMHSA has awarded grants to service providers instead of drug courts, which could have directed program resources to unintended beneficiaries, i.e. non-drug court clients. In addition, the awarding of previous grants to service providers, as opposed to drug courts themselves also increases the possibility that service providers may not be needed and/or utilized by the drug court, and limits the involvement of the actual Drug Court in determining the most appropriate and efficient services. The goal of the program is to address gaps in the continuum of care for drug court clients. Although the provider may not be the most effective recipient of Drug Court funds, the SAMHSA application process requires applicants to identify the target population and justify the need for the project with qualitative and quantitative data. Applicants must also provide a detailed description of the methods and approaches that will be used to reach the specific targeted populations. Additionally, grantees must demonstrate that they have existing relationships with drug courts, because the drug court judge is the catalyst for enrollment into the treatment program. Grantees determine whether resources are reaching the intended beneficiaries through data collected during intake assessments, and when clinical treatment and/or recovery support services are rendered. To avoid unintended subsidies, SAMHSA requires that Program funding for expansion and/or enhancement of services supplement, not supplant, any current funding for substance abuse clinical treatment and/or recovery support services

Evidence: Grantees submit bi-annual reports including financial reporting to SAMHSA that include information on the types of services provided. Grantees must provide a continuation application with a detailed budget breakdown of where funding is to be expended. Grantees must provide electronic GPRA data within 7 days to better assure "real time" data submission on clients admitted to the program. These data provide information on the target populations being served in order to ensure that those targeted in the grant application based on existing need are actually the beneficiaries of the services provided.

NO 0%
Section 1 - Program Purpose & Design Score 80%
Section 2 - Strategic Planning
Number Question Answer Score
2.1

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: The primary goals of the Program are to help substance using individuals achieve abstinence from substance use, improve quality of life, and reduce criminality. Drug court participants must be abstinent from substance use in order to complete a drug court program. Because the length of time it takes a participant to complete the program varies, the percentage of clients that complete SAMHSA's drug treatment program is the long-term measure used to reflect program performance. Completion here is defined as successfully completing treatment program elements specified at entry into the treatment program. This measure meaningfully reflects the Program's purpose because participants that complete all recommended treatment are more likely to remain abstinent from substance use and criminal behavior, and have improved quality of life. Drug Court participants are less likely to be successful if they do not complete the full range of treatment and recovery support services recommended by the judge. Another key objective of the Program is to expand the number of clients served. This is an important output measure because increasing the number of clients served strengthens the Program's ability to meet its long-term goal of reducing substance abuse and recidivism.

Evidence: Note: "Intake" for all measures is defined as intake to treatment funded by SAMHSA. Measure 1: Increase number of clients served ["clients" is defined as an unduplicated count of individuals who have received treatment]. Measure 2: the percent of clients that complete treatment. The 2009 Online Performance Appendix Pages 39 and 40 displays information on the first measure (http://www.samhsa.gov/Budget/FY2009/SAMHSA_Online_appendix.pdf). The 2002 and 2008 Requests for Applications include additional outcome data reported by grantees used to measure program effectiveness: http://www.samhsa.gov/grants/content/2002/ti02008_drugcourts.html (page 9) and http://www.samhsa.gov/Grants/2008/ti_08_007.doc (pages 9)

YES 12%
2.2

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: All long-term targets are set for 2013, reflecting a projection of the progress to take place five years out from the current year. The long term target for the percentage of clients that complete treatment is 79% for juvenile courts and 71% for adult courts. These are ambitious because other data show that these rates of treatment are significantly higher than what is typically seen in a treatment Program. For example, data from the Substance Abuse Prevention and Treatment Block Grant in 2005 for clients referred by the CJ system to outpatient treatment showed that 44% completed treatment. In addition, the Programs of Regional and National Significance, which have similar goals and objectives, data show a completion rate of about 64%. In addition, the ambitiousness of the target is also evidenced by the fact that as retention in treatment increases, SAMHSA has projected costs to go down. In other words, SAMHSA is requiring that grantees provide more efficient services without compromising the quality of the service. The Program aims to serve 5,587 clients in 2013, an increase of close to 5,000 compared to the number served in FY 2007. The number served by proposed increase in funding in 2009 is seen in 2010 and beyond as dollars are not released until the final day of the fiscal year. In order to set targets, an assumption has been made that new funds in 2009 will be split equally between juvenile and adult courts. Therefore, this target number is split equally, with juvenile and adult courts targeted to serve 2,794 clients each in 2013. The ambitiousness of the target is reflected in the link between the output, outcome and efficiency measures. In other words, SAMHSA has required that grantees provide the most efficient and effective services possible. The cost-per-client is projected to decrease by 5% each year which contributes to increases in the number of clients served. In order to ensure that the quality of service is still high, SAMHSA has also ambitiously set its long-term outcome measure target. While costs decrease, the outcome measure target of treatment completion is continually increased.

Evidence: Pages 39 and 40 of the 2009 of SAMHSA's Online Performance Appendix notes the baselines and targets for the long-term measures (http://www.samhsa.gov/budget/fy2009/SAMHSA_Online_appendix.pdf). The target for percentage of clients that complete treatment increases 6% from 2007 actuals for juvenile courts and 6% for adult courts. (Note: the OPA reflects data on the entire Drug Court Program. As agreed, this PART review only covers Adult and Juvenile Courts. This explains the apparent discrepancy in the data provided in the above response and the OPA.)

YES 12%
2.3

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: tbcThe Program has a limited number of specific and quantifiable measures assessed at six months post admission to the Program that relate directly to the long-term Program measures. The annual measures which directly correspond to the long-term Program goals and the overarching goals of the Drug Court Program are abstinence from substance use and no involvement with the criminal justice system (i.e. no past month arrests). Another critical component of the Program is to improve overall quality of life. Improvements in quality of life help clients meet the long-term program goals of achieving abstinence from substance use and criminal behavior. In order to assess the extent to which the Program is improving the quality of clients' lives, SAMHSA collects outcome-based data on: employment/engagement in productive activities, stable housing, health/social consequences of drug use and treatment completion. These measures are also included in the National Outcome Measures defined by SAMHSA as key priority areas relating to substance use. Success in each of these outcome areas contributes to reaching the Program's long-term goals of reduced substance abuse and recidivism. A new measure of substance use is being adopted this year to assess the effect of both DOJ's and SAMHSA's efforts - Percent of drug court participants who exhibit a reduction in substance use while in the drug court program.

Evidence: Measure 1: Had no past month substance use, measured at 6 months after intake ["no past month use" is no use of drugs or alcohol in the last 30 days]. Measure 2: Percentage of clients receiving services currently employed or engaged in productive activities. Measure 3: Percentage of clients receiving services with permanent places to live in the community. Measure 4: Percentage of clients receiving services with no involvement with the criminal justice system (i.e. no past month arrests). Measure 5: Percentage of clients receiving services with no/reduced alcohol or illegal drug related health, behavioral or social, consequences. Measure 6: Percentage of clients that graduated from the drug court treatment program. These measures and corresponding targets and baselines can be found in SAMHSA's Online Performance Appendix (http://www.samhsa.gov/Budget/FY2009/SAMHSA_Online_appendix.pdf). (Note: the OPA reflects data on the entire Drug Court Program. As agreed, this PART only covers Adult and Juvenile Courts. This is the reason for the apparent discrepancy in the data provided in the above response and the OPA.) The National Outcome Measures used to measure this program's achievement in improving quality of life have been determined to embody, "meaningful, real life outcomes for people who are striving to attain and sustain recovery; build resilience; and work, learn, live, and participate fully in their communities" (http://www.nationaloutcomemeasures.samhsa.gov/).

YES 12%
2.4

Does the program have baselines and ambitious targets for its annual measures?

Explanation: The Program has baselines and ambitious targets for its annual measures. Targets set for 2008 adult courts are based on actual performance in the previous years of the program with the percentage of clients targeted to meet each goal set at the following levels: abstinence at 72%, employment at 87%, housing at 81%, no past month arrests at 92%, no social/behavioral consequences at 92% and treatment completion at 73%. (Due to the absence of grantees for juvenile courts, there are no juvenile data for 2008.) Targets for adult and juvenile courts in 2009 are set at the following levels: abstinence at 73%, employment at 88%, housing at 82%, no past month arrests at 93%, no social/behavioral consequences at 93% and treatment completion at 74%. The increases for these measures are set at a one to two percent increase per year. However, they can be classified as ambitious because outcomes improve as costs per client served are expected to decrease by 5% each year. Rather than holding outcomes constant while costs decrease, which would in itself show improved efficiency, SAMHSA has chosen to be ambitious and progressively increase targets for outcomes. There is no baseline for the joint measure with DOJ yet, but SAMHSA will begin working with DOJ to determine data collection tools and a baseline.

Evidence: Pages 39 and 40 of the 2009 of SAMHSA's Online Performance Appendix reports these baselines and targets (http://www.samhsa.gov/Budget/FY2009/SAMHSA_Online_appendix.pdf). (Note: the OPA reflects data on the entire Drug Court Program. As agreed, this PART review only covers Adult and Juvenile Courts. This is the reason for the apparent discrepancy in the data provided in the above response and the OPA.)

YES 12%
2.5

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: SAMHSA requires all grantees to commit to and work towards the annual and long-term measures of the Program. SAMHSA collects data on demographic data, drug use characteristics, employment status, housing status, criminal justice involvement, social support and services received on each client served. Grantees are required to submit this data to SAMHSA within seven business days of the completion of the client interview via the use of a real-time data entry and reporting system, the Services Accountability Improvement System (SAIS). Data are monitored regularly to ensure that grantees are meeting the goals and objectives being measured. In addition, the FY 2008 Request for Application, requires Drug Court grantees to have established formal agreements to work with treatment providers and to ensure providers are aware of the Program's purpose and goals. SAMHSA has improved coordination with DOJ, State Substance Abuse agencies and the courts by requiring grantees to submit a Public Health Systems Impact Statement (PHSIS) if they are a community-based, non-governmental service provider and they did not transmit their application through the State. The PHSIS must be submitted to the head of appropriate State and local health agencies. The PHSIS is intended to keep State and local health officials informed of proposed health services grant applications that are submitted by community-based, non-governmental organizations within their jurisdictions. Although State and local governments are not required to submit the PHSIS, the 2008 RFA included language that required applicants to demonstrate evidence of direct and extensive consultation and collaboration with the corresponding State substance abuse agency in the planning, implementation and evaluation of the project by including a letter from the agency Director or designated representative that indicated the involvement of the agency. Furthermore, the applicant must submit a copy of the application to the Single State Agency (SSA- alcohol and drug abuse agency) which has 60 days for review and provide feedback and comment to SAMHSA. This allows the SSA to determine whether the specific proposed project comports to the established legislative and executive program priorities regarding drug courts in the State. DOJ and SAMHSA have required grantees to use the 10 Key Components of Drug Courts to operate their Program and both participate in the Federal Consortium (a forum both DOJ and SAMHSA use where changes or additions to approaches can be discussed between both agencies). SAMHSA has also expanded its partnerships with both federal and national entities to better ensure successful completion of long-term goals for drug courts. For example, DOJ and SAMHSA entered into an Inter-agency Agreement in FY 2007 to jointly fund and administer the 2007 juvenile drug court grants issued by DOJ.

Evidence: The 2008 Request for Application notes the grantee reporting and formal agreement requirements such as requiring grantees to assess performance measures and outcome questions including the effect of the intervention on participants, and various GPRA measures (http://www.samhsa.gov/Grants/2008/ti_08_007.doc page 9 and http://www.samhsa.gov/grants/content/2002/ti02008_drugcourts.pdf page 9).

YES 12%
2.6

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: In addition to collecting extensive performance data on a regular basis, grantees have also been required to conduct regular local assessments of their Program. In 2008, SAMHSA identified resources for a national cross-site evaluation. This evaluation contract will be awarded to a recognized institution with demonstrated ability to conduct high-level, high quality evaluations. The evaluation will be high in quality and rigor and will include additional data collection points not captured in the GPRA data collection process. The evaluation will be large in scope and cover the entire Drug Court Program to be funded in 2008. This cohort of grantees will be expected to comply with the regulations of the evaluation and submit data to SAMHSA in order to gauge program effectiveness on a regular basis throughout the period of performance of the grant. Given the Contract will be awarded to a non-governmental organization, the Evaluation is expected to be fully independent in nature and unbiased. The Request for Task Order Proposals (RFTOP) was sent to potential bidders on April 28, 2008. Contract proposals are due May 29, 2008. A Contract will be awarded by September 30, 2008.

Evidence: Treatment Drug Court Evaluation Draft Statement of Work

YES 12%
2.7

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: The SAMHSA budget request does not directly correlate with program accomplishments and goals. The requested budget increase for FY 2009 is supported by performance data that shows the program was meeting or exceeding its performance targets and explains how many grants the funding would support. However, it is not clear how budget requests impact the Program's ability to meet annual and long-term goals. Performance outcomes are not split among the different kinds of drug courts, i.e. adult and juvenile. This disconnect makes it difficult to assess the relationship between the budget and the Program's performance. SAMHSA also provided insufficient evidence that the resources/performance mix enables the program to achieve its performance goals. SAMHSA does not report all of the direct and indirect costs needed to meet its performance targets.

Evidence: The FY2009 Congressional Justification, p. CSAT-22, notes that the increased funding will fund all grant and contract continuations and 82 new Drug Court grants, however it doesn't explain how this will influence the expected Program outcomes, or provide a break-out by Juvenile and Adult Drug Courts. http://www.samhsa.gov/Budget/FY2009/SAMHSA_Online_appendix.pdf).

NO 0%
2.8

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: SAMHSA continually monitors grantees to ensure progress is being made in meeting goals and improving program performance. SAMHSA holds annual conferences with grantees to foster exchange of lessons learned among grantees, review state-of-art practices, and provides updates to program performance. All current grantees are in attendance at these annual meetings. Participants at the annual conferences discuss fiscal management practices, data reporting requirements (e.g. the Financial Status Report), and how to strengthen client and provider outreach. Program officers are alerted on a monthly basis of underperforming grantees. For these grantees, corrective actions are put in place, including use of various modes of technical assistance and training, site visits, and budgetary restrictions as appropriate. After reviewing the treatment drug court portfolio, SAMHSA has significantly revised the FY 2008 Request for Application to strengthen program coordination efforts among system participants. While the Program has not had deficiencies in meeting its long-term goals, SAMHSA took steps to improve the overall Program. Review of previous grants indicated that treatment provider grantees seem to have a more difficult time in meeting the goals for client loads. Since the door to eligibility resides with the drug court/judge, it was determined that they would become the only eligible applicants in the FY 2008 RFA. However, the judge does not hold the same power and oversight over clinical/treatment decision-making and/or provision of services, so, the 2008 Request for Application restricts eligibility to drug courts that can document ongoing relationships with community providers. The drug court judge has the authority to assign clients to treatment and serves as the catalyst for enrollment into community-based treatment programming. By only funding courts it will help to ensure that there is a steady flow of clients, which will minimize any recruitment issues that have been identified with past grantees. SAMHSA will also assess upcoming grantee performance using a number of accountability measures included in the 2008 Request for Application including possible supplemental awards based on performance. The 2008 cohort of grantees will undergo a cross-site evaluation which collect data regularly throughout the grant period and results will help SAMHSA determine the appropriateness of current targets and increase understanding of client outcomes.

Evidence: Grantees complete a standard status report semi-annually with information on collaboration, challenges, and implementation issues, as well as financial information, which are incorporated into semi-annual profiles for each grantee. The requirement for the incentives is described in the 2008 RFA.(http://www.samhsa.gov/Grants/2008/TI_08_012.pdf page 12). SAMHSA's FY 2009 Congressional Justification contains the long-term and annual Program goals regarding capacity and helping clients achieve abstinence from alcohol and drug use. (http://www.samhsa.gov/Budget/FY2009/SAMHSA_CJ2009.pdf)

YES 12%
Section 2 - Strategic Planning Score 88%
Section 3 - Program Management
Number Question Answer Score
3.1

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: SAMHSA requires grantees to collect and report outcome and output data on a regular basis and SAMHSA has monthly calls with grantees based on these reports. SAMHSA also collects and monitors administrative data from grantees on fraud and abuse. All data are submitted in compliance with the Government Performance and Results Act (GPRA) via the use of a real-time data entry and reporting system. Once data are submitted, a series of performance monitoring reports are automatically generated. Data collected relate specifically to relevant characteristics of the client with respect to their substance use behaviors. Data are collected on demographics, substances used, employment status, housing status, criminal justice involvement, social support and services received. Given the type of data being collected and the relevance to treatment plans, data are collected by the substance abuse treatment provider. Data are collected on a regular basis meaning that when each client receives an intake assessment, they also are administered a GPRA interview. Data are also collected at six months post admission and discharge. Once the interview form is completed with the client, the grantee enters it into the SAIS system within 7 business days; therefore, SAMHSA gets data regularly to monitor the Program. Data are used to adjust management priorities specifically as it relates to the provision of Technical Assistance, which is a primary management function of the Program. Specific clinical training, such as training on Motivational Enhancement Therapy, is provided to grantees to ensure positive outcomes. SAMHSA has also held grantee conference calls specifically related to outcome data to allow grantees to share lessons learned. Grantee performance as reflected in these reports is considered when making decisions about the Program such as setting/adjusting performance targets, modifying program priorities and implementing new management activities as necessary. In addition, grantees submit biannual reports providing SAMHSA with additional programmatic information. These reports are used in conjunction with other data to monitor grantee performance.

Evidence: SAMHSA collects timely performance information from grantees through their real-time data entry and reporting system, the Services Accountability Improvement System (https://www.samhsa-gpra.samhsa.gov/home/index.htm). Required data collection tools can be found in the 2008 RFA (pages 63-88, http://www.samhsa.gov/Grants/2007/TI_07_005.pdf), the 2005 Notice of Funding Availability (http://www.samhsa.gov/Grants/2005/nofa/ti05005_drugcourts.aspx). Program grantees are required by SAMHSA to complete and submit bi-annual progress reports with narrative on successes and challenges, and data on Program performance over the previous six months (https://www.samhsa-gpra.samhsa.gov/CSAT/System.aspx).

YES 10%
3.2

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: The level of responsibility for the Program begins with the agency administrator and cascades down to individual project officers who oversee the grant program. A primary staff person and team leader is assigned to monitor Program performance. These staff are held accountable for achieving key Program results through employee performance reviews that have built in metrics to hold responsible staff accountable for meeting Program goals. Action plans are implemented for underperforming staff. Grants staff manages program results through submissions to the Payment Management system (PMS), which indicates drawdown of Federal Funds. To enhance accountability, the FY 2008 RFA indicates that annual continuation awards will be contingent on grantee progress and timely submission of required data and reports. The RFA also includes supplemental awards for the strongest performers (contingent upon available funding) and provides a formula by which these grantees may receive incentive funds. There are several mechanisms by which program staff hold grantees accountable for meeting performance targets, including review of bi-annual progress reports, review of electronic submission of Government Performance and Results Act (GPRA) data, bi-monthly conference calls, on-site visits by consultants conducting Technical Assistance to the grantees, and at least one annual grantee meeting where performance is discussed. If a TA is completed by a consultant the consultant must provide a written document to the GPO stating what occurred on the visit, the TA provided to the grantee, the audience who received the TA, and future recommendations made to the grantee. SAMHSA's data reporting system alerts project officers if grantees are failing to meet GPRA goals. Program staff provide guidance, technical assistance and require corrective action plans to ensure goals are met. Grantees provide projected targeted amounts to be spent in budgets submitted in their applications as well as a narrative justification of the items included in their proposed budget, a description of existing resources and other support they expect to receive for the proposed project. Significant re-budgeting requires prior approval by SAMHSA's Grants Management.

Evidence: Government employees are held accountable for program performance by the Performance Management Appraisal System and through their individual performance plans. For drug court project officers and team leaders, employee performance is tied to Program performance indicators. Data and reporting requirements for grantees are documented in the 2002 and 2008 RFA's http://www.samhsa.gov/grants/content/2002/ti02008_drugcourts.html and (pages 5-7, 23-25) http://www.samhsa.gov/Grants/2008/ti_08_007.doc (pages 9, 10, 30, 31). Evidence of these data submissions can be found in the quarterly program profiles compiled by SAMHSA staff. A new performance mechanism to reward the strongest performing grantees has been incorporated into the 2008 RFA http://www.samhsa.gov/Grants/2008/ti_08_007.doc (pages 11-12). Project officers are required to document grantee performance and share this information with SAMHSA's grants management office.

YES 10%
3.3

Are funds (Federal and partners') obligated in a timely manner, spent for the intended purpose and accurately reported?

Explanation: Program awards are not obligated consistent with the overall SAMHSA program contracts and grants plan. After two years of grant closeout, multiple Financial Status Reports (FSR), SF-269 have not been validated and are still in dispute. Some of these grantees may be required to repay the government if there's an accounting error on the FSR. Although grantees have established schedules for obligations and must draw down funds, not all grantees have consistently met these schedules. Other FSRs have also found that activities were not accurately reported. There are procedures in place to ensure coordination between Court and provider to ensure that the provider is serving the Drug Court participants. In addition to the signed application indicating that the proposal is to serve drug court clients, each Treatment Drug Court RFA ( 2002-2008) has stipulated that the grantee must "send a Treatment Drug Court team consisting of a minimum of six people (judge, project director, clinical director, evaluator, and representatives from the prosecutor's office and the defense bar.

Evidence: Grantees report expenditures yearly via Financial Status Reports, and quarterly via Federal Cash Transaction (PSC272) reports. For new grantees not required to have OMB A-133 audits, SAMHSA's Department of Grants Management's financial advisory services officers conduct financial capability reviews covering the 17 financial management control areas. (http://www.samhsa.gov/Grants/generalinfo/GrantsFinancialManagement.pdf). Federal regulations (45 CFR Part 74 and 45 CFR Part 92) provide standards for financial management systems of grantee organizations.

NO 0%
3.4

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: The Program has an efficiency measure, reduction in costs per client served, with baselines and targets used to ensure that costs are reasonable. The efficiency measure will be used in tandem with the outcome measures to ensure that while costs decrease, the quality of service being provided does not diminish. In addition, an incentive procedure in the 2008 Program RFA promotes efficiency and effectiveness by rewarding those grantees who are performing above their target performance level. The program also has regular procedures in place to achieve efficiencies and cost effectiveness, including a cap on infrastructure spending at 15 percent of grant funds received.

Evidence: SAMHSA has an efficiency measure - reduction in cost per client served. This measure is targeted to decrease 25% by 2013. An analysis of grant budgets indicates that grantees are meeting the infrastructure costs measure, with most grantees well under the threshold of fifteen percent. The requirements for keeping infrastructure costs low are found in SAMHSA's 2008 RFA (page 9 http://www.samhsa.gov/grants/content/2002/ti02008_drugcourts.html). The RFA details language as follows with respect to program incentives: For year 3 of the Treatment Drug Court grant program, SAMHSA will review each grantee's GPRA data submissions and assess whether a grantee has: 1) met or exceeded its target for the number of clients served by 25 percent or more; 2) met or exceeded its target for 6 month follow-up rates; and 3) provided services within approved costs. Any grantee that has demonstrated appropriate financial management of the grant and has exceeded its targets for the number of clients served by 25 percent or more, exceeded its target for 6 month follow-ups, and provided services within allowable cost bands, may receive a supplemental award of up to 5 percent of the third year requested amount based on performance. Supplemental award amounts will be determined on a sliding scale based on availability of funds and the grantee's achievement of performance goals and demonstration of sound fiscal management.

YES 10%
3.5

Does the program collaborate and coordinate effectively with related programs?

Explanation: Although SAMHSA coordinates with DOJ to ensure an individual drug court does not receive grant funds from both agencies for overlapping services. SAMHSA has not demonstrated that it collaborates and coordinates sufficiently with DOJ's Drug Court Program, particularly with respect to adult and family drug courts. Although SAMHSA has shared efforts to maximize the impact of drug courts through the Federal Consortium Addressing Substance-Abusing Offenders it is unclear what meaningful actions in management and resource allocations have occurred from this meeting. SAMHSA and DOJ have also not effectively coordinated grant solicitations, joint consideration of grant applications/awards, and joint efforts to ensure that drug courts are not just fully functional from a criminal justice perspective, but also provide a full range of treatment and recovery support services. SAMHSA has worked with National Association of Drug Court Professionals (NADCP) by supporting the annual NADCP Drug Court Training Conference. SAMHSA has also worked to try and improve the coordination at the State and local level by requiring Drug Court applicants to submit a Public Health Systems Impact Statement to local and State Public Health agencies and to submit their application to the State Substance Abuse agencies (SSA's). This informs agencies involved in substance abuse issues of the type of funding requested and provides agencies the opportunity to comment on whether the proposed Program is consistent with the publicly funded priorities for substance abuse treatment in that locale.

Evidence: Since 2005, SAMHSA Treatment Drug Court RFAs have included language designed to prevent duplication in the funding of SAMHSA and DOJ grants to drug courts through coordination of the separate funding sources. For example, the 2008 RFA included the following language: "SAMHSA will coordinate with the Department of Justice's Office of Justice Programs, Bureau of Justice Assistance (BJA) on FY 2008 Treatment Drug Court awards to ensure that an individual Treatment Drug Court does not receive both a new BJA implementation grant and a new SAMHSA Treatment Drug Court expansion and/or enhancement grant." SAMHSA served as a reviewer on the Office of Juvenile Justice and Delinquency Prevention (DOJ) peer review only on the FY 2007 Juvenile Drug Court grant program applications.

NO 0%
3.6

Does the program use strong financial management practices?

Explanation: SAMHSA does not utilize all financial management practices to ensure that payments are made properly for the intended purpose to minimize erroneous payments. Financial information is also not provided in a timely fashion as evidenced by Financial Status Reports that still do not have an definite conclusion. Procedures for overseeing funds include enforcement of grants terms and conditions, mandated financial status reporting, electronic centralized payment and cash management system, and OMB Circular A-133 audit requirements. However, SAMHSA cannot ensure that all Drug Court funds in the President's Budget have been awarded to Drug Courts grants and obligated for the intended purposes as stated in the President's Budget. In addition, multiple State audits have revealed that SAMHSA lacked the procedures to ensure that Federal reports are accurately prepared and audit resolution was required for three of the drug court grantees.

Evidence: Grants management information related to various requirements is provided on the SAMHSA website. These include making a determination of the organization's ability to adequately administer Federal awards and requiring grantees to attend an orientation session that covers financial management processes (http://www.samhsa.gov/grants/management.aspx). There were no A-133 audit findings for Program grantees.. (http://www.gsa.gov/Portal/gsa/ep/formslibrary.do?viewType=DETAIL&formId=91E81CF31C11935585256AA100426D84). Payments are available through the DHHS Payment Management System a centralized grants payment and cash management system, operated by the HHS Program Support Center, Division of Payment Management (http://www.dpm.psc.gov/).

NO 0%
3.7

Has the program taken meaningful steps to address its management deficiencies?

Explanation: The Program has some systems in place to try and prevent management deficiencies and to identify and correct deficiencies as they occur. Government project officers in collaboration with grants management staff, oversee grants through extensive reporting requirements. SAMHSA also provides technical assistance to individual grantees to ensure successful project implementation and grantees are required to develop corrective action plans when appropriate. For example, if a grantee is not successfully recruiting clients, a corrective action plan may include steps to request technical assistance specifically on recruitment. This example has been identified as a common issue faced by grantees and steps have been taken to correct the problem. Also, during the mandatory annual SAMHSA/CSAT Grantee Meeting issues specific to SAMHSA grantees are identified and workshops are developed based on the needs of the grantees. SAMHSA has also taken meaningful steps to strengthen the Program in the FY 2008 RFA by encouraging and sustaining performance through a supplement to the existing award for grantees with exceptional performance. In order to better assure that corrections occur and are completed in a specific, agreed upon timeline with the grantee, Program staff closely monitor the GPRA status report to identify grantees that are not reaching their goals. Grantees that are not able to quickly rectify deficiencies are required to explain the issues and their perceived causes, and to submit a plan, worked on and agreed to by both grantee and GPO, detailing what corrective actions will be taken.

Evidence: Examples of corrective action plans submitted by grantees who were not meeting performance goals are provided. Plans detail steps to correct the problems. This language includes, working with probation officers to inform them of the importance of meeting targets. Language also describes getting "buy in" from probation officers in addition to potential TA requests. As needed, SAMHSA staff work with grantees to develop Corrective Action Plans. Corrective Action Planning requires coordination among the program staff, grants management staff, grantee, and possibly technical assistance contractors to assist the grantee to overcome any management deficiencies. The Team Leader-Project Officer Review Board meets during the period that SAMHSA is reviewing and processing discretionary grantee applications for continuations of funding. The purpose of the Board is to hear the project officers recommendations on each selected grant and then recommend specific programmatic of funding actions for each grant reviewed.

YES 10%
3.CO1

Are grants awarded based on a clear competitive process that includes a qualified assessment of merit?

Explanation: Grants are awarded through fair and open competition. All grant announcements are posted publicly on the SAMHSA website and by press release. SAMHSA is required by statute to conduct a thorough and impartial peer review of applications. Peer reviewers are knowledgeable in the field and objectively score applications based on clearly established review criteria. The grantee applications are reviewed and assigned a point value based on how well they meet the criteria for responses to the following five sections outlined in the RFA: A) statement of need, B) proposed evidence-based service/practice, C) proposed implementation approach, D) staff and organizational experience and E) performance assessment and data. Applicants' scores are ranked and the highest scored applications are awarded grant funds.

Evidence: The 2002 and 2008 Program RFAs explain the extensive review process and selection criteria for applicants ( page 29 http://www.samhsa.gov/grants/content/2002/ti02008_drugcourts.html and pages 21-29 http://www.samhsa.gov/Grants/2008/ti_08_007.aspx). SAMHSA uses an established external peer review process to assess the technical merit of all Program applications.

YES 10%
3.CO2

Does the program have oversight practices that provide sufficient knowledge of grantee activities?

Explanation: SAMHSA has sufficient oversight of Program grantees. Careful project management by Government Project Officers ensures that grantees are adhering to established program goals. SAMHSA regularly collects timely and credible performance information from the program grantees to manage the program and to improve performance. Program grantees must report financial and outcome data to SAMHSA. These reports include reporting categories which indicate how the grantee is achieving the programmatic goals (e.g., clients served, progress in achieving the stated project goals and objectives). To ensure data quality, the SAIS system has automatic validation and verification checks. SAMHSA also provides targeted technical assistance on programmatic, administrative/managerial, clinical and evaluation topics and issues. TA is provided through site visits to the grantee locations. Through its support contracts, SAMHSA has provided technical assistance and training support to each cohort of treatment drug courts grantees. In addition to formal technical assistance, conference calls are held bi-monthly with all grantees and phone calls with individual grantees occur frequently.

Evidence: Grantees are required to comply with several reporting requirements in the 2002 and 2008 RFA's http://www.samhsa.gov/grants/content/2002/ti02008_drugcourts.html and pages 9-11 http://www.samhsa.gov/Grants/2008/ti_08_007.aspx). Grantees are monitored through their required data submitted through the Services Accountability Improvement System (SAIS) (https://www.samhsa-gpra.samhsa.gov). Evidence of these data submissions can be found in the quarterly program profiles compiled by SAMHSA (the 2006 profile can be found at: https://www.samhsa-gpra.samhsa.gov/CSAT/view/docs/Program_Profiles_2006.pdf).

YES 10%
3.CO3

Does the program collect grantee performance data on an annual basis and make it available to the public in a transparent and meaningful manner?

Explanation: The program collects grantee performance data on a regular basis, within 7 business days of the client interview. Once submitted, data are aggregated and annual data available to the public on SAMHSA's website; however, data is not disaggregated at the grantee level nor by the type of Drug Court Program. Additional information on how grantees have implemented their programs is provided in required biannual reports; however this information is not broken out by the type of drug court or at the grantee level.

Evidence: Treatment Drug Court aggregate data is posted on the SAMHSA public website (https://www.samhsa-gpra.samhsa.gov/CSAT/view/docs/Program_Profiles_2006.pdf ). In addition, performance data for the program as a whole was reported in the 2009 SAMHSA budget justification (http://www.samhsa.gov/Budget/FY2009/index.aspx ).

NO 0%
Section 3 - Program Management Score 60%
Section 4 - Program Results/Accountability
Number Question Answer Score
4.1

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: The Program has demonstrated adequate progress in achieving its long-term goals. The Program has a long-term goal of increasing the number of clients served to 5,238 by 2012. In FY 2007 the juvenile court Program served 856 clients, 35 more than targeted for, and 379 more than the previous year. Although data are not available in 2007 for Adult Courts, data in 2005 show that 796 clients were served. While the figure decreases in 2006, this is due to the cyclical nature of funding and not poor Program performance. As grants ended, funding for adults for clients served in FY 06 was 51% of that from the previous year. The second long term measure is the number of clients completing treatment. Juvenile courts missed its target of 87% in 2007 by 1%, this target is projected to meet 89% in 2010. For adult courts, progress has been demonstrated in that the FY 05 percent of clients completing treatment was 61% and the FY 06 percent was 66%. SAMHSA is working on developing targets for the new long-term joint measure with the Department of Justice.

Evidence: SAMHSA's FY 2009 Online Performance Appendix contains information on the percentage of clients served: http://www.samhsa.gov/Budget/FY2009/SAMHSA_Online_appendix.pdf. Note: the OPA reflects data on the entire Drug Court Program.

LARGE EXTENT 13%
4.2

Does the program (including program partners) achieve its annual performance goals?

Explanation: Data from the last complete fiscal year demonstrate that the Program is meeting its six annual performance goals. The FY 2007 target for juvenile courts for number of clients served was exceeded with 856 clients having been served. The FY 2007 targets for juvenile courts were met for all measures with the exception of employment/productive activity which was nearly met (target missed by 1%). While adult courts were not funded in 07, data are available for the last complete year of Adult Courts and demonstrate achievement of targets for annual measures. All targets were met or exceed in FY 06.

Evidence: SAMHSA's FY 2009 Online Performance Appendix contains information on the percentage of clients served: http://www.samhsa.gov/Budget/FY2009/SAMHSA_Online_appendix.pdf. Note: the OPA reflects data on the entire Drug Court Program. As agreed, this PART review only covers Adult and Juvenile Courts. This explains the apparent discrepancy in the data provided in the above response and the OPA.) Data on the second measure is available in internal SAMHSA documentation.

YES 20%
4.3

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: The Drug Court approach has proven to be a cost-effective approach in providing a less costly alternatives to incarceration fro non-violent drug offenders. In FY 2007, the Program provided services to juvenile clients at a cost of about $6,000 per client, and in FY 2006 the Program spent about $8,000 per client served (2007 numbers are not available for adult courts). The Drug Court Program also uses the reduction in cost-per-client as its efficiency measure. Savings have been demonstrated over time with a decrease in cost of over $2,000 per juvenile client between FY 2006 and FY 2007 (similar data is unavailable for adult courts). SAMHSA will continue to measure its efficiency via use of the cost-per-client measure. Additionally, SAMHSA strictly monitors the spending of administrative funds used to implement its Drug Court Program requiring that no more than 15% go to infrastructure for the management of the grant. The 2008 Program RFA also includes supplemental awards for the strongest performers (contingent upon available funding) and provides a formula by which these grantees may receive incentive funds.

Evidence: The 2008 RFA requires grantees to provide details in their application of how they will control costs, prevent fraud, waste and abuse and hold infrastructure costs to no more than 15% of the grant (page 9) http://www.samhsa.gov/Grants/2008/TI_08_012.pdf), similar requirements are also noted in the 2002 RFA (page 9) http://www.samhsa.gov/grants/content/2002/ti02008_drugcourts.html). The 2009 Online Performance

YES 20%
4.4

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: The performance of this Program compares favorably to other programs. The Substance Abuse Prevention and Treatment Block Grant Program collects data specifically on clients referred by the CJ system. A review of that data in comparison to the Drug Court Program data suggest that the Drug Court Program has favorable results. For example, 52% were employed, 75% were abstinent from drugs, and 88% reported no past month arrests. When compared to the Drug Court Program, data from the same year for the Programs of Regional and National Significance (PRNS), show that the Drug Court Program is performing favorably. For example, in 2007 PRNS was 60% for abstinence, 53% employment, was 46% for housing. Comparatively in 2007 juvenile Program participants exhibited 71% abstinence, 86% employment, was 80% for housing and in 2006 adult Program participants exhibited 68% abstinence, 86% employment, was 77% for housing SAMHSA's program also compares favorably to the Department of Justice's (DOJ) drug court program. DOJ reported that 41% of program participants reoffended while in the program. Although SAMHSA's measure may be slightly different, only 9% of juveniles in 2007 and 10% of adults in 2006 had involvement with the criminal justice system. Additionally, DOJ reported that in 2007 37% of participants completed treatment, while SAMHSA's Program reported that 66% of adults in 2006 completed treatment and 73% of juveniles in 2007 completed treatment. Other evaluations of Drug Courts also show favorable results when compared to SAMHSA's Drug Court Program. For example, the Anarundel Drug Court Program reported a treatment completion rate of 55% compared to SAMHSA's 66 % and 73% for adults and juveniles, respectively. Additionally, the Anchorage Felony Drug Court/DUI Court and Bethel Threraputic Court comparisons with SAMHSA Drug Courts show SAMHSA has favorable results. For example, 63% of participants held a job in the Alaska Courts which compares to 76% of SAMHSA Drug Court participants.

Evidence: The Programs of Regional and National Significance include substance abuse treatment programs that serve women, minority populations, those affected with HIV, adolescents and general populations. Data on SAMHSA's drug court performance can be found in the 2009 Online Performance Appendix: (page 38) http://www.samhsa.gov/Budget/FY2009/SAMHSA_Online_appendix.pdf (note: Includes all drug courts) DOJ reported data can be found on http://www.expectmore.gov. Referenced evaluation data can be found at http://www.ojp.usdoj.gov/BJA/evaluation/psi_courts/drug6.htm#outcome

LARGE EXTENT 13%
4.5

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: SAMHSA is funding an independent evaluator to conduct a cross-site study of 2008 grantees to assess the outcomes of the program, but it is unclear when the findings of the evaluation will be available. The cross-site evaluation will measure the effectiveness of the drug courts and follow-up with clients after they have graduated from the program. Although a program level evaluation has not been conducted, SAMHSA mandates that grantees must individually implement evaluation plans to shape improvement and comply with program objectives. Additionally, there are some performance evaluations from SAMHSA grantees available that report mostly positive results. For example, an evaluation completed for the Madison County Adult Drug Court in Anderson, Indiana showed that the graduates of the program had substantial reductions in drug and alcohol use, and involvement in the criminal justice system. However, the evaluation showed only 33% percent of participants graduated from the program It also showed that education levels, employment, housing, and legitimate income increased. Another evaluation completed for an adult drug court in Falmouth, Massachusetts showed that the program was successful in reducing recidivism and improving treatment outcomes in drug court participants. They significantly reduced alcohol and other drug use and arrests and showed improvements in employment, personal relationships, and addressing financial issues. The University of California San Francisco conducted an evaluation of the San Mateo County Juvenile Drug Court G.I.R.L.S project. The project was designed for the specific needs of high-risk multi-recidivist adolescent girls with significant substance abuse and mental health problems. The outcomes showed statistically significant decreases in severity of both substance abuse and legal problems. Essex County Juvenile Drug Court carried out an evaluation that showed participants achieved substantial reductions in using alcohol and other drugs. The drug court also had a positive impact on the social functioning of the participants, specifically on the family and school. These evaluations provide some evidence of Program success; however, because each has some limitations in scope, independence and or quality a more comprehensive evaluation is required to support a yes to this question.

Evidence: No comprehensive Program evaluations are available, but there are some independent evaluations of smaller scope that report mostly positive outcomes, these include: 1. Falmouth Massachusetts Evaluation; 2. Madison County Indiana Evaluation; 3. San Mateo County California Evaluation; and 4. Essex County Massachusetts Evaluation.

SMALL EXTENT 7%
Section 4 - Program Results/Accountability Score 73%


Last updated: 09062008.2008SPR