What Works with Youthful Offenders: The Characteristics of Effective Programs and the Barriers to Effective Implementation Presented by: Edward J. Latessa, Ph.D. Professor & Director School of Criminal Justice University of Cincinnati E-mail: Edward.Latessa@UC.Edu
Several large studies we have conducted have helped us identify characteristics of effective programs 45,000 offenders (adult and juvenile) 450 programs (community, residential, & institutional)
Program Leadership and Development Program Leader Qualified Experienced Involved in selecting and training staff Conducts regular supervision with staff Balances supervision/oversight responsibilities with involvement in direct work with clients
Program Development Program based on review of research Interventions are piloted Program is valued by the community Funding is adequate/stable Program is well-established If coed program is designed so that interventions for men and women are separate
Staff Characteristics Qualified and Experienced Selected for skills and values consistent with offender rehabilitation Regular staff meetings Assessed on service delivery Receive clinical supervision Training (initial and ongoing) Staff input into the program Staff support treatment goals Ethical guidelines in place
Offender Assessment Appropriate clients selected based upon clear inclusion/exclusionary criteria Actuarial tool used to assess: Risk Factors Criminogenic need factors Responsivity factors Tool validated on similar population Program targets higher risk offenders (at least 70%)
Program Design Treatment Characteristics Treatment 3 to 9 months in length Dosage sufficient and matches risk level Offenders matched to treatment/staff based on needs and responsivity issues Family training incorporated Quality aftercare incorporated Group size does not exceed 10/1 ratio Clear completion criteria Completion rate 65-85%
Treatment Characteristics Program Delivery Criminogenic density 75% Use effective treatment model CBT Treatment manuals developed and followed Types and process for reinforcement appropriate Types and process for sanctioning behavior appropriate Skills training incorporated Skill modeled by staff Skill practiced by clients Feedback provided Graduated practice
Quality Assurance/Evaluation Quality Assurance Observation of treatment with feedback Satisfaction survey File review process External QA Reassessment on target needs/behaviors Recidivism tracked Program undergone outcome evaluation Evaluator working with/in program
Program Integrity and Recidivism The more of the attributes the program had the greater the reduction in recidivism
Impact of Program Factors Predicting Felony Adjudication for Juvenile Programs 70 Predicted Recidivism Rates 60 50 40 30 20 10 11 8 7 27 22 18 43 36 31 60 53 47 0 Low Moderate High Very High Program Score 0 Program Score 12 Program Score 24
Program Integrity Relationship Between Program Integrity Score & Treatment Effects for Residential Programs Reduced Recidivism Increased Recidivism 0-30 31-59 60-69 70+
Program Integrity Relationship Between Program Integrity Score And Treatment Effects for Community Supervision Programs 0.2 Reduced Recidivism 0.15 0.1 0.16 r-value 0.05 0 0.02 0.12 Increased Recidivism -0.05-0.1-0.15-0.15 0-19% 20-39% 40-59% 60+% Program Percentage Score
Unfortunately, our data also indicates that the vast majority of correctional programs do not score well on program integrity
Results from over 500 Correctional Program Assessments* 50 45 40 35 30 25 20 15 10 5 0 Ineffective Needs Improvement Effective Very Effective * Includes a wide range of correctional programs using the CPAI or the CPC
Some Common Perceptions about Program Fidelity and Quality Assurance 1. It is no more important than selecting and implementing the right EBP's program/curricula 2. It is complicated and require specialized staff and instrumentation 3. It is difficult to do because of time/staff/costs 4. It is as simple as pulling case files and notes to check for completeness 5. Not really necessary because our staff has already been trained
It is important to remember that we are taking about EVIDENCE BASED PROGRAMS A program may in fact be operating as designed, but not effectively. For example, a drug education program may be doing a great job of teaching offenders about the harm that drugs can do to them, without being effective in reducing drug usage.
There is considerable evidence that fidelity matters We consistently see that doing things well makes a big difference
Effects of Quality Programs Delivery for Evidenced Based Programs for Youth Offenders 40 30 Reduced Recidivism 20 10 0 Increased Recidivism -10-20 Functional Family Therapy Aggression Replacement Therapy Competently Delivered 38 24 Not Competent -16.7-10.4 Source: Outcome Evaluation of Washington State's Research-Based Programs for Juvenile Offenders. January 2004. Washington State Institute for Public Policy.
Therapist Competency Ratings and Recidivism 35 30 29 12 Month Felony Recidivism 25 25 22 20 15 17 14 10 5 0 Not Competent Marginal Competent Highly Competent Control Group Source: Outcome Evaluation of Washington State's Research-Based Programs for Juvenile Offenders. January 2004. Washington State Institute for Public Policy.
Recidivism Results from Ohio EPICS Study looking at Fidelity and High Risk Offenders Latessa, E., Smith, P., Schweitzer, m., and Labrecque, R. (2013). Evaluation of the Effective Practices in Community Supervision Model (EPICS) in Ohio. School of Criminal Justice, University of Cincinnati.
Predictive Validity of the LSI-R and Assessor Trainin 0.3 0.2 r value 0.1 0 Trained Not Trained Flores, T., C Lowenkamp, A. Holsinger, and E.Latessa (2006). Predicting Outcome with the Level of Service Inventory-Revised: The Importance of Implementation Integrity. Journal of Criminal Just (4).
Percent Change in Recidivism Effect of Program Integrity on Recidivism: Results from Meta Analysis Andrews and Dowden 1999 50 45 40 33 30 28 22 23 22 20 20 12 10 10 9 7 7 8 5 0 Trained Workers Specific Model Printed Manuals Supervised Workers Adequate Dosage Monitor Change Involved Researcher
Elements of Effective Correctional Practice and Recidivism 0.5 0.4 0.3 No Yes Effect Size 0.2 0.1 0 Effective Authority Problem Solving Structuring Skill Learning Effective Disapproval Effective Modeling Effective Reinforcement Structuring Skills Relationship Skills Source: Gendreau, P. (2003). Invited Address, Division 18, APA Annual Convention, Toronto, CA.
% Change in Recidivism 7 6 5 4 3 2 1 0-1 Results from Study of Adult Offenders: Cognitive-behavioral Groups 4.8-1 0 1 6.3-2 -1.4-1= Cog group offered but not well done. 0= Not offered. 1= Group offered and offered at least 4 or more hours per week and at least 50% of time in role playing.
Domestic Violence Group Scale % Change in Recidivism 8 7 6 5 4 3 2 1 0-1 -2-1.3 6.9 4.6 3.9-1 0 1 2-1=DV offered but no positive attributes. 0=No DV group. 1=Group with one or two positive attributes. 2=Group offered with 3 positive attributes. Attributes 1) at least 10 sessions, i2) incorporated role play, 3) staff trained on curriculum
Quality Assurance 1. Internal processes to ensure that assessments, services & interventions provided by the program are delivered as designed 2. External processes to ensure services and interventions provided by outside providers are delivered as designed Can include: On-going coaching of staff Group observation with feedback Case file audits Video taping groups Client satisfaction surveys/ exit interviews Clinical supervision Program audits Site visits and observation Certification process
Need to make Quality Assurance part of job Consider creating some standing committees to monitor and assist with QA across different areas: Assessment and case planning Treatment and Supervision practices Training and Coaching Performance and Evaluation
Strategies can include: Assessment Are they scored properly Test for inter-rater reliability Are case plans based on assessments Certification and re-certification of staff Booster training sessions
Strategies can include: Treatment and Supervision Require treatment programs do develop detailed program descriptions and then conduct periodic site visits Group/Officer offender observation with feedback Exit interviews with youth and familiy Booster sessions for coaches and staff
Strategies can include: Training and Coaching Schedule short in-service training sessions as part of staff meetings Booster sessions Evaluate all training sessions Evaluate all coaches
Strategies can include: Performance Measures and Evaluation Develop some performance indicators for offenders i.e. reductions in TVs, positive drug tests, etc. Develop some performance indicators for QA: i.e. 90% inter-rater reliability of all items on LSI Satisfactory or higher ratings on EPICS indicators
Effectiveness in Ohio programs associated with Frequency of group observation (3 or more times/year) Staff evaluation Receptive to supervision, ability to handle crisis situations, avoid negative interactions with clients and empathetic Internal and External audits
Quality Assurance/Evaluation 9 8 8.1 7 % Change in Recidivism 6 5 4 3 2 5.2 1.2 1 0 0-1 2 3 EVALUATION DOMAIN CATEGORIES
Some things get in the way
Lack of Strong Leadership Give change lip service, but do not really want to change approach No identified overseer of changes or unstable leadership Want to pick and choose elements Reluctant to challenge staff and hold people accountable Too busy to get involved in the treatment Don t understand the model Failure to develop detailed program description and program manual Failure to develop QA processes
Mid level supervisors Don t have the skills or competencies Failure to ensure staff are implementing as designed Are not part of the decision making Afraid to challenge staff or hold accountable Feel threatened (fear staff will know more than they do) Us against them mentality (aligned w/ staff to resist change)
Staff Natural resistance to change Are true believers in existing model (often because it worked for them ) Don t have the skills or competencies Requires work (I wasn t hired for this) Are not challenged or held accountable by supervisors Turnover
Assessment Do not take assessment seriously Ignore responsivity factors Assess offenders then don t use it everyone gets the same treatment Do not develop and/or update treatment plans Do not reassess Make errors and don t correct Do not adequately train staff in use or interpretation Assessment instruments are not validated
Treatment Want to cling to the old model Do just enough to pass the test (Learn the words but not the music) Use the parts they like or are easy Resist changes to the treatment schedule (particularly when the dosage is increased) Quick to conclude that changes to treatment aren t working (if a participant fails or treatment component requires modification) Lack of consistency in high fidelity program delivery
Lack of On-Going Quality Assurance Drift from model Introduction of new program elements that are not always consistent with the model Failure to have QA processes (including ongoing observation of service delivery with feedback) Failure to develop performance measures Failure to collect and analyze data Failure to respond to findings from QA processes
Other barriers Outside influences (i.e. political, financial, personal) Don t select staff based on values and skills Failure to develop internal capacity
What Else have I learned: Suggestions for overcoming some of the barriers? Need to find leaders to serve as champions Different levels are important Supervisors are the key to successful implementation Need additional training and to be made part of the process All staff need to be trained, but training alone isn t enough Need to provide on-going coaching and support as well as booster sessions
Overcoming barriers Need to distinguish between activities & core correctional programming Helps focus on criminogenic targets for change Reduces program drift Treatment needs need to be manualized Provides structure and easier to replicate Improves quality assurance Improves consistency
Overcoming Barriers Data makes a difference Get an evaluator involved (on staff, consultant, or local University) Professionally trained and Supervisory staff need to be more involved Help select curriculum Help train Serve as coaches / QA reviews Changing is Difficult Take it one program at a time
Remember Quality matters It can be measured It can be improved
Thank you