What Works with Youthful Offenders: The Characteristics of Effective Programs and the Barriers to Effective Implementation

Transcription

1 What Works with Youthful Offenders: The Characteristics of Effective Programs and the Barriers to Effective Implementation Presented by: Edward J. Latessa, Ph.D. Professor & Director School of Criminal Justice University of Cincinnati

2 Several large studies we have conducted have helped us identify characteristics of effective programs 45,000 offenders (adult and juvenile) 450 programs (community, residential, & institutional)

3 Program Leadership and Development Program Leader Qualified Experienced Involved in selecting and training staff Conducts regular supervision with staff Balances supervision/oversight responsibilities with involvement in direct work with clients

4 Program Development Program based on review of research Interventions are piloted Program is valued by the community Funding is adequate/stable Program is well-established If coed program is designed so that interventions for men and women are separate

5 Staff Characteristics Qualified and Experienced Selected for skills and values consistent with offender rehabilitation Regular staff meetings Assessed on service delivery Receive clinical supervision Training (initial and ongoing) Staff input into the program Staff support treatment goals Ethical guidelines in place

6 Offender Assessment Appropriate clients selected based upon clear inclusion/exclusionary criteria Actuarial tool used to assess: Risk Factors Criminogenic need factors Responsivity factors Tool validated on similar population Program targets higher risk offenders (at least 70%)

7 Program Design Treatment Characteristics Treatment 3 to 9 months in length Dosage sufficient and matches risk level Offenders matched to treatment/staff based on needs and responsivity issues Family training incorporated Quality aftercare incorporated Group size does not exceed 10/1 ratio Clear completion criteria Completion rate 65-85%

8 Treatment Characteristics Program Delivery Criminogenic density 75% Use effective treatment model CBT Treatment manuals developed and followed Types and process for reinforcement appropriate Types and process for sanctioning behavior appropriate Skills training incorporated Skill modeled by staff Skill practiced by clients Feedback provided Graduated practice

9 Quality Assurance/Evaluation Quality Assurance Observation of treatment with feedback Satisfaction survey File review process External QA Reassessment on target needs/behaviors Recidivism tracked Program undergone outcome evaluation Evaluator working with/in program

10 Program Integrity and Recidivism The more of the attributes the program had the greater the reduction in recidivism

11 Impact of Program Factors Predicting Felony Adjudication for Juvenile Programs 70 Predicted Recidivism Rates Low Moderate High Very High Program Score 0 Program Score 12 Program Score 24

12 Program Integrity Relationship Between Program Integrity Score & Treatment Effects for Residential Programs Reduced Recidivism Increased Recidivism

13 Program Integrity Relationship Between Program Integrity Score And Treatment Effects for Community Supervision Programs 0.2 Reduced Recidivism r-value Increased Recidivism % 20-39% 40-59% 60+% Program Percentage Score

14 Unfortunately, our data also indicates that the vast majority of correctional programs do not score well on program integrity

15 Results from over 500 Correctional Program Assessments* Ineffective Needs Improvement Effective Very Effective * Includes a wide range of correctional programs using the CPAI or the CPC

16 Some Common Perceptions about Program Fidelity and Quality Assurance 1. It is no more important than selecting and implementing the right EBP's program/curricula 2. It is complicated and require specialized staff and instrumentation 3. It is difficult to do because of time/staff/costs 4. It is as simple as pulling case files and notes to check for completeness 5. Not really necessary because our staff has already been trained

17 It is important to remember that we are taking about EVIDENCE BASED PROGRAMS A program may in fact be operating as designed, but not effectively. For example, a drug education program may be doing a great job of teaching offenders about the harm that drugs can do to them, without being effective in reducing drug usage.

18 There is considerable evidence that fidelity matters We consistently see that doing things well makes a big difference

19 Effects of Quality Programs Delivery for Evidenced Based Programs for Youth Offenders Reduced Recidivism Increased Recidivism Functional Family Therapy Aggression Replacement Therapy Competently Delivered Not Competent Source: Outcome Evaluation of Washington State's Research-Based Programs for Juvenile Offenders. January Washington State Institute for Public Policy.

20 Therapist Competency Ratings and Recidivism Month Felony Recidivism Not Competent Marginal Competent Highly Competent Control Group Source: Outcome Evaluation of Washington State's Research-Based Programs for Juvenile Offenders. January Washington State Institute for Public Policy.

21 Recidivism Results from Ohio EPICS Study looking at Fidelity and High Risk Offenders Latessa, E., Smith, P., Schweitzer, m., and Labrecque, R. (2013). Evaluation of the Effective Practices in Community Supervision Model (EPICS) in Ohio. School of Criminal Justice, University of Cincinnati.

22 Predictive Validity of the LSI-R and Assessor Trainin r value Trained Not Trained Flores, T., C Lowenkamp, A. Holsinger, and E.Latessa (2006). Predicting Outcome with the Level of Service Inventory-Revised: The Importance of Implementation Integrity. Journal of Criminal Just (4).

23 Percent Change in Recidivism Effect of Program Integrity on Recidivism: Results from Meta Analysis Andrews and Dowden Trained Workers Specific Model Printed Manuals Supervised Workers Adequate Dosage Monitor Change Involved Researcher

24 Elements of Effective Correctional Practice and Recidivism No Yes Effect Size Effective Authority Problem Solving Structuring Skill Learning Effective Disapproval Effective Modeling Effective Reinforcement Structuring Skills Relationship Skills Source: Gendreau, P. (2003). Invited Address, Division 18, APA Annual Convention, Toronto, CA.

25 % Change in Recidivism Results from Study of Adult Offenders: Cognitive-behavioral Groups = Cog group offered but not well done. 0= Not offered. 1= Group offered and offered at least 4 or more hours per week and at least 50% of time in role playing.

26 Domestic Violence Group Scale % Change in Recidivism =DV offered but no positive attributes. 0=No DV group. 1=Group with one or two positive attributes. 2=Group offered with 3 positive attributes. Attributes 1) at least 10 sessions, i2) incorporated role play, 3) staff trained on curriculum

27 Quality Assurance 1. Internal processes to ensure that assessments, services & interventions provided by the program are delivered as designed 2. External processes to ensure services and interventions provided by outside providers are delivered as designed Can include: On-going coaching of staff Group observation with feedback Case file audits Video taping groups Client satisfaction surveys/ exit interviews Clinical supervision Program audits Site visits and observation Certification process

28 Need to make Quality Assurance part of job Consider creating some standing committees to monitor and assist with QA across different areas: Assessment and case planning Treatment and Supervision practices Training and Coaching Performance and Evaluation

29 Strategies can include: Assessment Are they scored properly Test for inter-rater reliability Are case plans based on assessments Certification and re-certification of staff Booster training sessions

30 Strategies can include: Treatment and Supervision Require treatment programs do develop detailed program descriptions and then conduct periodic site visits Group/Officer offender observation with feedback Exit interviews with youth and familiy Booster sessions for coaches and staff

31 Strategies can include: Training and Coaching Schedule short in-service training sessions as part of staff meetings Booster sessions Evaluate all training sessions Evaluate all coaches

32 Strategies can include: Performance Measures and Evaluation Develop some performance indicators for offenders i.e. reductions in TVs, positive drug tests, etc. Develop some performance indicators for QA: i.e. 90% inter-rater reliability of all items on LSI Satisfactory or higher ratings on EPICS indicators

33 Effectiveness in Ohio programs associated with Frequency of group observation (3 or more times/year) Staff evaluation Receptive to supervision, ability to handle crisis situations, avoid negative interactions with clients and empathetic Internal and External audits

34 Quality Assurance/Evaluation % Change in Recidivism EVALUATION DOMAIN CATEGORIES

35 Some things get in the way

36 Lack of Strong Leadership Give change lip service, but do not really want to change approach No identified overseer of changes or unstable leadership Want to pick and choose elements Reluctant to challenge staff and hold people accountable Too busy to get involved in the treatment Don t understand the model Failure to develop detailed program description and program manual Failure to develop QA processes

37 Mid level supervisors Don t have the skills or competencies Failure to ensure staff are implementing as designed Are not part of the decision making Afraid to challenge staff or hold accountable Feel threatened (fear staff will know more than they do) Us against them mentality (aligned w/ staff to resist change)

38 Staff Natural resistance to change Are true believers in existing model (often because it worked for them ) Don t have the skills or competencies Requires work (I wasn t hired for this) Are not challenged or held accountable by supervisors Turnover

39 Assessment Do not take assessment seriously Ignore responsivity factors Assess offenders then don t use it everyone gets the same treatment Do not develop and/or update treatment plans Do not reassess Make errors and don t correct Do not adequately train staff in use or interpretation Assessment instruments are not validated

40 Treatment Want to cling to the old model Do just enough to pass the test (Learn the words but not the music) Use the parts they like or are easy Resist changes to the treatment schedule (particularly when the dosage is increased) Quick to conclude that changes to treatment aren t working (if a participant fails or treatment component requires modification) Lack of consistency in high fidelity program delivery

41 Lack of On-Going Quality Assurance Drift from model Introduction of new program elements that are not always consistent with the model Failure to have QA processes (including ongoing observation of service delivery with feedback) Failure to develop performance measures Failure to collect and analyze data Failure to respond to findings from QA processes

42 Other barriers Outside influences (i.e. political, financial, personal) Don t select staff based on values and skills Failure to develop internal capacity

43 What Else have I learned: Suggestions for overcoming some of the barriers? Need to find leaders to serve as champions Different levels are important Supervisors are the key to successful implementation Need additional training and to be made part of the process All staff need to be trained, but training alone isn t enough Need to provide on-going coaching and support as well as booster sessions

44 Overcoming barriers Need to distinguish between activities & core correctional programming Helps focus on criminogenic targets for change Reduces program drift Treatment needs need to be manualized Provides structure and easier to replicate Improves quality assurance Improves consistency

45 Overcoming Barriers Data makes a difference Get an evaluator involved (on staff, consultant, or local University) Professionally trained and Supervisory staff need to be more involved Help select curriculum Help train Serve as coaches / QA reviews Changing is Difficult Take it one program at a time

46 Remember Quality matters It can be measured It can be improved

47 Thank you