QA/CQI Symposium: Making Continuous Quality Improvement a Reality

Transcription

1 QA/CQI Symposium: Making Continuous Quality Improvement a Reality March 17, 2017 University of Cincinnati Corrections Institute (UCCI) Copyright 2017, University of Cincinnati, Corrections Institute, Ohio. All rights reserved. Slide Number 1 Session Objectives Discuss the importance of treatment fidelity within a CQI process Review examples of fidelity measures through group, individual interventions, risk assessments and documentation Examine the role of providing effective feedback to staff during the coaching process Copyright 2017, University of Cincinnati, Corrections Institute, Ohio. All rights reserved. Slide Number 2

2 CQI in Other Industries Grocery stores or department stores use secret shoppers How they are greeted, demeanor and helpfulness of staff, condition of the store, displays, etc. Overall scoring of customer service Provide direct feedback to the corporate office on the scores Recommend steps to improve operations Copyright 2017, University of Cincinnati, Corrections Institute, Ohio. All rights reserved. Slide Number 3 Section 1 Is CQI in Corrections Different? Copyright 2017, University of Cincinnati, Corrections Institute, Ohio. All rights reserved. Slide Number 4

3 First, What Do We Know? What Works Literature Principles of Effective Interventions Risk, Need, Responsivity Growing evidence based on individual program evaluations and meta-analyses CPC/CPAI Research We know that key organizational factors impact a program or agency s ability to reduce recidivism Copyright 2017, University of Cincinnati, Corrections Institute, Ohio. All rights reserved. Slide Number 5 Principles of Effective Interventions RISK NEED RESPONSIVITY FIDELITY WHO WHAT HOW HOW WELL Deliver more intense intervention to higher risk offenders Target criminogenic needs to reduce risk for recidivism Use CBT approaches Match mode/style of service to offender Deliver treatment services as designed Copyright 2017, University of Cincinnati, Corrections Institute, Ohio. All rights reserved. Slide Number 6

4 Major Set of Risk/Need Factors Antisocial Attitudes Antisocial Personality Antisocial Peers History of Antisocial Behavior Family Education / Employment Substance Abuse Prosocial Leisure Activities Copyright 2017, University of Cincinnati, Corrections Institute, Ohio. All rights reserved. Slide Number 7 Taking Stock of the Principles of Effective Interventions There are more than 40 published metaanalyses of the correctional treatment literature Results have been replicated with remarkable consistency Considerable support for the RNR framework across quantitative reviews of the literature Smith et al. (2009). Validating the Principles of Effective Intervention: A Systematic Review of the Contributions of Meta- Analysis in the Field of Corrections. Victims and Offenders: An International Journal of Evidence-Based Research, Policy, and Practice, 2, Copyright 2017, University of Cincinnati, Corrections Institute, Ohio. All rights reserved. Slide Number 8

5 25 RNR and Reductions in Recidivism: General Recidivism Change in Recidivism Rates Risk Need Responsivity -5 Yes No Andrews, Enhancing adherence to risk-need-responsivity: Making quality a matter of policy. Criminology and Public Policy, 5, Copyright 2017, University of Cincinnati, Corrections Institute, Ohio. All rights reserved. Slide Number 9 Principles of Effective Intervention Risk (Who) Need (What) Responsivity (How) Fidelity (How well) HOW WELL programs incorporate risk, need and responsivity Measures how programs are implemented and overall adherence to their respective modality of treatment Can be measured and monitored during group facilitation, individual sessions and in case planning Copyright 2017, University of Cincinnati, Corrections Institute, Ohio. All rights reserved. Slide Number 10

6 The Fidelity Principle Effective programs utilize CQI to help ensure program fidelity which is linked to successful program outcomes Poor fidelity can lead to null effects or adverse effects Impacts on recidivism are correlated with the quality of implementation Fidelity cannot be assumed Fidelity can be measured and monitored Lipsey MW, Landenberger NA, Wilson SJ. Effects of cognitive-behavioral programs for criminal offenders. Campbell Systematic Reviews. 2007:6. Copyright 2017, University of Cincinnati, Corrections Institute, Ohio. All rights reserved. Slide Number 11 Section 2 Why is CQI Important? Copyright 2017, University of Cincinnati, Corrections Institute, Ohio. All rights reserved. Slide Number 12

7 (CQI) What Is It Again? A method of continuously examining processes and making them better Key principles: Use of data and team approaches to improve decision making Involvement of entire organization to improve quality Continuous improvement of all processes and outcomes Copyright 2017, University of Cincinnati, Corrections Institute, Ohio. All rights reserved. Slide Number 13 Audit Process Emphasis on monitoring and compliance Necessary and helpful Can create undesired effect on staff behaviors Work towards gaining acceptable scores rather than mastering skills Target = Individual performance Focus on compliance QA Verses CQI Prospective Process Emphasis on data driven decisions Not blame-seeking Involves trust, respect, and communication Move toward staff responsibility for quality, problem solving and ownership of services Target = System and Processes Focus on attaining proficiency or mastery of important skill Copyright 2017, University of Cincinnati, Corrections Institute, Ohio. All rights reserved. Slide Number 14

8 CQI Focuses on Sources and Causes Identification of risk points and their contribution to the problem Identify changes in these processes that reduce risk of re-occurrence Copyright 2017, University of Cincinnati, Corrections Institute, Ohio. All rights reserved. Slide Number 15 Benefits of CQI Proof of effective services -Maintain or secure funding -Improve staff morale, confidence, and retention -Educate key stakeholders about services Highlights opportunities for improvement Data to inform quality improvement initiatives Monitor/ensure treatment fidelity Better public safety concerns Engaging in CQI helps organizations to achieve their mission Copyright 2017, University of Cincinnati, Corrections Institute, Ohio. All rights reserved. Slide Number 16

9 The Reality is Many programs are implementing evidencebased practices with little evidence of strong fidelity Correctional agencies have a responsibility to ensure effective services Responsibility for EBP needs to be aligned at all levels administration, management, line staff Copyright 2017, University of Cincinnati, Corrections Institute, Ohio. All rights reserved. Slide Number 17 More Reality Achieving and sustaining quality doesn t just magically happen It requires a deliberate effort on the part of all staff Copyright 2017, University of Cincinnati, Corrections Institute, Ohio. All rights reserved. Slide Number 18

10 Section 3 What Happens When We Don t Measure Our Programs? Copyright 2017, University of Cincinnati, Corrections Institute, Ohio. All rights reserved. Slide Number 19 On Boarding Youth Transitioning to Adulthood/ MH/Addiction My Head Hurts!!!! Sex Offender Programs in Ohio s Prisons CQI Case Planning The Odd Couple of Ethics and CQI Trauma Informed Care in Corrections QA/CQI for Program Sustainability Social Reinforcement to Probation/Parole Measuring and Improving Fidelity to Evidence-Based Practices Dosage Behavior Management System Development and Implementation How to go from As It To Desired/Perfect State Using Data to Transform Community Correction Interventions How I Became a Believer Overcoming Hurdles in the Coaching Process From the End of the Bench to Put Me In, Coach So what does this all mean for my agency? Let s talk how we can bring things together! Copyright 2017, University of Cincinnati, Corrections Institute, Ohio. All rights reserved. Slide Number 20

11 The Big Picture Program Integrity and Treatment Effects Copyright 2017, University of Cincinnati, Corrections Institute, Ohio. All rights reserved. Slide Number 21 Program Integrity And Treatment Effect for Adult Residential Programs 25 Change In Recidivism Rates As Scores for Integrity Rise Recidivism Rates Decrease % 31-59% 60-69% 70%+ Lowenkamp, C. T., & Latessa, E. J. (2002). Evaluation of Ohio s community based correctional facilities and halfway house programs (Technical Report). Cincinnati, OH: University of Cincinnati. Copyright 2017, University of Cincinnati, Corrections Institute, Ohio. All rights reserved. Slide Number 22

12 Program Integrity And Treatment Effect for Adult Non-Residential Programs Change In Recidivism Rates As Scores for Integrity Rise Recidivism Rates Decrease % (9) 20-39% (37) 40-59% (17) 60+% 60%+ (3) Program Percentage Score Lowenkamp, C. T., & Latessa, E. J. (2005a). Evaluation of Ohio s CCA funded programs (Technical Report). Cincinnati, OH: University of Cincinnati. Copyright 2017, University of Cincinnati, Corrections Institute, Ohio. All rights reserved. Slide Number 23 Program Integrity And Treatment Effect for Juvenile Programs 0.1 Average Effect Size As Scores for Integrity Rise Recidivism Rates Decrease -0.1 <60% 60-69% 70%+ From: Lowenkamp, C., Latessa, E., & Lemke, R. (2006). Evaluation of Ohio's RECLAIM Funded Programs, Community Corrections Facilities, and DYS Facilities: FY University of Cincinnati, OH Copyright 2017, University of Cincinnati, Corrections Institute, Ohio. All rights reserved. Slide Number 24

13 Section 4 Committing Your Agency Copyright 2017, University of Cincinnati, Corrections Institute, Ohio. All rights reserved. Slide Number 25 Hard-hitting Agency Questions When Developing CQI Standards? What is your agency s central mission statement? How do you achieve that mission statement? What is the role of EBP and fidelity? How will a CQI process help you accomplish your mission? How do you make data driven decisions? Copyright 2017, University of Cincinnati, Corrections Institute, Ohio. All rights reserved. Slide Number 26

14 Other Considerations What is your current assessment of staff in providing evidence-based practices? What are the challenges in achieving staff proficiency? What changes could facilitate improvement in this supervision? Copyright 2017, University of Cincinnati, Corrections Institute, Ohio. All rights reserved. Slide Number 27 CQI Magic Button Core Concepts in Designing CQI Plans Copyright 2017, University of Cincinnati, Corrections Institute, Ohio. All rights reserved. Slide Number 28

15 Common Definitions Outcomes (An expected result in an individual s behavior, knowledge or skills or the change in practices or policies attained in an activity or program) Indicators (An observable and measureable behavior or finding used to understand information about complex systems) Measures (An instrument, device, or method that provides information on an outcome) Benchmark (a similar measure for a similar group against which progress can be gauged) Target (A desired level of a measure at a specified time in the future) Baseline (the performance level for a measured before the program begins) Framework for assessing progress Malone, N., Mark, L., & Narayan, K. (2014). Understanding program monitoring: The relationships among outcomes, indicators, measures, and targets. (REL ). Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory Pacific. Copyright 2017, University of Cincinnati, Corrections Institute, Ohio. All rights reserved. Slide Number 29 It s A Lot For Just One Person Group Observations Individual Session Observations File Reviews Documentation Reviews Case Note Reviews Assessment Observations Coaching/Supervision Sessions Writing QA reports/making recommendations Checking Grievances/Complaints Satisfaction Surveys Reinforcing Tracking Sanction Tracking Training Observation Monthly Meeting Notes These are just programming considerations! Copyright 2017, University of Cincinnati, Corrections Institute, Ohio. All rights reserved. Slide Number 30

16 Designing a CQI Plan Develop CQI committee/team Decide what to measure Division of CQI team roles and responsibilities Monitoring proficient staff Monitoring staff who are not proficient Change planning Long term CQI considerations Copyright 2017, University of Cincinnati, Corrections Institute, Ohio. All rights reserved. Slide Number 31 Designing a CQI Plan Formalize your plan for CQI What to monitor How to monitor Use objective data to inform decision making Develop reinforcements Develop change planning Consider regular group booster sessions to address agency deficiencies Consider sustainability (policies/procedures) Copyright 2017, University of Cincinnati, Corrections Institute, Ohio. All rights reserved. Slide Number 32

17 Section 5 What Can We Measure? Copyright 2017, University of Cincinnati, Corrections Institute, Ohio. All rights reserved. Slide Number 33 Where CQI Should Exist in Other Life Areas! At 15 or 16 you go to driver s education and get your license = Driving with fidelity Add #SelfieTime to your ride = #UhOh Are you the master driver you once were? Copyright 2017, University of Cincinnati, Corrections Institute, Ohio. All rights reserved. Slide Number 34

18 Common Areas to Monitor Copyright 2017, University of Cincinnati, Corrections Institute, Ohio. All rights reserved. Slide Number 35 Common Areas to Monitor Risk Assessments Instrument completion Case Management Plan Assessment Interview Inter-rater reliability Individual Interventions Targets criminogenic needs Core Correctional Practices Makes good use of time Addresses responsivity Group Observations Frequency of role plays Follows curriculum structure Behavior management components Quality Documentation Clinical implications Operational implications Risk management Reflection of providers Copyright 2017, University of Cincinnati, Corrections Institute, Ohio. All rights reserved. Slide Number 36

19 Offender Change and Other Indicators Copyright 2017, University of Cincinnati, Corrections Institute, Ohio. All rights reserved. Slide Number 37 Offender Change and Other Indicators Offender Change Client Satisfaction Surveys Dosage and Structured Time Treatment Progress Employee Satisfaction Pay/benefits Adequate resources Communication Training Satisfaction Client Satisfaction Access to services Involvement in treatment planning Safety System/Organization Incidents Complaints and grievances Quality of external services Contracted services Copyright 2017, University of Cincinnati, Corrections Institute, Ohio. All rights reserved. Slide Number 38

20 Section 6 Supervising and Coaching Copyright 2017, University of Cincinnati, Corrections Institute, Ohio. All rights reserved. Slide Number 39 Transitioning from Quality Assurance to Quality Improvement Quality Assurance Systemic process of checking to see whether a service is meeting specified requirements Emphasis on auditing for compliance with set standards Continuous Quality Improvement Formal approach to the analysis of performance and efforts to improve it Emphasis on continuously improving processes to meet standards Supervision and coaching assists in attaining proficiency Copyright 2017, University of Cincinnati, Corrections Institute, Ohio. All rights reserved. Slide Number 40

21 Supervision What Is It? Intervention provided by a proficient staff member Has the purposes of: Enhancing professional functioning of the staff person Monitoring the quality of professional services offered to participants Copyright 2017, University of Cincinnati, Corrections Institute, Ohio. All rights reserved. Slide Number 41 Process of Supervision 1. Direct observation of sessions 2. Structured feedback about adherence and competence 3. Coaching to improve implementation with proficiency 4. Continue evaluation with frequency until reach proficiency Copyright 2017, University of Cincinnati, Corrections Institute, Ohio. All rights reserved. Slide Number 42

22 Supervision Strategies Demonstration/modeling Role-playing Corrective feedback and coaching Reinforcement Copyright 2017, University of Cincinnati, Corrections Institute, Ohio. All rights reserved. Slide Number 43 Organizational Requirements for Effective Supervision Supervisors trained and competent in evidence-based practices Supervisors trained on how to supervise and coach staff Infrastructure that allows for (and requires) supervision Supervisors given time to supervise Supervisors held accountable for supervision Copyright 2017, University of Cincinnati, Corrections Institute, Ohio. All rights reserved. Slide Number 44

23 Feedback and Coaching Copyright 2017, University of Cincinnati, Corrections Institute, Ohio. All rights reserved. Slide Number 45 Without Observation No chance to reinforce good work No correcting mistakes or inconsistencies No provision for gaps in skill No assurance of fidelity with model Participant problem behaviors increase Copyright 2017, University of Cincinnati, Corrections Institute, Ohio. All rights reserved. Slide Number 46

24 Coaching Video Clip 1: Kelly Provides Reinforcement to Jaime Copyright 2017, University of Cincinnati, Corrections Institute, Ohio. All rights reserved. Slide Number 47 Coaching Video Clip 2: Kelly Provides Constructive Feedback to Jaime Copyright 2017, University of Cincinnati, Corrections Institute, Ohio. All rights reserved. Slide Number 48

25 Benefits of Peer Coaching Peer coaching can be as effective as traditional supervision Encourages camaraderie and working relations Positively related to desired behavior change Some evidence that it improves retention Sparks, G., & Bruder, S. (1987). Before and after peer coaching. Educational Leadership, 3, Copyright 2017, University of Cincinnati, Corrections Institute, Ohio. All rights reserved. Slide Number 49 Overcoming Resistance Administration must walk the walk Ensure early successes to increase buy-in Recognition of staff for using the process Openly acknowledge the extra work required Demonstrate front-end planning to minimize workload issues Copyright 2017, University of Cincinnati, Corrections Institute, Ohio. All rights reserved. Slide Number 50

26 Potential Strategies Start small For example, desk top review of assessments versus observation-based ratings Use technology to increase efficiencies For example, videotape interactions for observation-based ratings Take the time to build expertise Train on evaluation methodology Insure understanding of purpose (e.g., QI versus punishment) Copyright 2017, University of Cincinnati, Corrections Institute, Ohio. All rights reserved. Slide Number 51 We Can t Forget Our Successes! If results are not measured, successes cannot be distinguished from failures If successes cannot be distinguished, they cannot be replicated If failures cannot be identified, they cannot be corrected If results cannot be demonstrated, support cannot be secured Excerpt from Burrell, William D. (2017). Risk and Needs Assessment in Probation and Parole: The Persistent Gap Between Promise and Practice. In Taxman, Faye S. Risk and Need Assessment: Theory and Practice. New York: Taylor and Francis. Copyright 2017, University of Cincinnati, Corrections Institute, Ohio. All rights reserved. Slide Number 52

27 Competent Staff Better Quality Implementation Re-Offending We can decrease re-offending behavior or We can make them worse! Less Competent Staff Poorer Quality Implementation Re-Offending Copyright 2017, University of Cincinnati, Corrections Institute, Ohio. All rights reserved. Slide Number 53 Thank you for your participation in this course! Produced by the University of Cincinnati Corrections Institute Copyright 2017, University of Cincinnati, Corrections Institute, Ohio. All rights reserved. Slide Number 54

28 Eric Willoughby Jr. Research Associate Copyright 2017, University of Cincinnati, Corrections Institute, Ohio. All rights reserved. Slide Number 55