Planning & Program Implementation National Replication 9/30/2015
GOALS FOR THE PRESENTATION Introduction of a framework to use when considering replication Share web site resources Share printed questionnaires to complete when doing your assessment Introduce various work products to support replication EBAssests EBAnalytics EBAccountability Discuss success and challenges when implementing Considerations when implementing
What Type of Cook are You? Do You Follow the Recipe? Do You Improvise or Substitute? Do You Need Pictures? Do You Measure?
Website Resources 1. NREPP (NATIONAL REGISTRY OF EVIDENCE BASED PROGRAMS AND PRACTICES) ex. IDENTIFIED PROBLEM IN COMMUNITY IS: DELINQUENCY AND CRIME AMONG YOUNG ADULTS http://www.nrepp.samhsa.gov/ > enter crime in the Find An Intervention box> 18 results, then narrow search by Age {select 18 25 }; by Outcome Categories {select Crime/Delinquency}; by Geographic Locations {select Urban & Rural}, now narrowed to 2 models to research. 2. THE CALIFORNIA EVIDENCE BASED CLEARING FOR CHILD WELFARE ex. IDENTIFIED PROBLEM IN COMMUNITY IS: LACK OF SUPPORT FOR PARENTS http://www.cebc4cw.org/ > Select & Implement Programs>Topic Areas>Alpha Listing yields 5 top rated models to research. 3. BLUEPRINTS FOR HEALTHY YOUTH DEVELOPMENT ex. IDENTIFIED PROBLEM IN COMMUNITY IS: BULLYING http://www.blueprintsprograms.com/ > Program Search>Search All Criteria at Once>Problem Behavior>now narrow search by Program Outcomes {select Bullying }; and by Target Population {select Late Childhood & Early Adolescence}, now narrowed to 3 models to research. 4. SAMHSA (SUBSTANCE ABUSE AND MENTAL HEALTH SERVICES ADMINISTRATION) ex. IDENTIFIED PROBLEM IN COMMUNITY IS: SUBSTANCE ABUSE http://www.samhsa.gov/ >Programs & Campaigns>EBP Webguide>Mental Health Treatment>Effective Child Therapy:Evidence based Mental Health Treatment for Children & Adolescents>>Effective Child Therapy>Professionals and Educators>About Specific Treatments>Specific Evidence based Treatment Programs>Substance Abuse yields 6 programs, 5 with Internet links.
WHAT OTHER RESOURCES ARE THERE Identifying and Selecting Evidence Based Programs and Practices: Questions To Consider Outcome Identification EBP Identification Evidence for Effectiveness Organizational and Community Fit Capacity and Resources Monitoring and Sustainability
Research Design: At least 1 rigorous study 2 ; with positive findings Replication: Replicated with positive findings ; or Sustainability: Outcomes are sustained for at least a 1 year Research Design: Pre Post study or Post study; with positive findings Replication: Not Replicated; and Sustainability: Outcomes are not sustained for 1 year Exemplary (Evidence Based) Effective Promising Emerging Research Design: At least 1 RCT 1 plus another rigorous study 2 ; both with positive findings Replication: Replicated with positive findings ; and Sustainability: Outcomes are sustained for at least a 1 year Research Design: At least 1 rigorous study 2 ; with positive findings Replication: Not Replicated; and Sustainability: Outcomes are not sustained for 1 year Innovative Practice Based in theory, collecting quality data, has logic model, and monitoring fidelity with positive findings. Cannot Rate k 1 RCT Randomized Controlled Trial 2 Rigorous Study Randomized Controlled Trial or Quasi Experimental Study
Questions To Ask as You Explore the Possible Use of an Intervention Implementations Adaptations Staffing Quality Assurance Mechanisms Training and Technical Assistance Costs
Assessment Organizational and System Readiness Implementation
Assessment Organizational and System Readiness Implementation Administrative Oversight Service Provider Oversight Referral/Utilization Management Stakeholder Management Model Fidelity Promotion Data Collection and Evaluation Funding and Sustainability Guidance
Assessment (Month 1) Administrative Oversight Service Provider Oversight Referral/Utilization Management Stakeholder Management Model Fidelity Promotion Data Collection and Evaluation Funding & Sustainability Guidance SWOT Analysis How will the Project be Managed Understand how the Target Population will get linked to the service Procure for the service delivery requesting provider proposals Develop organizational readiness checklist for the selected provider Identify current referral stream for selected provider and cross check for target population Identify outcome targets for # served, length of treatment & utilization Meet with primary referral source and share referral process Identity each key stakeholder and determine support for the service while assess knowledge of the evidence based intervention Meet with selected service provider and assess experience and capacity with EBP s including quality assurance, data collection, evaluation, program improvement plans, staff retention and incentives, hiring/correction action/firing practices Set benchmarks for outcomes Identify sources for obtaining outcome data Determine need for outside evaluation Meet with funder to identify current funding amounts, restrictions, review reimbursement procedures and insure all costs are covered including training, licensing, travel etc. Determine if Federal funds or other funds could be used in the future for the intervention
Organizational & System Readiness (Months 2 4) Administrative Oversight Finalize /Order referral, reporting and information forms that will need to collected for the target population Draft initial policy and procedure manual Service Provider Oversight Contract for services from funder drafted along with securing supporting documentation (i.e. background screening, insurance certificates etc.) Assist with hiring Complete review of EBP readiness checklist Set up Monthly check in calls with administration for progress Referral/Utilization Management Identify POC for referral agencies Train referral agencies on referral procedures Stakeholder Management Train stakeholders on the model, expectations and benefits Establish outcome reporting process and content Model Fidelity Promotion Schedule initial model training Establish benchmarks for fidelity & potential monetary incentives Establish supervision procedures Establish monthly calls with model experts Data Collection & Evaluation Formalize data collection, benchmarks and reporting format Funding and Sustainability Guidance Finalize budgets and look for cost savings if able to bundle service costs Develop long term sustainability funding for the EBP
Implementation (Month 5and on) Administrative Oversight Ensure Provider is complying with Policy and Procedure Manual Revise Manual as needed Service Provider Oversight Referral/Utilization Management Stakeholder Management Monitor model compliance by randomly attending supervision and file review Meet monthly with Program Director and Agency administration to review implementation of EBP Track staff turnover Larger rollouts convene monthly Provider calls Monitor referral, appropriateness and utilization numbers monthly Meet with referral sources as needed Share outcome data as agreed upon frequency and content Participate in Community Meetings dealing with the relevant service delivery needs Model Fidelity Promotion Review and monitor fidelity reports and treatment pacing information Facilitate monthly calls with model experts Data Collection and Evaluation Funding and Sustainability Guidance Review monthly accuracy of data reported from demographics to dates of service Report on identified outcomes Meet with outside evaluator to ensure data set quarterly Monitor expense and make recommendations for overages or under utilization Pursue long term funding as needed
Importance of hiring right (the first time) PROBLEM: > 50% turnover and average 85 day vacancy rate identified in 2010 WHAT WE DID: Turnover analysis of every contracted agency for last 4 years (2007 2010) HR assessment of each agency s recruitment, hiring & retention practices Researched model requirements for staff competencies WHAT WE FOUND: Many therapists seemed unprepared for the EBP realities such as in home therapy, intense oversight & scrutiny, and being held accountable for outcomes Models had very specific requirements and expectations SOLUTIONS: In collaboration with the model experts and with the EBP program managers and team supervisors, we created a list of EBP Core Competencies based on the Alaskan Core Competencies as identified in a survey done by the Annapolis Coalition in 2010. We added several descriptors to each Core Competency In collaboration with the model experts and the agencies, we wrote a comprehensive model specific Hiring Manual o o Paradigm of screening in is flipped to screening out Initial Screening is designed to screen out Structured Interviewing with scoring is stressed RESULTS: 27% turnover and average 45 day vacancy rate achieved by the end of 2011
CORE COMPETENCIES FOR MASTERS LEVEL EBP'S with descriptors BASIC KNOWLEDGE ABOUT CLINICAL PRACTICES/THEORIES (i.e. CBT) DESIRE TO LEARN/SELF MOTIVATED EMOTIONAL INTELLIGENCE/ABILITY & TO ENGAGE/ GOOD COMMUNICATION SKILLS COPING SKILLS ANALYTICAL THINKER/ABILITY TO INTEGRATE PROFESSIONALISM CULTURAL COMPETENCY/DIVERSITY SENSITIVITY BASIC THERAPY SKILLS COGNITIVE/BEHAVIORAL APPROACH DESIRE FOR LIFE LONG LEARNING ABILITY TO ENGAGE SELF CARE/STRESS MANAGEMENT ABILITY TO SHIFT TO PROCESS CONSCIENTIOUSNESS TO DO UNSTRUCTURED WORK INDEPENDENTLY AND WELL. RELATE WITHOUT PATRONIZING ABILITY TO CLARIFY AWARENESS OF FAMILY SYSTEMS WILLINGNESS TO ACCEPT FEEDBACK AND ADJUST ACCORDINGLY BELIEF THAT FAMILIES ARE DOING THE BEST THEY CAN, GIVEN THE SKILLS THEY HAVE. ABILITY TO MANAGE SITUATIONS WITH FAMILIES AS BARRIERS ARE PRESENTED PROBE, NOT TAKE THINGS AT FACE VALUE RESPECT AND UNDERSTANDING FOR TEAMWORK AND COMPANY SYSTEMS. BEING COMFORTABLE WITH CULTURAL DIFFERENCES BEING BRIEF AND DIRECT *PRAGMATIC FAMILY THERAPIES EMBRACE TRAINING FAMILY AND CHILD ADVOCATE/ CHEERLEADER ABILITY TO MANAGE CRISIS SITUATIONS CALMLY CAN CONCEPTUALIZE AND TRANSLATE MENTAL MAPS/CONSTRUCTS FOR FAMILIES GOOD TEAM PLAYER NON JUDGMENTAL ABILITY TO REFRAME *SOCIAL ECOLOGICAL THEORY AMBITION TO DO THE BEST STRENGTH FOCUSED TAKING RESPONSIBILITY PRODUCING TIMELY AND USEFUL NOTES & DOCUMENTATION UP FOR THE CHALLENGE ABILITY TO "SELL" THERAPY IDEAS TO FAMILIES BEING DEPENDABLE & RELIABLE HAVING PROFESSIONAL WORKING RELATIONSHIP WITH EXTERNAL STAKEHOLDERS. USE SETBACKS TO GROW ABILITY TO EMPATHIZE ETHICAL AND TRUSTWORTHY BEING ORGANIZED COLOR CODES FINE TUNED LISTENING SKILLS *PATTERN DETECTION SKILLS GOOD TIME MANAGEMENT SKILLS SKILLS & KNOWLEDGEmeasured by Interview Questions SHOWS COMPASSIONATE AND TACTFUL CANDOR WHEN COMMUNICATING WITH FAMILY *LOGICAL THINKING ABILITY *HIGH LEVEL OF SELF AWARENESS BEHAVIOR/DRIVESmeasured by Behavioral Inventory testing USES HUMOR APPROPRIATELY (IN A NURTURING HEALING WAY) BOTH measured by Interview Questions and Behavioral Inventory testing Sources: Annapolis Study, BSFT, EBA, FFT, MST, Florida Redirections Providers *FLEXIBLE, CREATIVE, OPEN MINDED
100.0% Reduction Percentages by Grantee Reductions in STP Admissions & Felony Commitments from 2012 Baseline (EBA Counties in Green) 80.0% 60.0% 62.0% 84.4% 84.2% 79.6% 76.1% 75.6% 72.9% 70.1% 70.0% 68.6% 66.3% 65.3% 65.2% 63.6% 63.4% 61.5% 61.1% 60.7% 60.6% 58.9% 58.9% 55.3% 54.6% 47.8% 47.0% 40.0% 39.0% 20.0% 15% Reduction 18.2% 8.3% 0.0% 6.7% 15.2% 20.0%
40.0% 100.0% Reduction Percentages for EBA Related Grantees 100.0% Reduction Percentages for Non EBA Related Grantees 84.4% 84.2% 80.0% 60.0% 70.1% 70.0% 66.3% 63.6% 61.1% 65.3% Average 58.9% 55.3% 80.0% 60.0% 79.6% 76.1% 75.6% 72.9% 68.6% 65.3% 65.2% 63.4% 61.5% 60.7% 60.6% 58.9% 54.6% 47.8% 50.7% Average 40.0% 39.0% 40.0% 47.0% 20.0% 20.0% 18.2% 8.3% 0.0% 0.0% 6.7% 20.0% 20.0% 15.2%
njaner@ebanetwork.com mmcalister@ebanetwork.com