Pie Chart. Laszlo Thoth

Transcription

1 Pie Chart Laszlo Thoth

2 COA's Performance and Quality Improvement Standards Joe Frisino, MSW Council on Accreditation

3 Who is COA? Independent, private NFP Founded 1977 by CWLA and ACF Accredit full array of community-based behavioral health care and social service organizations More than 1500 private and public organizations accredited or in process Partnership w/ TACFS via NOSAC 3

4 What is Accreditation? System of accountability Self-regulation by the field Largely voluntary Evaluates structure 4

5 System of Accountability Licensing and Governmental Oversight Professional Credentialing Accreditation 5

6 Basics of Accreditation Systems National Consensus-Based Standards Self-Study Process On-Site Review by Independent and Professionally Trained Reviewers Fair, Objective, and Open Decision Making Opportunities to Correct Deficiencies 6

7 Characteristics of COA Accreditation Standards are Free Four-Year Accreditation Process takes Months Includes all Programs and Services Review all Aspects of Operations Person-centered Service Delivery Volunteer Peer Reviewers

8 Accreditation Process Timeline

9 Accreditation and Capacity Building 9

10 Capacity Building Capacity Building Efforts Increased Organizational Capacity Improved Performance & Effectiveness Light, P. (2004). Sustaining Nonprofit Performance. Washington, DC: Brookings Institution Press. 10

11 Capacity Building Literature Active, Representative Board Mission Clear and Understood Throughout the Organization Stakeholder Involvement Active in the Community Stable, Diversified Resources Strong Management and Operations 11

12 COA s Accreditation Logic Model 12

13 Performance and Quality Improvement

14 Stable Diversified Funding FIN 4 The organization pursues stable, predictable sources of revenue through diversification and balance in funding streams consistent with the organization's mission or purpose and programs.

15

16 Resource Marguerite Casey Foundation Organizational Capacity Assessment Tool 16

17 Standards 17

18 Insert graphic 01

19 COA's 8th Edition Standards Rigorous, Relevant, Realistic Developed by the field Goal statements -- don't have to be perfect Describe What not How

20 Management & Operations Standards Administrative and Service Environment Behavior Support and Management Client Rights Ethical Practice Financial Management Governance Human Resources Performance and Quality Improvement Risk Prevention and Management Training and Supervision

21 Service Standards: Common Elements Access / Outreach Assessment Service Planning Service Philosophy Service Elements Case Closing Aftercare and Follow-up Personnel

22 Structure & Nomenclature Purpose States the overall purpose or aim of the section. Core Concept State key service delivery concepts that support the Purpose. Practice Standards Contain detailed practices that operationalize the Core Concepts and Purpose. 22

23 Performance and Quality Improvement 23

24 PQI: What's in a Name? PQI, CQI, TQM, QI, QA... The standards accommodate the organization s preferred language and not to prescribe particular techniques 24

25 25

26

27 27

28 Where to Start? 28

29 Where to Start Create a process that is useful Start Where You Are Place program within existing efforts What are you already measuring? What are you current reporting obligations? What existing processes give management and supervisors information to make decisions/improve performance? What else would you like to know? What would like your stakeholders to know? 29

30 Use Available Tools Tools Index Worksheets, Templates, etc. Solutions Database 30

31 PQI Tools

32 1. Plan / Document PQI Bottom Line a. Includes all programs and sites 2. Involve Stakeholders 3. Measure: a. Management/Operations Performance b. Client Outcomes/Outputs c. Program Results /Service Quality d. Client and Staff Satisfaction e. Risk Management Reports f. Case Record Review 4. Aggregate Data Reports 5. Evidence that results were reviewed 6. Evidence that something was done with the data 32

33 Culture of Improvement 33

34 PQI 1: Culture and Leadership The organization s leadership promotes a culture that values service quality and ongoing efforts by the full organization, its partners, and contractors to achieve strong performance, program goals, and positive results for service recipients. 34

35 Culture and Leadership 35

36 PQI 1 Narrative Question #1 Explain how your organization defines and represents a culture that values quality, including: a. how strategic priorities influence the quality improvement process; b. how the CEO/Executive Director and senior managers promote a culture of service delivery excellence, customer satisfaction and continual improvement; and c. fiscal and staff resources devoted to PQI. 36

37 Planning 37

38 Does your organization currently have a quality improvement plan? No Minimal Pretty Good Comprehensive 38

39 PQI 2: Planning and Infrastructure The infrastructure that supports performance and quality improvement is: sufficient to identify organization-wide issues implement solutions that improve overall efficiency promote accessible, effective services in all regions and sites. 39

40 PQI 2 40

41 Purpose of the PQI Document Provide key information to leadership Monitor progress toward achieving strategic goals Management and operations Program data Pull all efforts together Clarify and focus on overall organizational performance Make connections and see patterns that might otherwise be obscured 41

42 Plan Outline I. Introduction A. PQI Philosophy B. Roles and Responsibilities C. Logic Model or Process Description D. Stakeholders II. Outcomes and Measures A. Long term/ Strategic Goals and Objectives B. Management/Operational Performance C. Program/Service Delivery Effectiveness D. Client Outcomes III. Operational Procedures A. Who, What, When, Where, How 42

43 PQI Structure 43

44 Foster Care Program Committee DD Services 44

45 PQI Committees Determine the committee s mandate How much decision-making power will be granted? Oversee /guide PQI activities or advisory Determine Structure Steering committee model: more inclusive of stakeholders Independent Management team Who will sit on committees Process Issues Review raw data or summaries? Which data?

46 PQI Coordinator Keep the process moving forward Stay two steps ahead Keep your eye on milestones, deadlines, and due dates Motivate staff Acknowledge and reward work and accomplishments Guide committee

47 Involving Stakeholders 47

48 November 11

49 What is a Stakeholder? Your stakeholders are all the people who have an interest or stake in your agency s success at achieving its mission or purpose. Adapted from the Fieldstone Alliance

50 Stakeholders Who are your key stakeholders?

51 Why Involve Stakeholders? Hear the voices of people directly impacted by your services Strong support in the literature Builds on the strengths of the community Strengthens your organization's connections to its community

52 Where Do We Start? Invite their participation and build on existing relationships Are any stakeholders currently involved? How? How might potential representatives be approached? Anyone in the org have connections? What information will they need to participate? For stakeholders with an ongoing role what is the time commitment?

53 Ways to Involve External Stakeholders Sit on PQI committee Participate in ad hoc or ongoing work groups Provide community input on program goals and client outcomes Review reports and provide feedback Planning input Stakeholder advisory committee focusing on community issues Surveys / Focus Groups

54 Ways to Involve Internal Stakeholders Sit on PQI committee -- could rotate staff Sit on work groups or sub-committees Help identify outcomes / tools etc. Administer tools and collect data Read results and provide feedback Case record review -- could rotate staff Planning: short- and long-term Help define mission and values

55 Evidence of Implementation Meaningful participation Proactively seek input Use written agendas! Take minutes!

56 What to Measure? 56

57 PQI 3: What to Measure? An inclusive approach to establishing measured performance goals, client outcomes, indicators, and sources of data ensures broad-based support for useful performance and outcomes measurement. 57

58 PQI 3 58

59 Start Where You Are What Are You Already Measuring? Include current state / funder reporting requirements: Program outputs Client outcomes CAFAS scores United Way CFSR Evaluate the extent to which current reporting covers standards expectations 59

60 Monitoring v. Measuring Routine information related to operations E.g., risk management information Program Outputs / Management Performance Data Quantitative # units of service delivered # of trainings per quarter # of clients served per month Outcomes Data Qualitative: What did the client gain as result of having used the service?

61 COA Expectations Management/Operations Performance Client Outcomes/Outputs Program Results /Service Quality Client and Staff Satisfaction Risk Management Reports Case Record Review 61

62 PQI 3.03: Three Performance Levels 62

63 Strategic Goals and Objectives GOV 6.03 c. Measurable goals and objectives d. Appropriate strategies for meeting identified goals Increase Community Awareness of Programs & Services Diversify Governing Body Diversify Funding November 11 Council on Accreditation 2008 All Rights Reserved

64 Management and Operations Performance Comparative Unit Costs Analysis Compare costs between different services, by geography, between similar programs Data sources: FIN 5.04, FIN 5.05 Workforce Stability Compare staff turnover rate by service, job category, education/experience Link to recruitment and training costs Data sources: HR 4.03, HR 4.04, HR 5.03 Safe Service Environment Compare between programs and overall, total incident and by type Facilities, accidents, injuries, out-of-control behavior, complaints Possible data sources: ASE 3.02, ASE 4, ASE 5, ASE 6.02 November 11 Council on Accreditation 2005 All Rights Reserved

65 Public Agencies: System-Level Performance System-wide integration of agency operations (PA-AM 2.05) Clear communication throughout service provider system including field offices (PA-AM 2.06) Implementation of uniform management procedures (PA-AM 2.05) Integration and coordination of service provision including ease of access to services (PA-AM 3.02, PA-AM 3.03) Integration of performance and outcomes data (PA-AM 2.05) Common program and client outcomes (PA-AM 3.04)

66 Program and Service Delivery Performance PQI 4.01 Collection of service delivery information focuses on key quality factors, including: a. appropriateness; b. effectiveness; and c. any or all of the dimensions of quality.

67 Program and Service Delivery Performance PQI 4.01 Collection of service delivery information focuses on key quality factors, including: a. appropriateness; b. effectiveness; and c. any or all of the dimensions of quality. Widely accepted dimensions of service quality include: accessibility availability efficiency continuity safety timeliness respectfulness Service delivery indicators influencing program results could include: timeliness and comprehensiveness of individualized assessments family involvement client satisfaction staff training November 11 Council on Accreditation 2008 All Rights Reserved

68 Client Outcomes Change in clinical status Change in functional status Health, welfare, and safety Permanency of life situation Quality of life 68

69 Client Outcomes GLS 4.03 The service plan is based on the assessment and includes: a. agreed upon goals, desired outcomes, and timeframes for achieving them

70 Collecting and Analyzing Data 70

71 PQI 4: Data Collection and Analysis 71

72 PQI 4.02 The organization aggregates and reviews several sources of information to identify patterns and trends, including: a. quarterly file review reports; b. quarterly review of incidents, accidents, and grievances; c. customer satisfaction data, usually annually; d. customer outcomes data, usually annually; and e. management and operations data and reports. 72

73 Define the Data Collection Process Formal detailed procedures who, what, when, where, how: Deciding what to measure Identifying specific outcomes, outputs, indicators, tools, etc. Obtaining data/information Reviewing data and making changes Communicating results 73

74 Case Record Reviews Quarterly Review presence, clarity, quality and continuity of required documents See RPM 7 Quality Issues Sampling Reviewers November 11 Council on Accreditation 2005 All Rights Reserved

75 Sample Size Free Online Sample Size Calculator Select enough records to: Identify patterns Justify making changes ors/samplesize.cfm

76 Data Reports Tailor to the Audience Staff, Board, Clients, Community Frequency Make Reports User-friendly Don t overwhelm them with data Strive for Clarity Provide Context and Interpretation

77 Breaking Data Out for Analysis Time Trends Compare recent data with data from previous periods Compare Actual Outcomes to Targets Compare Data by Client Characteristics Age/Gender/Race/Education Compare Data Between Similar Programs Compare Data with Results from Similar Programs in Other Organizations The Urban Institute (2004). Using Outcomes Information: Making Data Pay Off. Free download:

78 Sample Comparison Over Time Urban Institute(2004). Using Outcomes Information

79

80 Using PQI Data 80

81 PQI 5: Use and Communication Findings based on improvement efforts are disseminated to personnel and stakeholders and are used to improve programs and practice. 81

82 PQI 5: Primary Activities Review results Identify areas in need of improvement Implement and evaluate improvements Modify implemented improvements as needed Keep staff informed and involved

83 PQI 5: Use and Communication 83

84 Using Performance Data Tell Your Story to Funders, Community Identify Practices Needing Improvement Process Improvement Identify / Replicate Successful Practices Build on Strengths Identify Training Needs Recognize and Motivate Programs, Staff, and Volunteers Use Information for Making Decisions The Urban Institute (2004). Using Outcomes Information: Making Data Pay. Free download:

85

86 Unusual or Unexpected Results Don t Jump to Conclusions Internal Factors Problems with program design Problems with program implementation Staff turnover Changes in strategic direction External Factors Changing conditions Changes in client population/characteristics Unexpected funding, staff reductions Changes in social, political, regulatory environment

87 Support 87

88 PQI 6: Staff and Stakeholder Support Staff and stakeholders receive information and support that increases their capacity to participate in, conduct, and sustain performance and quality improvement activities 88

89 PQI 6: Primary Activities Inform and educate stakeholders about your organization's PQI program Orient and educate staff per the expectations of their role and responsibilities Manager and supervisors maintain organizational focus on PQI

90 PQI 6: Staff and Stakeholder Support 90

91 Resources 91

92 Center for Learning and Performance Technologies Resources Hundreds of free and inexpensive applications The Urban Institute Outcomes Indicators Project Website The Center for What Works: Performance Measurement Toolkit Building a Performance Measurement System: A How-To Guide Root Cause: The United Way: Outcomes Measurement Resource Network Benchmarking for Nonprofits by Jason Paul Fieldstone Alliance. American Society for Quality The California Evidence-Based Clearinghouse for Child Welfare

93 Resources TechSoup Large discounts on software Microsoft Office Suite for $18...usually $450 Microsoft MapPoint for $12...usually $299 Many useful articles and webinars Idealware Provides candid Consumer-Reports-style reviews and articles about software of interest to nonprofits A Consumers Guide to Low-Cost Data Visualization Tools A Few Good Online Survey Tools

94 Free or Cheap Resources Data Analysis / Spreadsheets SOFA Statistics ChartGo Google Docs Zoho Sheet Sheet. com StatCrunch Survey Tools Survey Monkey Zoomerang Idealware: A Few Good Online Survey Tools Includes list of advanced online survey tools

95 Free or Cheap Resources Free Software Sites Jane's E-Learning Tools Directory Gizmo's Freeware

96 Council on Accreditation 120 Wall Street, 11 th Floor New York, NY telephone