Practical implications of variation on service delivery in the community sector

Transcription

1 Practical implications of variation on service delivery in the community sector The Use of Traditional Quality Measurement Tools to Promote Positive Change

2 DRIVERS Amazing work done by people working in the front line of NFP s ACOSS Poverty Report or 17.7% of kids in Australia live in poverty Marginalised Australians: Characteristics and Predictors of Exit over Ten Years University of Canberra, Helen Berry and others. 13% of Australians marginalised 3 million

3 Personal Experience Vinnies Clayton Church Homes Strathmont Centre Queen Elizabeth Hospital Lyell McEwin Diabetes Centre Uniting Care Wesley Northern Adelaide Community Health City of Marion Work Cover Career Systems Meals on Wheels Kids Future Kids City of Charles Sturt Legal Services SA TQEH Palliative Care OARS Blackwood Community Hospital Prospect Council Lutheran Community Care

4 Measuring the Outcomes of Community Organisations Australian Research Alliance for Children and Youth 2009, Page xi The complex system that is the Australian community sector poses a number of barriers to measuring outcomes. There are a varied range of client programs and services influencing outcome measurement and achievement A range of complex social issues is being addressed The system is in a constant state of change There is a culture of non-measurement within many COs which reduces their motivation and capacity to measure outcomes and other organisation issues such as size and specificity of focus The evidence base about what works for the Australian community sector is lacking Many outcomes are only evidenced in the long term The links between intervention and outcome and impact are not always straightforward or definitive

5 Current Measurement in the Community Sector Input Output focused Objective focused Social Return on Investment Results Based Accountability Logic Models Program Logic Approaches Realist Evaluation

6 The Question What is being done to help people at the front line use measurement tools to build for improvement?

7 The Problem as I See It The front line worker is more often than not passionate and dedicated to their clients They believe they don t have time to sit back and reflect when there is a need to meet the next crisis They don t recognise the strength of the data they already collect

8 Further Learning Options Example The Data Set in your Hands Student Week 1 Week 2 Week 3 Week 4 AA 0,0,1,1,0 1,0,0,0,0 1,0,0,0,0 0,0,0,1,0 5 AB 1,1,1,1,0 1,1,1,1,1 1,1,1,0,1 1,1,0,1,1 17 AC 0,0,0,1,0 0,0,0,0,1 1,0,0,0,0 0,0,1,0,0 4 AD 0,0,0,0,0 0,0,0,0,0 0,0,0,0,0 0,0,0,0,0 0 AE 1,0,0,0,1 1,0,1,1,1 1,1,1,1,0 1,1,0,1,1 14 AF 1,0,1,0,1 0,1,1,0,1 0,0,1,0,0 0,1,0,0,0 8 AG 0,0,0,0,1 0,0,1,1,0 1,0,0,0,` 0,0,0,1,0 6 AH 0,1,0,0,0 0,1,0,1,1 1,0,1,1,1 0,0,1,1,1 11 AI 1,0,0,0,0 1,0,1,0,0 1,1,1,0,1 0,1,1,0,0 9 AJ 0,0,1,0,1 0,1,0,1,0 0,0,1,0,1 0,1,0,1,0 8 AK 1,0,0,0,0 1,0,1,0,0 0,1,0,0,0 1,0,0,0,0 5 AL 1,0,0,0,0 1,0,1,0,1 0,0,1,0,1 0,0,1,0,1 8 AM 0,0,0,0,0 0,0,0,0,0 0,1,0,0,0 0,0,0,0,0 1 AN 0,1,1,0,1 1,0,0,0,0 1,1,1,1,1 1,0,1,1,1 13 AO 1,1,1,1,1 1,1,0,1,1 1,1,1,1,1 1,1,1,1,1 19 AP 0,1,0,0,0 0,0,0,0,0 0,0,0,1,0 0,0,0,0,0 2 AQ 0,0,0,0,0 0,0,0,0,0 0,0,0,0,0 0,0,0,0,0 0 AR 11 0,0,0,1,0 1,1,1,1,0 0,1,1,0,1 1,1,0,1 AS 10 AT 2 AU 9 AV 5 AW 6 AX 6 AY 5 AZ 2 BA 2 BB 18 BC 3 BG 5 BE 9 BF 3 BG 4 BH 9 BI 4 Totals 243 1,1,0,0,1 0,0,1,1,1 0,0,1,0,0 1,0,1,0,1 1,0,0,0,0 0,0,0,0,0 0,1,0,0,0 0,0,0,0,0 0,1,0,1,1 0,1,1,0,0 1,0,0,1,0 0,0,1,0,1 0,0,1,1,0 1,0,0,0,0 0,1,0,1,0 0,0,0,0,0 0,1,0,1,0 1,0,1,0,0 1,0,0,0,0 0,0,0,0,1 1,0,0,0,1 0,0,0,0,1 0,0,1,1,0 0,0,0,0,1 0,0,1,0,0 1,1,0,1,0 0,0,0,1,0 0,0,0,0,0 0,0,0,0,1 0,0,0,0,0 0,1,0,0,0 0,0,0,0,0 1,0,0,0,0 0,0,0,0,0 0,0,0,1,0 0,0,0,0,0 0,1,0,1,1 1,1,1,1,1 1,1,1,1,1 1,1,1,1,1 0,0,0,0,1 0,1,1,0,0 0,0,0,0,0 0,0,0,0,0 0,0,0,0,0 1,0,0,1,0 0,0,1,1,0 0,0,0,0,1 1,0,0,0,1 0,0,1,0,0 0,0,1,1,1 1,0,1,0,1 0,1,0,0,1 0,0,0,0,0 0,0,0,1,0 0,0,0,0,0 1,0,0,0,0 1,0,0,0,0 0,0,0,1,0 0,0,0,0,1 0,0,0,1,1 0,0,0,0,0 1,1,1,1,0 1,0,0,1,1 0,0,0,0,0 0,1,1,0,0 1,0,0,0,0 0,0,0,1,0 13,10,8,10, 15 15,10,15,1 0,12 14,13,16,1 6,11 9,8,11,11,1 6

9 First Step Attendance figures: For 35 enrolled teenagers week Monday Tuesday Wednesday Thursday Friday

10 Histogram Start of Histogram of Attendance over 20 days for 35 students X X X X Attendance in Days

11 Histogram Completed Histogram of Attendance over 20 days for 35 students X 4 X X X 3 X X X X X X 2 X X X X X X X X X 1 X X X X X X X X X X X X X X X X Attendance in Days

12 Graph Paper Version Histogram

13 Why Graph Paper Version? It is simple and quick Provides ownership of the data Results are believable

14 Graph Paper Run Chart

15 Run Chart for Daily Attendance: Average is 12.2 students per day; Range is from 8 students to 16 students per day Average 10 8 Series

16 Control(?) Chart for Daily Attendance 1 st 4 Weeks I Prefer in Community Services to call it a Capability or Prediction Chart 18 Upper Control Limit Average Series 1 Lower Control Limit The Capability Chart provides a base line for measures of improvement

17 Test for the Hawthorne Effect 1927 Hawthorne Plant Western Electric Professor Elton Mayo Harvard Business School No matter what was done productivity increased Workers Responding to the knowledge of being observed Responding to Management taking an interest in them

18 4 Weeks of Student Involvement in Graphing Attendance Average increased to 13.25

19 Staffing at the FLO 2.0 FTE Support workers 3.0 FTE Case Managers

20 Time Spent with Top 3 Students Support Worker A: hours per week Support Worker B: 8 10 hours per week Total of 60 hours contact hours per week; 30% to 42% What would happen if the next 12 students were given more time and the top 3 students told they were being advanced to more responsible self study to prepare them for Re-entry into formal education? Support Staff decided to trial more time with the next 4 students

21 Graph Paper Version Histogram

22 Histogram on Graph Paper

23 New Run Chart after Increasing Time with 4 students

24 Further Issues Discussion between support staff and case managers about working more closely together Increase in marking and feedback load Demands from other students to be included in the approach that they could see was working

25 Hip Hop Positive Role Models for the kids Fits in with healthy lifestyle break dancing is strenuous Hip Hop artists constantly write down on paper ideas and feelings they can use in their songs A great way for kids with low literacy to be motivated to learn Used Force field analysis to plan this process

26 Other Measurement Areas for the FLO Number and Quantity of Assignments handed in? Self Esteem? Relationship with Parents? Conflict resolution? What else have they learnt?

27 Other Uses for Capability Charts Work Health and Safety Does your organisation operate a stable system of work health and safety? Does HR use Control Charts to determine where to aim efforts at improvement? Does HR show clear improvements in WH&S incidents?

28 Definition of Measurement A set of observations that reduce uncertainty where the result is expressed as a measure

29 4 Useful Measurement Assumptions Your problem is not as unique as you think You have more data than you think You need less data than you think There is a useful measurement that is simpler than you think

30 And the Big Picture! Connections for Organisational Excellence Long Range Plan Alignment Skills Motivation Resources Short Range Plans Achievement Alignment Skills Motivation Resources Short Range Plans Confusion Long Range Plan Skills Motivation Resources Short Range Plans Sabotage Long Range Plan Alignment Motivation Resources Short Range Plans Anxiety Long Range Plan Alignment Skills Resources Short Range Plans Resistance Long Range Plan Alignment Skills Motivation Short Range Plans Frustration Long Range Plan Alignment Skills Motivation Resources Disengaged Adapted from: T. Knoster Adapted by T. Knoster from Enterprise Group Ltd